Eth0 not working (ClearFog / LX2160)

womble · May 12, 2022, 4:50pm

I’m having no luck using the copper or SFP interfaces on my new ClearFog board.

I’ve re-flashed my MMC to the latest image (lx2160acex7_2000_700_3200_8_5_2-ddab3ad.img.xz) downloaded from the Solid-Run images repository, which was built on 11th of May. I am booting from an SD card flashed with that image with the rootfs on MMC.

The ClearFog’s RJ45 Ethernet connector is connected to our LAN, however this does not appear to be working from within Linux (as it used to with previous images) and I am unable to communicate using it.

I have also tried using SFPs in the 4xSFP+ cage with similar results.

Any ideas on potential causes and fixes would be appreciated.

Womble

If I interrupt u-boot during the boot process, I am able to get an IP address from the DHCP server over that interface. This demonstrates that the interface is correctly connected to the LAN

=> dhcp
DPMAC17@rgmii-id Waiting for PHY auto negotiation to complete.. done
BOOTP broadcast 1
BOOTP broadcast 2
BOOTP broadcast 3
DHCP client bound to address 192.168.1.226 (1015 ms)

following boot into Linux, the eth0 link is present:

root@localhost:~# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/sit 0.0.0.0 brd 0.0.0.0
**3: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000**
link/ether d0:63:b4:03:29:56 brd ff:ff:ff:ff:ff:ff

However, bringing the link up and attempting to obtain an address via DHCP results in no received packets and no IP address. Manually configuring an address on the local subnet doesn’t work either

root@localhost:~# ip link set dev eth0 up                                                                                                   
root@localhost:~# ip link show dev eth0 
3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether d0:63:b4:03:29:56 brd ff:ff:ff:ff:ff:ff
root@localhost:~# ip -s link show dev eth0                                                                                                        
3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether d0:63:b4:03:29:56 brd ff:ff:ff:ff:ff:ff
    RX: bytes  packets  errors  dropped overrun mcast   
    0          0        0       0       0       0       
    TX: bytes  packets  errors  dropped carrier collsns 
    90         1        0       0       0       0       
root@localhost:~# dhclient eth0 
^C
root@localhost:~# ip -s link show dev eth0                                                                                                        
3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether d0:63:b4:03:29:56 brd ff:ff:ff:ff:ff:ff
    RX: bytes  packets  errors  dropped overrun mcast   
    0          0        0       0       0       0       
    TX: bytes  packets  errors  dropped carrier collsns 
    774        3        0       0       0       0   

root@localhost:~# ip addr add dev eth0 192.168.1.226/24                                                                                           

root@localhost:~# ip addr show dev eth0                                                                                                           
3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether d0:63:b4:03:29:56 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.226/24 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::d263:b4ff:fe03:2956/64 scope link 
       valid_lft forever preferred_lft forever
root@localhost:~# ping 192.168.1.1
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
^C
--- 192.168.1.1 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1006ms

** dmesg output (trimmed to fit, retaining networking stuff):**

[    0.683710] 21d0000.serial: ttyAMA1 at MMIO 0x21d0000 (irq = 26, base_baud = 0) is a SBSA
[    0.694265] fsl_mc_bus 80c000000.fsl-mc: MC firmware version: 10.28.1
[    0.704244] fsl_mc_dprc dprc.1: DMA mask not set
[    0.721872] HugeTLB registered 1.00 GiB page size, pre-allocated 2 pages
[    0.728578] HugeTLB registered 32.0 MiB page size, pre-allocated 0 pages
[    0.735269] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.741960] HugeTLB registered 64.0 KiB page size, pre-allocated 0 pages
[    0.749518] cryptd: max_cpu_qlen set to 1000
[    1.910337] raid6: using algorithm neonx8 gen() 6041 MB/s
[    1.915725] raid6: .... xor() 4156 MB/s, rmw enabled
[    1.920679] raid6: using neon recovery algorithm
[    1.925551] ACPI: Interpreter disabled.
[    1.930213] iommu: Default domain type: Passthrough 
[    1.994546] mc: Linux media interface: v0.10
[    2.018416] PTP clock support registered
[    2.022378] EDAC MC: Ver: 3.0.0
[    2.026044] No BMan portals available!
[    2.029951] QMan: Allocated lookup table at (____ptrval____), entry count 65537
[    2.037361] No QMan portals available!
[    2.041256] No USDPAA memory, no 'fsl,usdpaa-mem' in device-tree
[    2.047502] FPGA manager framework
[    2.050927] Advanced Linux Sound Architecture Driver Initialized.
[    2.057608] clocksource: Switched to clocksource arch_sys_counter
[    2.063791] VFS: Disk quotas dquot_6.6.0
[    2.067735] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    2.074684] pnp: PnP ACPI: disabled
[    2.081070] NET: Registered protocol family 2
[    2.085700] tcp_listen_portaddr_hash hash table entries: 8192 (order: 5, 131072 bytes, linear)
[    2.094403] TCP established hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[    2.102887] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes, linear)
[    2.110683] TCP: Hash tables configured (established 131072 bind 65536)
[    2.117380] UDP hash table entries: 8192 (order: 6, 262144 bytes, linear)
[    2.124318] UDP-Lite hash table entries: 8192 (order: 6, 262144 bytes, linear)
[    2.131766] NET: Registered protocol family 1
[    2.136301] RPC: Registered named UNIX socket transport module.
[    2.142226] RPC: Registered udp transport module.
[    2.146920] RPC: Registered tcp transport module.
[    2.151616] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    2.158049] PCI: CLS 0 bytes, default 64
[    2.162810] hw perfevents: enabled with armv8_cortex_a72 PMU driver, 7 counters available
[    2.171133] kvm [1]: IPA Size Limit: 44 bits
[    2.176019] kvm [1]: vgic-v2@c0e0000
[    2.179600] kvm [1]: GIC system register CPU interface enabled
[    2.185622] kvm [1]: vgic interrupt IRQ9
[    2.189733] kvm [1]: Hyp mode initialized successfully
[    2.196869] workingset: timestamp_bits=42 max_order=22 bucket_order=0
[    2.203727] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    2.209698] NFS: Registering the id_resolver key type
[    2.214751] Key type id_resolver registered
[    2.218925] Key type id_legacy registered
[    2.222940] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    2.229633] nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering...
[    2.237026] jffs2: version 2.2. (NAND) �© 2001-2006 Red Hat, Inc.
[    2.243219] fuse: init (API version 7.32)
[    2.272493] NET: Registered protocol family 38
[    2.276935] xor: measuring software checksum speed
[    2.282817]    8regs           :  8958 MB/sec
[    2.288248]    32regs          :  9065 MB/sec
[    2.293720]    arm64_neon      :  8759 MB/sec
[    2.298067] xor: using function: 32regs (9065 MB/sec)
[    2.303123] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 243)
[    2.310560] io scheduler mq-deadline registered
[    2.315084] io scheduler kyber registered
[    2.320866] serdes-28g 1ea0000.serdes_phy: PLLF: disabled, unlocked
[    2.327130] serdes-28g 1ea0000.serdes_phy:   Reference clock: 100MHz
[    2.333387] serdes-28g 1ea0000.serdes_phy:   clock net: 5GHz
[    2.338949] serdes-28g 1ea0000.serdes_phy:   Supported interfaces:
[    2.345032] serdes-28g 1ea0000.serdes_phy:           sgmii
[    2.349814] serdes-28g 1ea0000.serdes_phy:           qsgmii
[    2.354681] serdes-28g 1ea0000.serdes_phy:           1000base-x
[    2.359897] serdes-28g 1ea0000.serdes_phy: PLLS: enabled, locked
[    2.365894] serdes-28g 1ea0000.serdes_phy:   Reference clock: 161.1328125MHz
[    2.372844] serdes-28g 1ea0000.serdes_phy:   clock net: 10.3125GHz
[    2.378927] serdes-28g 1ea0000.serdes_phy:   Supported interfaces:
[    2.385010] serdes-28g 1ea0000.serdes_phy:           10gbase-r
[    2.390137] serdes-28g 1ea0000.serdes_phy:           usxgmii
[    2.395128] serdes-28g 1ea0000.serdes_phy: created PHY for lane #0
[    2.401326] serdes-28g 1ea0000.serdes_phy: created PHY for lane #1
[    2.407522] serdes-28g 1ea0000.serdes_phy: created PHY for lane #2
[    2.413718] serdes-28g 1ea0000.serdes_phy: created PHY for lane #3
[    2.419914] serdes-28g 1ea0000.serdes_phy: created PHY for lane #4
[    2.426108] serdes-28g 1ea0000.serdes_phy: created PHY for lane #5
[    2.432310] serdes-28g 1ea0000.serdes_phy: created PHY for lane #6
[    2.438507] serdes-28g 1ea0000.serdes_phy: created PHY for lane #7
[    2.444719] serdes-28g 1eb0000.serdes_phy: PLLF: enabled, locked
[    2.450718] serdes-28g 1eb0000.serdes_phy:   Reference clock: 100MHz
[    2.456976] serdes-28g 1eb0000.serdes_phy:   clock net: 5GHz
[    2.462539] serdes-28g 1eb0000.serdes_phy:   Supported interfaces:
[    2.468622] serdes-28g 1eb0000.serdes_phy:           sgmii
[    2.473403] serdes-28g 1eb0000.serdes_phy:           qsgmii
[    2.478271] serdes-28g 1eb0000.serdes_phy:           1000base-x
[    2.483487] serdes-28g 1eb0000.serdes_phy: PLLS: enabled, locked
[    2.489483] serdes-28g 1eb0000.serdes_phy:   Reference clock: 100MHz
[    2.495741] serdes-28g 1eb0000.serdes_phy:   Supported interfaces:
[    2.501851] serdes-28g 1eb0000.serdes_phy: created PHY for lane #0
[    2.508046] serdes-28g 1eb0000.serdes_phy: created PHY for lane #1
[    2.514243] serdes-28g 1eb0000.serdes_phy: created PHY for lane #2
[    2.520438] serdes-28g 1eb0000.serdes_phy: created PHY for lane #3
[    2.526636] serdes-28g 1eb0000.serdes_phy: created PHY for lane #4
[    2.532832] serdes-28g 1eb0000.serdes_phy: created PHY for lane #5
[    2.539027] serdes-28g 1eb0000.serdes_phy: created PHY for lane #6
[    2.545223] serdes-28g 1eb0000.serdes_phy: created PHY for lane #7
[    2.554437] layerscape-pcie 3600000.pcie: host bridge /soc/pcie@3600000 ranges:
[    2.561750] layerscape-pcie 3600000.pcie:      MEM 0x9040000000..0x907fffffff -> 0x0040000000
[    2.570349] layerscape-pcie 3600000.pcie: PCI host bridge to bus 0000:00
[    2.577045] pci_bus 0000:00: root bus resource [bus 00-ff]
[    2.582523] pci_bus 0000:00: root bus resource [mem 0x9040000000-0x907fffffff] (bus address [0x40000000-0x7fffffff])
[    2.593052] pci 0000:00:00.0: [1957:8d80] type 01 class 0x060400
[    2.599110] pci 0000:00:00.0: supports D1 D2
[    2.603371] pci 0000:00:00.0: PME# supported from D0 D1 D2 D3hot
[    2.611044] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[    2.616275] pci 0000:00:00.0: Max Payload Size set to  256/ 256 (was  128), Max Read Rq  256
[    2.624813] layerscape-pcie 3800000.pcie: host bridge /soc/pcie@3800000 ranges:
[    2.632137] layerscape-pcie 3800000.pcie:      MEM 0xa040000000..0xa07fffffff -> 0x0040000000
[    2.640740] layerscape-pcie 3800000.pcie: PCI host bridge to bus 0001:00
[    2.647435] pci_bus 0001:00: root bus resource [bus 00-ff]
[    2.652913] pci_bus 0001:00: root bus resource [mem 0xa040000000-0xa07fffffff] (bus address [0x40000000-0x7fffffff])
[    2.663438] pci 0001:00:00.0: [1957:8d80] type 01 class 0x060400
[    2.669509] pci 0001:00:00.0: supports D1 D2
[    2.673771] pci 0001:00:00.0: PME# supported from D0 D1 D2 D3hot
[    2.681399] pci 0001:00:00.0: PCI bridge to [bus 01-ff]
[    2.686633] pci 0001:00:00.0: Max Payload Size set to  256/ 256 (was  128), Max Read Rq  256
[    2.695645] EINJ: ACPI disabled.
[    2.702211] Bus freq driver module loaded
[    2.709397] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    2.716602] SuperH (H)SCI(F) driver initialized
[    2.721350] msm_serial: driver initialized
[    2.726268] arm-smmu 5000000.iommu: probing hardware configuration...
[    2.732704] arm-smmu 5000000.iommu: SMMUv2 with:
[    2.737312] arm-smmu 5000000.iommu:  stage 1 translation
[    2.742616] arm-smmu 5000000.iommu:  stage 2 translation
[    2.747918] arm-smmu 5000000.iommu:  nested translation
[    2.753134] arm-smmu 5000000.iommu:  coherent table walk
[    2.758436] arm-smmu 5000000.iommu:  (IDR0.CTTW overridden by FW configuration)
[    2.765737] arm-smmu 5000000.iommu:  stream matching with 128 register groups
[    2.772864] arm-smmu 5000000.iommu:  64 context banks (0 stage-2 only)
[    2.779384] arm-smmu 5000000.iommu:  Supported page sizes: 0x61311000
[    2.785816] arm-smmu 5000000.iommu:  Stage-1: 48-bit VA -> 48-bit IPA
[    2.792247] arm-smmu 5000000.iommu:  Stage-2: 48-bit IPA -> 48-bit PA
[    2.804630] brd: module loaded
[    2.812019] loop: module loaded
[    2.815296] pci-endpoint-test 0000:00:00.0: Adding to iommu group 0
[    2.821666] pci-endpoint-test 0001:00:00.0: Adding to iommu group 1
[    2.828375] megasas: 07.714.04.00-rc1
[    2.832632] imx ahci driver is registered.
[    2.836895] ahci-qoriq 3200000.sata: supply ahci not found, using dummy regulator
[    2.844417] ahci-qoriq 3200000.sata: supply phy not found, using dummy regulator
[    2.851825] ahci-qoriq 3200000.sata: supply target not found, using dummy regulator
[    2.859532] ahci-qoriq 3200000.sata: AHCI 0001.0301 32 slots 1 ports 6 Gbps 0x1 impl platform mode
[    2.868484] ahci-qoriq 3200000.sata: flags: 64bit ncq sntf pm clo only pmp fbs pio slum part ccc sds apst 
[    2.878575] scsi host0: ahci-qoriq
[    2.882109] ata1: SATA max UDMA/133 mmio [mem 0x03200000-0x0320ffff] port 0x100 irq 33
[    2.890076] ahci-qoriq 3210000.sata: supply ahci not found, using dummy regulator
[    2.897595] ahci-qoriq 3210000.sata: supply phy not found, using dummy regulator
[    2.905002] ahci-qoriq 3210000.sata: supply target not found, using dummy regulator
[    2.912696] ahci-qoriq 3210000.sata: AHCI 0001.0301 32 slots 1 ports 6 Gbps 0x1 impl platform mode
[    2.921648] ahci-qoriq 3210000.sata: flags: 64bit ncq sntf pm clo only pmp fbs pio slum part ccc sds apst 
[    2.931555] scsi host1: ahci-qoriq
[    2.935022] ata2: SATA max UDMA/133 mmio [mem 0x03210000-0x0321ffff] port 0x100 irq 34
[    2.942982] ahci-qoriq 3220000.sata: supply ahci not found, using dummy regulator
[    2.950496] ahci-qoriq 3220000.sata: supply phy not found, using dummy regulator
[    2.957904] ahci-qoriq 3220000.sata: supply target not found, using dummy regulator
[    2.965597] ahci-qoriq 3220000.sata: AHCI 0001.0301 32 slots 1 ports 6 Gbps 0x1 impl platform mode
[    2.974549] ahci-qoriq 3220000.sata: flags: 64bit ncq sntf pm clo only pmp fbs pio slum part ccc sds apst 
[    2.984471] scsi host2: ahci-qoriq
[    2.987937] ata3: SATA max UDMA/133 mmio [mem 0x03220000-0x0322ffff] port 0x100 irq 35
[    2.995895] ahci-qoriq 3230000.sata: supply ahci not found, using dummy regulator
[    3.003412] ahci-qoriq 3230000.sata: supply phy not found, using dummy regulator
[    3.010821] ahci-qoriq 3230000.sata: supply target not found, using dummy regulator
[    3.018515] ahci-qoriq 3230000.sata: AHCI 0001.0301 32 slots 1 ports 6 Gbps 0x1 impl platform mode
[    3.027467] ahci-qoriq 3230000.sata: flags: 64bit ncq sntf pm clo only pmp fbs pio slum part ccc sds apst 
[    3.037375] scsi host3: ahci-qoriq
[    3.040841] ata4: SATA max UDMA/133 mmio [mem 0x03230000-0x0323ffff] port 0x100 irq 36
[    3.049970] spi-nor spi0.0: found mt35xu512aba, expected m25p80
[    3.055912] spi-nor spi0.0: mt35xu512aba (65536 Kbytes)
[    3.074948] libphy: Fixed MDIO Bus: probed
[    3.079809] tun: Universal TUN/TAP device driver, 1.6
[    3.085174] thunder_xcv, ver 1.0
[    3.088413] thunder_bgx, ver 1.0
[    3.091646] nicpf, ver 1.0
[    3.094618] libphy: Freescale XGMAC MDIO Bus: probed
[    3.100247] libphy: Freescale XGMAC MDIO Bus: probed
[    3.105354] libphy: Freescale XGMAC MDIO Bus: probed
[    3.110754] libphy: Freescale XGMAC MDIO Bus: probed
[    3.116102] libphy: Freescale XGMAC MDIO Bus: probed
[    3.121385] libphy: Freescale XGMAC MDIO Bus: probed
[    3.126647] Freescale FM module, FMD API version 21.1.0
[    3.131932] Freescale FM Ports module
[    3.135586] fsl_mac: fsl_mac: FSL FMan MAC API based driver
[    3.141197] fsl_dpa: FSL DPAA Ethernet driver
[    3.145583] fsl_advanced: FSL DPAA Advanced drivers:
[    3.150538] fsl_proxy: FSL DPAA Proxy initialization driver
[    3.156137] fsl_oh: FSL FMan Offline Parsing port driver
[    3.161785] hclge is initializing
[    3.496379] usbcore: registered new interface driver cdc_acm
[    3.634737] ftm-alarm 2800000.timer: registered as rtc1
[    3.684533] pca954x 0-0077: registered 8 multiplexed busses for I2C mux pca9547
[    3.696729] amc6821 4-0018: Revision 3
[    3.731770] random: fast init done
[    3.801614] usb 3-1: new high-speed USB device number 2 using xhci-hcd
[    4.000330] hub 3-1:1.0: USB hub found
[    4.004096] hub 3-1:1.0: 4 ports detected
[    4.081632] usb 4-1: new SuperSpeed Gen 1 USB device number 2 using xhci-hcd
[    4.118627] sbsa-gwdt 23a0000.watchdog: Initialized with 30s timeout @ 25000050 Hz, action=0. [enabled]
[    4.130218] qoriq-cpufreq qoriq-cpufreq: Freescale QorIQ CPU frequency scaling driver
[    4.138942] sdhci: Secure Digital Host Controller Interface driver
[    4.145134] sdhci: Copyright(c) Pierre Ossman
[    4.149738] Synopsys Designware Multimedia Card Interface Driver
[    4.156159] sdhci-pltfm: SDHCI platform and OF driver helper
[    4.160383] hub 4-1:1.0: USB hub found
[    4.165633] ledtrig-cpu: registered to indicate activity on CPUs
[    4.165873] hub 4-1:1.0: 4 ports detected
[    4.172041] SMCCC: SOC_ID: ARCH_SOC_ID not implemented, skipping ....
[    4.182248] usbcore: registered new interface driver usbhid
[    4.187812] usbhid: USB HID core driver
[    4.188114] mmc1: SDHCI controller on 2150000.esdhc [2150000.esdhc] using ADMA 64-bit
[    4.188173] mmc0: SDHCI controller on 2140000.esdhc [2140000.esdhc] using ADMA 64-bit
[    4.192282] No fsl,qman node
[    4.210170] Freescale USDPAA process driver
[    4.214343] fsl-usdpaa: no region found
[    4.218174] Freescale USDPAA process IRQ driver
[    4.224399] optee: probing for conduit method.
[    4.228851] optee: api uid mismatch
[    4.232337] optee: probe of firmware:optee failed with error -22
[    4.241238] pktgen: Packet Generator for packet performance testing. Version: 2.75
[    4.250197] NET: Registered protocol family 26
[    4.254664] u32 classifier
[    4.257381]     input device check on
[    4.261036]     Actions configured
[    4.264854] Initializing XFRM netlink socket
[    4.269172] NET: Registered protocol family 10
[    4.273945] Segment Routing with IPv6
[    4.277649] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[    4.283765] NET: Registered protocol family 17
[    4.288218] NET: Registered protocol family 15
[    4.292673] Bridge firewalling registered
[    4.295247] mmc1: new HS400 MMC card at address 0001
[    4.301983] mmcblk1: mmc1:0001 DG4064 59.2 GiB 
[    4.310485] mmcblk1boot0: mmc1:0001 DG4064 partition 1 4.00 MiB
[    4.322048] mmcblk1boot1: mmc1:0001 DG4064 partition 2 4.00 MiB
[    4.326206] tsn generic netlink module v1 init...
[    4.332170] mmcblk1rpmb: mmc1:0001 DG4064 partition 3 4.00 MiB, chardev (507:0)
[    4.336826] Key type dns_resolver registered
[    4.346209]  mmcblk1: p1
[    4.348562] registered taskstats version 1
[    4.355762] Btrfs loaded, crc32c=crc32c-generic
[    4.357839] mmc0: new ultra high speed SDR104 SDXC card at address e624
[    4.362007] fsl_mc_dprc dprc.1: Adding to iommu group 2
[    4.365616] usb 3-1.3: new high-speed USB device number 3 using xhci-hcd
[    4.367207] mmcblk0: mmc0:e624 SC256 238 GiB 
[    4.384027]  mmcblk0: p1
[    4.425972] fsl_mc_allocator dpbp.1: Adding to iommu group 2
[    4.431702] fsl_mc_allocator dpbp.0: Adding to iommu group 2
[    4.438434] fsl_mc_allocator dpmcp.35: Adding to iommu group 2
[    4.444827] fsl_mc_allocator dpmcp.34: Adding to iommu group 2
[    4.451227] fsl_mc_allocator dpmcp.33: Adding to iommu group 2
[    4.457641] fsl_mc_allocator dpmcp.32: Adding to iommu group 2
[    4.464040] fsl_mc_allocator dpmcp.31: Adding to iommu group 2
[    4.470432] fsl_mc_allocator dpmcp.30: Adding to iommu group 2
[    4.476825] fsl_mc_allocator dpmcp.29: Adding to iommu group 2
[    4.483217] fsl_mc_allocator dpmcp.28: Adding to iommu group 2
[    4.489614] fsl_mc_allocator dpmcp.27: Adding to iommu group 2
[    4.496005] fsl_mc_allocator dpmcp.26: Adding to iommu group 2
[    4.502400] fsl_mc_allocator dpmcp.25: Adding to iommu group 2
[    4.508794] fsl_mc_allocator dpmcp.24: Adding to iommu group 2
[    4.512349] hub 3-1.3:1.0: USB hub found
[    4.515189] fsl_mc_allocator dpmcp.23: Adding to iommu group 2
[    4.518611] hub 3-1.3:1.0: 4 ports detected
[    4.524945] fsl_mc_allocator dpmcp.22: Adding to iommu group 2
[    4.534949] fsl_mc_allocator dpmcp.21: Adding to iommu group 2
[    4.541342] fsl_mc_allocator dpmcp.20: Adding to iommu group 2
[    4.547734] fsl_mc_allocator dpmcp.19: Adding to iommu group 2
[    4.554135] fsl_mc_allocator dpmcp.18: Adding to iommu group 2
[    4.560528] fsl_mc_allocator dpmcp.17: Adding to iommu group 2
[    4.566930] fsl_mc_allocator dpmcp.16: Adding to iommu group 2
[    4.573329] fsl_mc_allocator dpmcp.15: Adding to iommu group 2
[    4.579729] fsl_mc_allocator dpmcp.14: Adding to iommu group 2
[    4.586130] fsl_mc_allocator dpmcp.13: Adding to iommu group 2
[    4.592524] fsl_mc_allocator dpmcp.12: Adding to iommu group 2
[    4.598920] fsl_mc_allocator dpmcp.11: Adding to iommu group 2
[    4.605314] fsl_mc_allocator dpmcp.10: Adding to iommu group 2
[    4.611710] fsl_mc_allocator dpmcp.9: Adding to iommu group 2
[    4.618024] fsl_mc_allocator dpmcp.8: Adding to iommu group 2
[    4.624332] fsl_mc_allocator dpmcp.7: Adding to iommu group 2
[    4.630648] fsl_mc_allocator dpmcp.6: Adding to iommu group 2
[    4.636956] fsl_mc_allocator dpmcp.5: Adding to iommu group 2
[    4.643265] fsl_mc_allocator dpmcp.4: Adding to iommu group 2
[    4.649575] fsl_mc_allocator dpmcp.3: Adding to iommu group 2
[    4.655884] fsl_mc_allocator dpmcp.2: Adding to iommu group 2
[    4.662193] fsl_mc_allocator dpmcp.1: Adding to iommu group 2
[    4.668003] fsl_mc_allocator dpcon.31: Adding to iommu group 2
[    4.673905] fsl_mc_allocator dpcon.30: Adding to iommu group 2
[    4.679798] fsl_mc_allocator dpcon.29: Adding to iommu group 2
[    4.685693] fsl_mc_allocator dpcon.28: Adding to iommu group 2
[    4.691589] fsl_mc_allocator dpcon.27: Adding to iommu group 2
[    4.697484] fsl_mc_allocator dpcon.26: Adding to iommu group 2
[    4.703378] fsl_mc_allocator dpcon.25: Adding to iommu group 2
[    4.709274] fsl_mc_allocator dpcon.24: Adding to iommu group 2
[    4.715169] fsl_mc_allocator dpcon.23: Adding to iommu group 2
[    4.721064] fsl_mc_allocator dpcon.22: Adding to iommu group 2
[    4.726963] fsl_mc_allocator dpcon.21: Adding to iommu group 2
[    4.732859] fsl_mc_allocator dpcon.20: Adding to iommu group 2
[    4.738755] fsl_mc_allocator dpcon.19: Adding to iommu group 2
[    4.744652] fsl_mc_allocator dpcon.18: Adding to iommu group 2
[    4.750549] fsl_mc_allocator dpcon.17: Adding to iommu group 2
[    4.756446] fsl_mc_allocator dpcon.16: Adding to iommu group 2
[    4.762350] fsl_mc_allocator dpcon.15: Adding to iommu group 2
[    4.768247] fsl_mc_allocator dpcon.14: Adding to iommu group 2
[    4.774145] fsl_mc_allocator dpcon.13: Adding to iommu group 2
[    4.780048] fsl_mc_allocator dpcon.12: Adding to iommu group 2
[    4.785947] fsl_mc_allocator dpcon.11: Adding to iommu group 2
[    4.791845] fsl_mc_allocator dpcon.10: Adding to iommu group 2
[    4.797747] fsl_mc_allocator dpcon.9: Adding to iommu group 2
[    4.803557] fsl_mc_allocator dpcon.8: Adding to iommu group 2
[    4.809369] fsl_mc_allocator dpcon.7: Adding to iommu group 2
[    4.815179] fsl_mc_allocator dpcon.6: Adding to iommu group 2
[    4.820995] fsl_mc_allocator dpcon.5: Adding to iommu group 2
[    4.826806] fsl_mc_allocator dpcon.4: Adding to iommu group 2
[    4.832619] fsl_mc_allocator dpcon.3: Adding to iommu group 2
[    4.838430] fsl_mc_allocator dpcon.2: Adding to iommu group 2
[    4.844243] fsl_mc_allocator dpcon.1: Adding to iommu group 2
[    4.850056] fsl_mc_allocator dpcon.0: Adding to iommu group 2
[    4.855872] fsl_dpaa2_eth dpni.0: Adding to iommu group 2
[    4.931606] fsl_dpaa2_mac dpmac.17: Adding to iommu group 2
[    4.937786] fsl_dpaa2_mac dpmac.10: Adding to iommu group 2
[    4.948399] fsl_dpaa2_mac dpmac.9: Adding to iommu group 2
[    4.958752] fsl_dpaa2_mac dpmac.8: Adding to iommu group 2
[    4.968845] fsl_dpaa2_mac dpmac.7: Adding to iommu group 2
[    4.979122] fsl_dpaa2_mac dpmac.6: Adding to iommu group 2
[    4.989313] fsl_dpaa2_mac dpmac.6: Error connecting to the MAC endpoint
[    4.996582] fsl_dpaa2_mac dpmac.5: Adding to iommu group 2
[    5.006648] fsl_dpaa2_mac dpmac.5: Error connecting to the MAC endpoint
[    5.013733] fsl_dpaa2_mac dpmac.4: Adding to iommu group 2
[    5.023936] fsl_dpaa2_mac dpmac.4: Error connecting to the MAC endpoint
[    5.031185] fsl_dpaa2_mac dpmac.3: Adding to iommu group 2
[    5.041297] fsl_dpaa2_mac dpmac.3: Error connecting to the MAC endpoint
[    5.048574] fsl_dpaa2_ptp dprtc.0: Adding to iommu group 2
[    5.056247] ptp_qoriq: device tree node missing required elements, try automatic configuration
[    5.064940] pps pps0: new PPS source ptp0
[    5.070620] fsl_mc_dpio dpio.15: Adding to iommu group 2
[    5.079544] fsl_mc_dpio dpio.15: probed
[    5.085473] fsl_mc_dpio dpio.14: Adding to iommu group 2
[    5.094385] fsl_mc_dpio dpio.14: probed
[    5.100312] fsl_mc_dpio dpio.13: Adding to iommu group 2
[    5.109225] fsl_mc_dpio dpio.13: probed
[    5.115152] fsl_mc_dpio dpio.12: Adding to iommu group 2
[    5.124108] fsl_mc_dpio dpio.12: probed
[    5.130134] fsl_mc_dpio dpio.11: Adding to iommu group 2
[    5.139069] fsl_mc_dpio dpio.11: probed
[    5.144991] fsl_mc_dpio dpio.10: Adding to iommu group 2
[    5.153902] fsl_mc_dpio dpio.10: probed
[    5.159819] fsl_mc_dpio dpio.9: Adding to iommu group 2
[    5.168643] fsl_mc_dpio dpio.9: probed
[    5.174486] fsl_mc_dpio dpio.8: Adding to iommu group 2
[    5.183346] fsl_mc_dpio dpio.8: probed
[    5.189275] fsl_mc_dpio dpio.7: Adding to iommu group 2
[    5.198140] fsl_mc_dpio dpio.7: probed
[    5.203980] fsl_mc_dpio dpio.6: Adding to iommu group 2
[    5.212805] fsl_mc_dpio dpio.6: probed
[    5.218646] fsl_mc_dpio dpio.5: Adding to iommu group 2
[    5.227507] fsl_mc_dpio dpio.5: probed
[    5.233438] fsl_mc_dpio dpio.4: Adding to iommu group 2
[    5.242279] fsl_mc_dpio dpio.4: probed
[    5.248119] fsl_mc_dpio dpio.3: Adding to iommu group 2
[    5.256940] fsl_mc_dpio dpio.3: probed
[    5.262782] fsl_mc_dpio dpio.2: Adding to iommu group 2
[    5.271646] fsl_mc_dpio dpio.2: probed
[    5.277572] fsl_mc_dpio dpio.1: Adding to iommu group 2
[    5.286426] fsl_mc_dpio dpio.1: probed
[    5.292262] fsl_mc_dpio dpio.0: Adding to iommu group 2
[    5.301085] fsl_mc_dpio dpio.0: probed
[    5.308486] fsl_mc_dprc dprc.1: DPRC device bound to driver
[    5.314213] pcieport 0000:00:00.0: PME: Signaling with IRQ 37
[    5.320099] pcieport 0001:00:00.0: PME: Signaling with IRQ 38
[    5.326513] libphy: SFP I2C Bus: probed
[    5.330399] sfp sfp-0: Host maximum power 2.0W
[    5.334974] sfp sfp-0: No tx_disable pin: SFP modules will always be emitting.
[    5.342751] libphy: SFP I2C Bus: probed
[    5.346626] sfp sfp-1: Host maximum power 2.0W
[    5.351187] sfp sfp-1: No tx_disable pin: SFP modules will always be emitting.
[    5.358930] libphy: SFP I2C Bus: probed
[    5.362810] sfp sfp-2: Host maximum power 2.0W
[    5.367365] sfp sfp-2: No tx_disable pin: SFP modules will always be emitting.
[    5.375105] libphy: SFP I2C Bus: probed
[    5.378977] sfp sfp-3: Host maximum power 2.0W
[    5.383536] sfp sfp-3: No tx_disable pin: SFP modules will always be emitting.
[    5.626843] fsl_dpaa2_eth dpni.0 (unnamed net_device) (uninitialized): PHY [0x0000000008b96000:01] driver [Qualcomm Atheros AR8035] (irq=POLL)
[    5.640694] fsl_dpaa2_eth dpni.0: Probed interface eth0
[    5.648659] fsl_dpaa2_mac dpmac.6: fsl_mc_allocate_irqs err -22
[    5.655109] fsl_dpaa2_mac dpmac.6: fsl_mc_driver_probe failed: -14
[    5.661335] fsl_dpaa2_mac: probe of dpmac.6 failed with error -14
[    5.669542] fsl_dpaa2_mac dpmac.5: fsl_mc_allocate_irqs err -22
[    5.675985] fsl_dpaa2_mac dpmac.5: fsl_mc_driver_probe failed: -14
[    5.681465] sfp sfp-1: module FS               SFP-10GSR-85     rev      sn F2130331012      dc 211116  
[    5.682209] fsl_dpaa2_mac: probe of dpmac.5 failed with error -14
[    5.699831] fsl_dpaa2_mac dpmac.4: fsl_mc_allocate_irqs err -22
[    5.706261] fsl_dpaa2_mac dpmac.4: fsl_mc_driver_probe failed: -14
[    5.712454] fsl_dpaa2_mac: probe of dpmac.4 failed with error -14
[    5.720605] fsl_dpaa2_mac dpmac.3: fsl_mc_allocate_irqs err -22
[    5.727031] fsl_dpaa2_mac dpmac.3: fsl_mc_driver_probe failed: -14
[    5.728205] sfp sfp-3: module FS               SFP-10GSR-85     rev      sn F2130331017      dc 211116  
[    5.733227] fsl_dpaa2_mac: probe of dpmac.3 failed with error -14
[    5.749739] input: gpio-keys as /devices/platform/gpio-keys/input/input0
[    5.757379] ALSA device list:
[    5.760359]   No soundcards found.
[    5.799586] EXT4-fs (mmcblk1p1): recovery complete
[    5.804939] EXT4-fs (mmcblk1p1): mounted filesystem with ordered data mode. Opts: (null)
[    5.813055] VFS: Mounted root (ext4 filesystem) on device 179:1.
[    5.819356] devtmpfs: mounted
[    5.823838] Freeing unused kernel memory: 6720K
[    5.828463] Run /sbin/init as init process
[    5.832553]   with arguments:
[    5.832555]     /sbin/init
[    5.832556]   with environment:
[    5.832558]     HOME=/
[    5.832560]     TERM=linux
[    6.914767] systemd-journald[303]: Received client request to flush runtime journal.
[    6.933764] systemd-journald[303]: File /var/log/journal/411c1793e8d24b9e85330daf9431bb3a/system.journal corrupted or uncleanly shut down, renaming and replacing.
[    8.621626] random: crng init done
[    8.621631] random: 7 urandom warning(s) missed due to ratelimiting
[  200.900525] fsl_dpaa2_eth dpni.0 eth0: configuring for phy/rgmii-id link mode
[  204.995997] fsl_dpaa2_eth dpni.0 eth0: Link is Up - 1Gbps/Full - flow control off
[  204.996057] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

OfficerDribble · May 13, 2022, 9:16am

I am seeing a similar problem after flashing ubuntu from ddab3ad into my CEX7 eMMC. eth0 appears but fails to get an IP. During boot I am seeing this fsl-mc firmware warning.

...
Moving Image from 0x81100000 to 0x81200000, end=83b20000
## Flattened Device Tree blob at 81000000
   Booting using the fdt blob at 0x81000000
   Loading Device Tree to 000000009fff2000, end 000000009fffff64 ... OK
Releasing fan controller full speed gpio
fsl-mc: Deploying data path layout ... WARNING: Firmware returned an error (GSR: 0x3f)

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
[    0.000000] Linux version 5.10.35-00018-gbb124648d42c (root@4b13fa7d6a6c) (aarch64-linux-gnu-gcc (Linaro GCC 7.4-2019.02) 7.4.1 20181213 [linaro-7.4-2019.02 revision 56ec6f6b99cc167ff0c2f8e1a2eed33b1edc85d
4], GNU ld (Linaro_Binutils-2019.02) 2.28.2.20170706) #1 SMP PREEMPT Wed May 11 17:07:05 UTC 2022
[    0.000000] Machine model: SolidRun LX2160A Clearfog CX
...

During U-Boot I see this

...
crc32+ 
fsl-mc: Booting Management Complex ... SUCCESS
fsl-mc: Management Complex booted (version: 10.28.1, boot status: 0x1)
Hit any key to stop autoboot:  0 
...

jnettlet · May 16, 2022, 6:04am

This looks like more incorrect patches were merged into the lx2160a_build for the BSP. I will take a look.

PacMan · May 16, 2022, 2:19pm

Just a small note: Home-built bc46e34 works for me. Try and see if that build works.

OfficerDribble · May 17, 2022, 12:27pm

Hi PacMan,

I have downloaded the pre-built SolidRrun image tagged as bc46e34 and flashed into eMMC. On reset I get exactly the same results as above. Note the firmware error before “Starting kernel”

Retrieving file: /boot/fsl-lx2160a-cex7.dtb
28471 bytes read in 18 ms (1.5 MiB/s)
## Flattened Device Tree blob at 81000000
   Booting using the fdt blob at 0x81000000
   Loading Device Tree to 000000009fff6000, end 000000009fffff36 ... OK
Releasing fan controller full speed gpio
fsl-mc: Deploying data path layout ... WARNING: Firmware returned an error (GSR: 0x3f)

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
[    0.000000] Linux version 5.4.47-00007-g8edfda9bc (root@0f1f486dcf65) (gcc version 7.4.1 20181213 [linaro-7.4-2019.02 revision 56ec6f6b99cc167ff0c2f8e1a2eed33b1edc85d4] (Linaro GCC 7.4-2019.02)) #1 SMP PREEMPT Tue Jun 15 19:24:47 UTC 2021
[    0.000000] Machine model: SolidRun LX2160A COM express type 7 module
[    0.000000] earlycon: pl11 at MMIO32 0x00000000021c0000 (options '')
[    0.000000] printk: bootconsole [pl11] enabled
...
root@localhost:~# ls-listni
dprc.1/dpni.0 (interface: eth0, end point: dpmac.17)
root@localhost:~# ifconfig
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 1600  bytes 113624 (113.6 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1600  bytes 113624 (113.6 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Can you share your kernel boot log?

I am now trying to extract MC boot logs from memory after the failure and make sense of them.

Thanks,
OD

PacMan · May 17, 2022, 12:44pm

Did you remember to issue …
dhclient
… on first run ?
-After that, it should be possible to enlarge the partition and then issue ‘unminimize’.
-Currently, I don’t have a good way of logging the boot output (I have a terminal, but there’s no copy and paste, so I’m using only screnshots).
… I might be able to snapshot my working bootable image, compress it and put it up on my server; just to see if that can help nailing down the problem.

OfficerDribble · May 17, 2022, 1:13pm

The 1Gbe copper interface is working. Forgot the dhclient! Doh

The 4x 10Gbe interfaces are not showing up and these are the ones that we require. This is the contents of the MC log buffer using U-Boot to boot the MC and extract the logs assuming this is related to the missing interfaces. I do see in dpl-eth.8x10g.19.dts 4 connections are commented out I’ll enable them and see what happens.

=> md 23e1400000
23e1400000: 202c575b 54414c50 4d524f46 5520205d    [W, PLATFORM]  U
23e1400010: 20545241 6e697270 61662074 64656c69    ART print failed
23e1400020: 6c61202c 6564206c 20677562 61746164    , all debug data
23e1400030: 6c697720 6562206c 69727020 6465746e     will be printed
23e1400040: 206f7420 66667562 0a2e7265 202c455b     to buffer..[E, 
23e1400050: 6e65706f 7665645f 3a656369 31373332    open_device:2371
23e1400060: 5044202c 5d43414d 6f6e2020 76656420    , DPMAC]  no dev
23e1400070: 0a656369 202c455b 696e7064 6f72705f    ice.[E, dpni_pro
23e1400080: 635f6562 31765f62 3037343a 44202c34    be_cb_v1:4704, D
23e1400090: 5d494e50 61432020 2074276e 61657263    PNI]  Can't crea
23e14000a0: 44206574 20494e50 31303030 2c455b0a    te DPNI 0001.[E,
23e14000b0: 62757320 65646f6e 6f72705f 73736563     subnode_process
23e14000c0: 3535313a 5020205d 69626f72 6d20676e    :155]  Probing m
23e14000d0: 6c75646f 64272065 27696e70 74657220    odule 'dpni' ret
23e14000e0: 206e7275 6f727265 6f632072 2d206564    urn error code -
23e14000f0: 202e3931 746e6f43 65756e69 6c706420    19. Continue dpl
=> 
23e1400100: 6f727020 73736563 2e676e69 5b0a2e2e     processing....[
23e1400110: 6f202c45 5f6e6570 69766564 323a6563    E, open_device:2
23e1400120: 2c313733 4d504420 205d4341 206f6e20    371, DPMAC]  no 
23e1400130: 69766564 5b0a6563 64202c45 5f696e70    device.[E, dpni_
23e1400140: 626f7270 62635f65 3a31765f 34303734    probe_cb_v1:4704
23e1400150: 5044202c 205d494e 6e614320 63207427    , DPNI]  Can't c
23e1400160: 74616572 50442065 3020494e 0a323030    reate DPNI 0002.
23e1400170: 202c455b 6e627573 5f65646f 636f7270    [E, subnode_proc
23e1400180: 3a737365 5d353531 72502020 6e69626f    ess:155]  Probin
23e1400190: 6f6d2067 656c7564 70642720 2027696e    g module 'dpni' 
23e14001a0: 75746572 65206e72 726f7272 646f6320    return error cod
23e14001b0: 312d2065 43202e39 69746e6f 2065756e    e -19. Continue 
23e14001c0: 206c7064 636f7270 69737365 2e2e676e    dpl processing..
23e14001d0: 455b0a2e 706f202c 645f6e65 63697665    ..[E, open_devic
23e14001e0: 33323a65 202c3137 414d5044 20205d43    e:2371, DPMAC]  
23e14001f0: 64206f6e 63697665 455b0a65 7064202c    no device.[E, dp
=> 
23e1400200: 705f696e 65626f72 5f62635f 343a3176    ni_probe_cb_v1:4
23e1400210: 2c343037 4e504420 20205d49 276e6143    704, DPNI]  Can'
23e1400220: 72632074 65746165 4e504420 30302049    t create DPNI 00
23e1400230: 5b0a3330 73202c45 6f6e6275 705f6564    03.[E, subnode_p
23e1400240: 65636f72 313a7373 205d3535 6f725020    rocess:155]  Pro
23e1400250: 676e6962 646f6d20 20656c75 6e706427    bing module 'dpn
23e1400260: 72202769 72757465 7265206e 20726f72    i' return error 
23e1400270: 65646f63 39312d20 6f43202e 6e69746e    code -19. Contin
23e1400280: 64206575 70206c70 65636f72 6e697373    ue dpl processin
23e1400290: 2e2e2e67 2c455b0a 65706f20 65645f6e    g....[E, open_de
23e14002a0: 65636976 3733323a 44202c31 43414d50    vice:2371, DPMAC
23e14002b0: 6e20205d 6564206f 65636976 2c455b0a    ]  no device.[E,
23e14002c0: 6e706420 72705f69 5f65626f 765f6263     dpni_probe_cb_v
23e14002d0: 37343a31 202c3430 494e5044 4320205d    1:4704, DPNI]  C
23e14002e0: 74276e61 65726320 20657461 494e5044    an't create DPNI
23e14002f0: 30303020 455b0a34 7573202c 646f6e62     0004.[E, subnod
=> 
23e1400300: 72705f65 7365636f 35313a73 20205d35    e_process:155]  
23e1400310: 626f7250 20676e69 75646f6d 2720656c    Probing module '
23e1400320: 696e7064 65722027 6e727574 72726520    dpni' return err
23e1400330: 6320726f 2065646f 2e39312d 6e6f4320    or code -19. Con
23e1400340: 756e6974 70642065 7270206c 7365636f    tinue dpl proces
23e1400350: 676e6973 0a2e2e2e 202c455b 6e65706f    sing....[E, open
23e1400360: 7665645f 3a656369 31373332 5044202c    _device:2371, DP
23e1400370: 5d43414d 6f6e2020 76656420 0a656369    MAC]  no device.
23e1400380: 202c455b 696e7064 6f72705f 635f6562    [E, dpni_probe_c
23e1400390: 31765f62 3037343a 44202c34 5d494e50    b_v1:4704, DPNI]
23e14003a0: 61432020 2074276e 61657263 44206574      Can't create D
23e14003b0: 20494e50 35303030 2c455b0a 62757320    PNI 0005.[E, sub
23e14003c0: 65646f6e 6f72705f 73736563 3535313a    node_process:155
23e14003d0: 5020205d 69626f72 6d20676e 6c75646f    ]  Probing modul
23e14003e0: 64272065 27696e70 74657220 206e7275    e 'dpni' return 
23e14003f0: 6f727265 6f632072 2d206564 202e3931    error code -19. 
=> 
23e1400400: 746e6f43 65756e69 6c706420 6f727020    Continue dpl pro
23e1400410: 73736563 2e676e69 5b0a2e2e 6f202c45    cessing....[E, o
23e1400420: 5f6e6570 69766564 323a6563 2c313733    pen_device:2371,
23e1400430: 4d504420 205d4341 206f6e20 69766564     DPMAC]  no devi
23e1400440: 5b0a6563 64202c45 5f696e70 626f7270    ce.[E, dpni_prob
23e1400450: 62635f65 3a31765f 34303734 5044202c    e_cb_v1:4704, DP
23e1400460: 205d494e 6e614320 63207427 74616572    NI]  Can't creat
23e1400470: 50442065 3020494e 0a363030 202c455b    e DPNI 0006.[E, 
23e1400480: 6e627573 5f65646f 636f7270 3a737365    subnode_process:
23e1400490: 5d353531 72502020 6e69626f 6f6d2067    155]  Probing mo
23e14004a0: 656c7564 70642720 2027696e 75746572    dule 'dpni' retu
23e14004b0: 65206e72 726f7272 646f6320 312d2065    rn error code -1
23e14004c0: 43202e39 69746e6f 2065756e 206c7064    9. Continue dpl 
23e14004d0: 636f7270 69737365 2e2e676e 455b0a2e    processing....[E
23e14004e0: 706f202c 645f6e65 63697665 33323a65    , open_device:23
23e14004f0: 202c3137 414d5044 20205d43 64206f6e    71, DPMAC]  no d
=> 
23e1400500: 63697665 455b0a65 7064202c 705f696e    evice.[E, dpni_p
23e1400510: 65626f72 5f62635f 343a3176 2c343037    robe_cb_v1:4704,
23e1400520: 4e504420 20205d49 276e6143 72632074     DPNI]  Can't cr
23e1400530: 65746165 4e504420 30302049 5b0a3730    eate DPNI 0007.[
23e1400540: 73202c45 6f6e6275 705f6564 65636f72    E, subnode_proce
23e1400550: 313a7373 205d3535 6f725020 676e6962    ss:155]  Probing
23e1400560: 646f6d20 20656c75 6e706427 72202769     module 'dpni' r
23e1400570: 72757465 7265206e 20726f72 65646f63    eturn error code
23e1400580: 39312d20 6f43202e 6e69746e 64206575     -19. Continue d
23e1400590: 70206c70 65636f72 6e697373 2e2e2e67    pl processing...
23e14005a0: 2c455b0a 65706f20 65645f6e 65636976    .[E, open_device
23e14005b0: 3733323a 44202c31 43414d50 6e20205d    :2371, DPMAC]  n
23e14005c0: 6564206f 65636976 2c455b0a 6e706420    o device.[E, dpn
23e14005d0: 72705f69 5f65626f 765f6263 37343a31    i_probe_cb_v1:47
23e14005e0: 202c3430 494e5044 4320205d 74276e61    04, DPNI]  Can't
23e14005f0: 65726320 20657461 494e5044 30303020     create DPNI 000
=> 
23e1400600: 455b0a38 7573202c 646f6e62 72705f65    8.[E, subnode_pr
23e1400610: 7365636f 35313a73 20205d35 626f7250    ocess:155]  Prob
23e1400620: 20676e69 75646f6d 2720656c 696e7064    ing module 'dpni
23e1400630: 65722027 6e727574 72726520 6320726f    ' return error c
23e1400640: 2065646f 2e39312d 6e6f4320 756e6974    ode -19. Continu
23e1400650: 70642065 7270206c 7365636f 676e6973    e dpl processing
23e1400660: 0a2e2e2e 202c455b 6e65706f 7665645f    ....[E, open_dev
23e1400670: 3a656369 31373332 5044202c 5d43414d    ice:2371, DPMAC]
23e1400680: 6f6e2020 76656420 0a656369 202c455b      no device.[E, 
23e1400690: 616d7064 72705f63 5f65626f 373a6263    dpmac_probe_cb:7
23e14006a0: 202c3636 414d5044 20205d43 276e6143    66, DPMAC]  Can'
23e14006b0: 72632074 65746165 4d504420 30204341    t create DPMAC 0
23e14006c0: 0a323030 202c455b 6e627573 5f65646f    002.[E, subnode_
23e14006d0: 636f7270 3a737365 5d353531 72502020    process:155]  Pr
23e14006e0: 6e69626f 6f6d2067 656c7564 70642720    obing module 'dp
23e14006f0: 2763616d 74657220 206e7275 6f727265    mac' return erro
=> 
23e1400700: 6f632072 2d206564 202e3931 746e6f43    r code -19. Cont
23e1400710: 65756e69 6c706420 6f727020 73736563    inue dpl process
23e1400720: 2e676e69 5b0a2e2e 6f202c45 5f6e6570    ing....[E, open_
23e1400730: 69766564 323a6563 2c313733 4d504420    device:2371, DPM
23e1400740: 205d4341 206f6e20 69766564 5b0a6563    AC]  no device.[
23e1400750: 64202c45 63616d70 6f72705f 635f6562    E, dpmac_probe_c
23e1400760: 36373a62 44202c36 43414d50 4320205d    b:766, DPMAC]  C
23e1400770: 74276e61 65726320 20657461 414d5044    an't create DPMA
23e1400780: 30302043 5b0a3231 73202c45 6f6e6275    C 0012.[E, subno
23e1400790: 705f6564 65636f72 313a7373 205d3535    de_process:155] 
23e14007a0: 6f725020 676e6962 646f6d20 20656c75     Probing module 
23e14007b0: 6d706427 20276361 75746572 65206e72    'dpmac' return e
23e14007c0: 726f7272 646f6320 312d2065 43202e39    rror code -19. C
23e14007d0: 69746e6f 2065756e 206c7064 636f7270    ontinue dpl proc
23e14007e0: 69737365 2e2e676e 455b0a2e 7064202c    essing....[E, dp
23e14007f0: 72705f6c 7365636f 32353a73 20205d37    l_process:527]  
=> 
23e1400800: 6f727245 68772072 20656c69 73726170    Error while pars
23e1400810: 20676e69 6e6f6327 7463656e 736e6f69    ing 'connections
23e1400820: 53202e27 2070696b 636f7270 69737365    '. Skip processi
23e1400830: 7420676e 72206568 20747365 4420666f    ng the rest of D
23e1400840: 0a2e4c50 202c455b 6e69616d 3638313a    PL..[E, main:186
23e1400850: 4420205d 70204c50 65636f72 6e697373    ]  DPL processin
23e1400860: 61662067 64656c69 6f63203b 6e69746e    g failed; contin
23e1400870: 676e6975 0a2e2e2e 6e6e7552 20676e69    uing....Running 
23e1400880: 6120434d 202c7070 74696177 20676e69    MC app, waiting 
23e1400890: 20726f66 6e657665 2e207374 000a2e2e    for events .....
23e14008a0: 00000000 00000000 00000000 00000000    ................
23e14008b0: 00000000 00000000 00000000 00000000    ................
23e14008c0: 00000000 00000000 00000000 00000000    ................
23e14008d0: 00000000 00000000 00000000 00000000    ................
23e14008e0: 00000000 00000000 00000000 00000000    ................
23e14008f0: 00000000 00000000 00000000 00000000    ................

PacMan · May 17, 2022, 3:11pm

I tend to forget a lot of things myself. The important part is that you have at least basic networking available.
Try issuing …
cat /extlinux/extlinux.conf
… and check if the ‘APPEND’ line starts with something like …
APPEND arm-smmu.disable_bypass=0 iommu.passthrough=1
… if not, try inserting those two arguments, write and reboot and see if you can succeed in …

ls-addni dpmac.7
ls-addni dpmac.8
ls-addni dpmac.9
ls-addni dpmac.10
dhclient
ip a

If issuing dhclient after adding the interfaces, you should get an IP-address from your router.
-Or you could of course manually assign an IP-address.

… If this helps, then you should be able to proceed with expanding the image and issue a …
unminimize
… (please do not attempt to unminimize without first expanding the image, it’ll definitely fail)
I’m currently in the process of prepping a bootable SD/MicroSD card image of bc46e34; so far my testing went well (except from my TP-Link junk-router prevented my Mac from getting any DNS-lookups until I powered it off and on again - but my ClearFog will replace that when I move to a new location).

PacMan · May 17, 2022, 5:59pm

Note: I edited my post since I first wrote it; issue dhclient after you create the interfaces with ls-addni dpmac.#
-I still have to learn how to write a proper systemd service for bringing up the interfaces while Linux is booting.
There’s a few hints here …
https://unix.stackexchange.com/questions/229048
… in particular the ‘Before’ and ‘Wants’ part …

File: /etc/systemd/system/ls-addni.service

[Unit]
DefaultDependencies=no
Description=Bring up high speed network interfaces
Before=network-pre.target
Wants=network-pre-target

[Service]
Type=oneshot
ExecStart=/usr/local/sbin/cx-lx2-network.sh
TimeoutSec=0
RemainAfterExit=yes

[Install]
RequiredBy=network-pre.target
# (maybe WantedBy=network-pre.target)

File: /usr/local/sbin/cx-lx2-network.sh

#!/bin/bash
for n in {3..10}; do ls-addni dpmac.$n; done
dhclient

If you wish a different order, then just change {3…10} to something different; for instance …
for n in 7 9 8 10 6 5 4 3; do ls-addni dpmac.$n; done

… There’s likely a lot of things that should be done differently, but hopefully it’ll help you in finding the correct solution.
-And maybe someone already did all the work for us.

Note: This seem to work for me; when the ‘login-prompt’ is reached, I’m able to log in via the 10Gbit network. I think it would be a bad idea to start configuring the network in the script; that should probably be done using interfaces.d or netplan … Yes, that includes that ‘dhclient’ should be removed as well.

PacMan · May 18, 2022, 7:01am

Image is finally finished and tested.
Can be downloaded here for a short while in case you need it.
http://scratch.gpio.dk/generic.img.bz2

The image requires a 4GB card or larger, since I made 2GB free space for unminimizing.
Can be written with something like this …
bunzip -kc generic.img.bz2 | sudo dd of=/dev/sdX

login: admin, password: 1234
After logging in, issue the usual …

dhclient
unminimize
apt install nano
passwd

… and the image should be ready for use.
This image works for me and I’m able to use the fast network.
Unfortunately I didn’t succeed in bringing up the network automatically.
(Perhaps I need RequiredBy=networking.service or RequiredBy=network.target instead of network-pre.target in the .service file).

Network interfaces can be made available by running the script in /usr/local/sbin manually …
cx-lx2-network.sh

Note: systemd can automatically execute programs/scripts if making ls-addni.device files, so actions can be made when interfaces become available.

OfficerDribble · May 18, 2022, 7:29am

Many thanks for this. I’ll give it a test now.

At this moment in time all I need to do is loop back traffic from 1 SFP to a second SFP using a fibre cable to emulate the system under test.

I have had no luck at all in getting linux to ping from one interface to the other. The ARP table is being partially filled as shown below.

eth1 = dpmac.7 = 192.168.1.1/24
eth2 = dpmac.9 = 192.168.1.2/24

Then I ping from eth1 to eth2 and from eth2 to eth1. No ICMP responses are received

ping 192.168.1.1 -I eth2
ping 192.168.1.2 -I eth1

root@linux:~# arp -an
? (192.168.1.2) at d0:63:b4:03:2a:b4 [ether] on eth1
? (192.168.61.70) at b0:7b:25:19:c5:f2 [ether] on eth0
? (192.168.61.1) at 00:00:5e:00:01:3d [ether] on eth0
? (192.168.1.1) at <incomplete> on eth2

Wait a few seconds and try again

root@linux:~# arp -an
? (192.168.1.2) at <incomplete> on eth1
? (192.168.61.70) at b0:7b:25:19:c5:f2 [ether] on eth0
? (192.168.61.1) at 00:00:5e:00:01:3d [ether] on eth0
? (192.168.1.1) at d0:63:b4:03:2a:b2 [ether] on eth2

Again thanks for you image I shall try this same test with your image.

womble · May 18, 2022, 4:35pm

Thanks @jnettlet, if you can shed any light on this networking issue that would be much appreciated. I’m currently looking to de-risk 10Gbit serdes operation with other system components before we commit to PCB design activities so progressing this is very important to us.

We really need to get to a position where the networking functionality is operational (both from Linux and DPDK) and we can send 10Gbit data flows over SFPs and DAC cables using an up-to-date kernel and BSP. I’m really reluctant to go back to an older build that appears to work as we need something with the latest fixes, SFP support and a build that we can maintain and use as we go forward with development.

It appears that something is very wrong with the LX2160A networking configuration with the latest builds beyond just the RGMII / copper interface. The SFP+ cages exhibit similar problems - it’s rare that the interface counters will ever show receipt of a packet on any of the interfaces although we’ve seen an occasional ARP probe succeed in one direction.

PacMan · May 19, 2022, 2:58pm

I’ve now been able to build dbab3ad, write a bootable image to MicroSD and boot from it.
It seems I’m having problems with the high speed interfaces as well. ls-addni dpmac.X gives me some error-messages I did not see in bc46e34:
https://pastebin.com/Jdieg8Wq

-I am currently stepping back one build at a time, to see where I get the same results as bc46e34.

OfficerDribble · May 19, 2022, 3:52pm

Working with bc46e34 I have been unable to get the 10Gbe interfaces to work as linux eth devices so I’ve switched focus to trying to get dpdk & pktgen to work. I may be missing something here, I have had trouble finding a definitive set of instructions for doing, but this is but this is what I’m getting

root@linux:~# dynamic_dpl.sh dpmac.7 dpmac.9
parent - dprc.1
Creating Non nested DPRC
NEW DPRCs
dprc.1
  dprc.2
Using board type as 2160
Using High Performance Buffers

##################### Container  dprc.2  is created ####################

Container dprc.2 have following resources :=>

 * 3 DPMCP
 * 16 DPBP
 * 8 DPCON
 * 16 DPSECI
 * 2 DPNI
 * 34 DPIO
 * 2 DPCI
 * 8 DPDMAI
 * 0 DPRTC


######################### Configured Interfaces #########################

Interface Name        Endpoint              Mac Address              
==============        ========              ==================       
dpni.1                dpmac.7               -Dynamic-                
dpni.2                dpmac.9               -Dynamic-

Then I run pktgen so I can try and create some traffic

root@linux:~# pktgen



Copyright(c) <2010-2021>, Intel Corporation. All rights reserved. Powered by DPDK
EAL: Detected CPU lcores: 16
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: No available 2048 kB hugepages reported
EAL: No available 32768 kB hugepages reported
EAL: No available 64 kB hugepages reported
EAL: VFIO support initialized
TELEMETRY: No legacy callbacks, legacy socket not created
Lua 5.3.3  Copyright (C) 1994-2016 Lua.org, PUC-Rio

*** Copyright(c) <2010-2021>, Intel Corporation. All rights reserved.
*** Pktgen  created by: Keith Wiles -- >>> Powered by DPDK <<<

 Port: Name         IfIndex Alias        NUMA  PCI

!PANIC!: *** Did not find any ports to use ***
PANIC in pktgen_config_ports():
*** Did not find any ports to use ***
6: [pktgen(+0xca8c) [0xaaaab8f67a8c]]
5: [/lib/aarch64-linux-gnu/libc.so.6(__libc_start_main+0xe8) [0xffff83167e10]]
4: [pktgen(+0xc5a8) [0xaaaab8f675a8]]
3: [pktgen(+0x31fa4) [0xaaaab8f8cfa4]]
2: [/lib/librte_eal.so.22(__rte_panic+0xb0) [0xffff834862c0]]
1: [/lib/librte_eal.so.22(rte_dump_stack+0x34) [0xffff834abce4]]
Aborted

There are the created interfaces

root@linux:~# ls-listni
dprc.1/dpni.0 (interface: eth0, end point: dpmac.17)
dprc.1/dprc.2/dpni.2 (end point: dpmac.9)
dprc.1/dprc.2/dpni.1 (end point: dpmac.7)

This is what dpdk thinks it can see

root@linux:~# dpdk-devbind.py -s
lspci: Unable to load libkmod resources: error -12
lspci: Unable to load libkmod resources: error -12
lspci: Unable to load libkmod resources: error -12
lspci: Unable to load libkmod resources: error -12
lspci: Unable to load libkmod resources: error -12
lspci: Unable to load libkmod resources: error -12
lspci: Unable to load libkmod resources: error -12
lspci: Unable to load libkmod resources: error -12

No 'Network' devices detected
=============================

No 'Baseband' devices detected
==============================

No 'Crypto' devices detected
============================

No 'Eventdev' devices detected
==============================

No 'Mempool' devices detected
=============================

No 'Compress' devices detected
==============================

No 'Misc (rawdev)' devices detected
===================================

No 'Regex' devices detected
===========================

I’m not sure why the hugepages warnings come up as dpdk seems happy with my default config. I have tried adding more pages but this makes no difference.

root@linux:~# dpdk-hugepages.py -s
Node Pages Size Total
0    2     1Gb    2Gb

Hugepages mounted on /dev/hugepages

PacMan · May 20, 2022, 2:31pm

Good news!
I’ve now been able to successfully build, boot and set up ddab3ad (latest commit).
Many of my problems were caused by the well-known “Error 40; user error.”

I got thrown off by the ‘depmod’ - error during build. It turns out that the image works anyway, but this error very likely needs some attention.
I still get a bunch of errors while issuing ls-addni lines, and am now only able to use interfaces 7…9 (not 3…6) of the CX. Build bc46e34 seemed to allow 3-6 as well; at some point I’ll probably try and find where the change happend.

What’s new ?
I figured out that I should use ‘nmcli’ for IP-configuration. Much easier and much more reliable than my other attempts.

I’ve gotten my systemd service working and even though I shouldn’t, I’ve made the same script also initialize the ip-addresses.

File: /etc/systemd/system/ls-addni.service:

[Unit]
Description=Bring up high-speed network interfaces
Before=network-pre.target
Wants=network-pre.target

[Service]
Type=oneshot
ExecStart=/usr/local/sbin/cx-lx2-network.sh
TimeoutSec=0
RemainAfterExit=yes

[Install]
RequiredBy=network.target
#will not work correctly: RequiredBy=network-pre.target
#will not work correctly: WantedBy=network-pre.target

… and the scripts …

File: /usr/local/sbin/cx-lx2-network.sh:

#!/bin/bash

for n in {3..10}; do ls-addni dpmac.$n;done
# The following is wrong to do here (but it does bring network up during startup for me):
#dhclient
[ - x /usr/local/sbin/lx2-netconf.sh ] && /usr/local/sbin/lx2-netconf.sh

File: /usr/local/sbin/lx2-netconf.sh:

#!/bin/bash

eth0ip=10.0.1.110/16
eth1ip=10.0.1.1/16
eth2ip=10.0.1.112/16
eth3ip=10.0.1.113/16
eth4ip=10.0.1.114/16
eth5ip=10.0.1.115/16
eth6ip=10.0.1.116/16
eth7ip=10.0.1.117/16
eth8ip=10.0.1.118/16

# delete all connections:
uuidlist=($(nmcli -f uuid -c no -t con show))
for uuid in "${uuidlist[@]}"; do
  nmcli con del $uuid
done


# Add our own configuration:
devlist=($(nmcli -c no -f device -t dev))
for dev in "${devlist[@]}";do
  dev_ip=${dev}ip
  cfg=(
    con add
    type ethernet
    con-name "$dev"
    ifname "$dev"
    ipv4.method manual
    ipv4.dns 208.67.222.222,208.67.220.220
  )
  case $dev in
#  eth0) nmcli ${cfg[@]} ipv4.addresses ${!dev_ip} ;;
  eth0) echo -n ;;  # so far this has only blocked other interfaces from working
  eth*) nmcli ${cfg[@]} ipv4.addresses ${!dev_ip} gw4 10.0.0.1
  nmcli con up $dev
  ;;
  esac
done

Make scripts executable and enable ls-addni service:

chmod +x /usr/local/sbin/{lx2-netconf,cx-lx2-network}.sh
systemctl enable ls-addni.service

On first boot, I do the following:

dhclient eth1; unminimize
apt update && apt install -y nano
reboot

‘dhclient eth1’ will probably ruin the static ip setup by lx2-netconf.sh, which is why I didn’t run the script only once.
As soon as you’ve unminimized the partition, you can delete or rename lx2-netconf.sh

In case you want it, here’s my image: http://scratch.multitrading.dk/ddab3ad.img.bz2

Logging in via ssh should be possible using …

ssh 10.0.1.1 -l admin
# password: 1234

After rebooting, network should be available with the specified static IP addresses.

Though still slightly ‘hackish’, I hope this will help both of you.

PacMan · May 21, 2022, 7:33pm

Seems there’s a “smart copy-and-paste” error above, which I can no longer edit:
[ - x /usr/local/sbin/lx2-netconf.sh ] && /usr/local/sbin/lx2-netconf.sh

… should be …
[ -x /usr/local/sbin/lx2-netconf.sh ] && /usr/local/sbin/lx2-netconf.sh

-otherwise the script will produce an error and not configure IPs on first and second run.

womble · May 23, 2022, 3:39pm

I’ve tried replacing the 5.10 kernel Image from a ddab3ad build with pre-built 5.10 (LSDK 21.08) kernel from NXP https://www.nxp.com/lgfiles/sdk/lsdk2108/boot_LS_arm64_lts_5.10.tgz

The NXP kernel fixes the issue of eth0 (copper interface) not working so the issue is caused by a difference in configuration between those kernels or one of the patches applied by lx2160a_build. I’ll try reverting some of the patches and see if they are causing the issue.

OfficerDribble · May 23, 2022, 4:40pm

I have rebuilt ddab3ad (5.10 kernel) with no linux-LSDK-21.08 patches and the 1Gbe link is working as expected. Next challenge is to see if DPDK/PKTGEN can drive the 10Gbps links.

OfficerDribble · May 23, 2022, 5:02pm

well this bodes well compared to before. Job for tomorrow to see if I can get PKTGEN working

root@localhost:~# dpdk-testpmd 
EAL: Detected 16 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
fslmc: Skipping invalid device (power)
EAL: Selected IOVA mode 'VA'
EAL: No available hugepages reported in hugepages-2048kB
EAL: No available hugepages reported in hugepages-32768kB
EAL: No available hugepages reported in hugepages-64kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
PMD: dpni.1: netdev created
PMD: dpni.2: netdev created
EAL: No legacy callbacks, legacy socket not created
testpmd: create a new mbuf pool <mb_pool_0>: n=267456, size=2176, socket=0
testpmd: preferred mempool ops selected: dpaa2
Configuring Port 0 (socket 0)
Port 0: D0:63:B4:03:2A:B2
Configuring Port 1 (socket 0)
Port 1: D0:63:B4:03:2A:B4
Checking link statuses...
Done
No commandline core given, start packet forwarding
io packet forwarding - ports=2 - cores=1 - streams=2 - NUMA support enabled, MP allocation mode: native
Logical Core 1 (socket 0) forwards packets on 2 streams:
  RX P=0/Q=0 (socket 0) -> TX P=1/Q=0 (socket 0) peer=02:00:00:00:00:01
  RX P=1/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=02:00:00:00:00:00

  io packet forwarding packets/burst=32
  nb forwarding cores=1 - nb forwarding ports=2
  port 0: RX queue number: 1 Tx queue number: 1
    Rx offloads=0x0 Tx offloads=0x10000
    RX queue: 0
      RX desc=512 - RX free threshold=1
      RX threshold registers: pthresh=0 hthresh=0  wthresh=0
      RX Offloads=0x0
    TX queue: 0
      TX desc=512 - TX free threshold=0
      TX threshold registers: pthresh=0 hthresh=0  wthresh=0
      TX offloads=0x0 - TX RS bit threshold=0
  port 1: RX queue number: 1 Tx queue number: 1
    Rx offloads=0x0 Tx offloads=0x10000
    RX queue: 0
      RX desc=512 - RX free threshold=1
      RX threshold registers: pthresh=0 hthresh=0  wthresh=0
      RX Offloads=0x0
    TX queue: 0
      TX desc=512 - TX free threshold=0
      TX threshold registers: pthresh=0 hthresh=0  wthresh=0
      TX offloads=0x0 - TX RS bit threshold=0

OfficerDribble · May 25, 2022, 7:03am

I now have have a “working” build of pktgen running on my CEX7 but I am not seeing any tx or rx packets on any of the 4 ports (dpmac.7, dpmac.8, dpmac.9 or dpmac.10). Anybody got any ideas?

This is using my kernel built with no SolidRun patches. I wonder if what @womble has reported is related and if a subset of the current patches are required in order to actually get traffic flowing.

root@linux:~# uname -a
Linux linux 5.10.35 #1 SMP PREEMPT Mon May 23 16:38:19 BST 2022 aarch64 aarch64 aarch64 GNU/Linux
root@linux:~# ls-listni
dprc.1/dpni.0 (interface: eth0, end point: dpmac.17)
dprc.1/dprc.2/dpni.4 (end point: dpmac.10)
dprc.1/dprc.2/dpni.3 (end point: dpmac.9)
dprc.1/dprc.2/dpni.2 (end point: dpmac.8)
dprc.1/dprc.2/dpni.1 (end point: dpmac.7)
root@linux:~# pktgen -l 0-9 -n 1 -- -v -P -m [1:2].0,[3:4].1,[5:6].2,[7:8].3



Copyright(c) <2010-2021>, Intel Corporation. All rights reserved. Powered by DPDK
EAL: Detected 16 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
fslmc: Skipping invalid device (power)
EAL: Selected IOVA mode 'VA'
EAL: No available hugepages reported in hugepages-2048kB
EAL: No available hugepages reported in hugepages-32768kB
EAL: No available hugepages reported in hugepages-64kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
PMD: dpni.1: netdev created
PMD: dpni.2: netdev created
PMD: dpni.3: netdev created
PMD: dpni.4: netdev created
EAL: No legacy callbacks, legacy socket not created
Lua 5.3.3  Copyright (C) 1994-2016 Lua.org, PUC-Rio

*** Copyright(c) <2010-2021>, Intel Corporation. All rights reserved.
*** Pktgen  created by: Keith Wiles -- >>> Powered by DPDK <<<

>>> Packet Burst 32, RX Desc 256, TX Desc 512, mbufs/port 4096, mbuf cache 512
 Port: Name         IfIndex Alias        NUMA  PCI
    0: net_dpaa2       0                   0   
    1: net_dpaa2       0                   0   
    2: net_dpaa2       0                   0   
    3: net_dpaa2       0                   0   


=== port to lcore mapping table (# lcores 10) ===
   lcore:    0       1       2       3       4       5       6       7       8       9      Total
port   0: ( D: T) ( 1: 0) ( 0: 1) ( 0: 0) ( 0: 0) ( 0: 0) ( 0: 0) ( 0: 0) ( 0: 0) ( 0: 0) = ( 1: 1)
port   1: ( D: T) ( 0: 0) ( 0: 0) ( 1: 0) ( 0: 1) ( 0: 0) ( 0: 0) ( 0: 0) ( 0: 0) ( 0: 0) = ( 1: 1)
port   2: ( D: T) ( 0: 0) ( 0: 0) ( 0: 0) ( 0: 0) ( 1: 0) ( 0: 1) ( 0: 0) ( 0: 0) ( 0: 0) = ( 1: 1)
port   3: ( D: T) ( 0: 0) ( 0: 0) ( 0: 0) ( 0: 0) ( 0: 0) ( 0: 0) ( 1: 0) ( 0: 1) ( 0: 0) = ( 1: 1)
Total   : ( 0: 0) ( 1: 0) ( 0: 1) ( 1: 0) ( 0: 1) ( 1: 0) ( 0: 1) ( 1: 0) ( 0: 1) ( 0: 0)
  Display and Timer on lcore 0, rx:tx counts per port/lcore

>>>> Configuring 4 ports, MBUF Size 9746, MBUF Cache Size 512
Lcore:
    1, RX-Only
                RX_cnt( 1): (pid= 0:qid= 0) 
    2, TX-Only
                TX_cnt( 1): (pid= 0:qid= 0) 
    3, RX-Only
                RX_cnt( 1): (pid= 1:qid= 0) 
    4, TX-Only
                TX_cnt( 1): (pid= 1:qid= 0) 
    5, RX-Only
                RX_cnt( 1): (pid= 2:qid= 0) 
    6, TX-Only
                TX_cnt( 1): (pid= 2:qid= 0) 
    7, RX-Only
                RX_cnt( 1): (pid= 3:qid= 0) 
    8, TX-Only
                TX_cnt( 1): (pid= 3:qid= 0) 

Port :
    0, nb_lcores  2, private 0xaaaae0b5c200, lcores:  1  2 
    1, nb_lcores  2, private 0xaaaae0f32600, lcores:  3  4 
    2, nb_lcores  2, private 0xaaaae1308a00, lcores:  5  6 
    3, nb_lcores  2, private 0xaaaae16dee00, lcores:  7  8 


Initialize Port 0 -- TxQ 1, RxQ 1
** Device Info (dpni.1, if_index:0, flags 00000066) **
   min_rx_bufsize :  512  max_rx_pktlen     :10240  hash_key_size :    0
   max_rx_queues  :  128  max_tx_queues     :   16  max_vfs       :    0
   max_mac_addrs  :   16  max_hash_mac_addrs:    0  max_vmdq_pools:   16
   vmdq_queue_base:    0  vmdq_queue_num    :    0  vmdq_pool_base:    0
   nb_rx_queues   :    0  nb_tx_queues      :    0  speed_capa    : 00005d60

   flow_type_rss_offloads:000000029e03fffc  reta_size             :    0
   rx_offload_capa       :VLAN_STRIP IPV4_CKSUM UDP_CKSUM TCP_CKSUM OUTER_IPV4_CKSUM VLAN_FILTER JUMBO_FRAME SCATTER TIMESTAMP SCTP_CKSUM OUTER_UDP_CKSUM 
   tx_offload_capa       :VLAN_INSERT IPV4_CKSUM UDP_CKSUM TCP_CKSUM SCTP_CKSUM OUTER_IPV4_CKSUM MT_LOCKFREE MULTI_SEGS MBUF_FAST_FREE 
   rx_queue_offload_capa :0000000000000000  tx_queue_offload_capa :0000000000000000
   dev_capa              :0000000000000000

  RX Conf:
     pthresh        :    0 hthresh          :    0 wthresh        :    0
     Free Thresh    :    0 Drop Enable      :    0 Deferred Start :    0
     offloads       :0000000000000000
  TX Conf:
     pthresh        :    0 hthresh          :    0 wthresh        :    0
     Free Thresh    :    0 RS Thresh        :    0 Deferred Start :    0
     offloads       :0000000000000000
  Rx: descriptor Limits
     nb_max         :65535  nb_min          :    0  nb_align      :    1
     nb_seg_max     :65535  nb_mtu_seg_max  :65535
  Tx: descriptor Limits
     nb_max         :65535  nb_min          :    0  nb_align      :    1
     nb_seg_max     :65535  nb_mtu_seg_max  :65535
  Rx: Port Config
     burst_size     :   32  ring_size       :  512  nb_queues     :    1
  Tx: Port Config
     burst_size     :   32  ring_size       :  512  nb_queues     :    1
  Switch Info: (null)
     domain_id      :65535  port_id         :    0

    Create: Default RX  0:0  - Memory used (MBUFs  4096 x (size 9746 + Hdr 128)) + 192 =  39497 KB, headroom 128
      Set RX queue stats mapping pid 0, q 0, lcore 1


    Create: Default TX  0:0  - Memory used (MBUFs  4096 x (size 9746 + Hdr 128)) + 192 =  39497 KB, headroom 128
    Create: Range TX    0:0  - Memory used (MBUFs  4096 x (size 9746 + Hdr 128)) + 192 =  39497 KB, headroom 128
    Create: Rate TX     0:0  - Memory used (MBUFs  4096 x (size 9746 + Hdr 128)) + 192 =  39497 KB, headroom 128
    Create: Sequence TX 0:0  - Memory used (MBUFs  4096 x (size 9746 + Hdr 128)) + 192 =  39497 KB, headroom 128
    Create: Special TX  0:0  - Memory used (MBUFs    64 x (size 9746 + Hdr 128)) + 192 =    618 KB, headroom 128

                                                                       Port memory used = 198103 KB
Src MAC d0:63:b4:03:2a:b2
 <Promiscuous mode Enabled>
Initialize Port 1 -- TxQ 1, RxQ 1
** Device Info (dpni.2, if_index:0, flags 00000066) **
   min_rx_bufsize :  512  max_rx_pktlen     :10240  hash_key_size :    0
   max_rx_queues  :  128  max_tx_queues     :   16  max_vfs       :    0
   max_mac_addrs  :   16  max_hash_mac_addrs:    0  max_vmdq_pools:   16
   vmdq_queue_base:    0  vmdq_queue_num    :    0  vmdq_pool_base:    0
   nb_rx_queues   :    0  nb_tx_queues      :    0  speed_capa    : 00005d60

   flow_type_rss_offloads:000000029e03fffc  reta_size             :    0
   rx_offload_capa       :VLAN_STRIP IPV4_CKSUM UDP_CKSUM TCP_CKSUM OUTER_IPV4_CKSUM VLAN_FILTER JUMBO_FRAME SCATTER TIMESTAMP SCTP_CKSUM OUTER_UDP_CKSUM 
   tx_offload_capa       :VLAN_INSERT IPV4_CKSUM UDP_CKSUM TCP_CKSUM SCTP_CKSUM OUTER_IPV4_CKSUM MT_LOCKFREE MULTI_SEGS MBUF_FAST_FREE 
   rx_queue_offload_capa :0000000000000000  tx_queue_offload_capa :0000000000000000
   dev_capa              :0000000000000000

  RX Conf:
     pthresh        :    0 hthresh          :    0 wthresh        :    0
     Free Thresh    :    0 Drop Enable      :    0 Deferred Start :    0
     offloads       :0000000000000000
  TX Conf:
     pthresh        :    0 hthresh          :    0 wthresh        :    0
     Free Thresh    :    0 RS Thresh        :    0 Deferred Start :    0
     offloads       :0000000000000000
  Rx: descriptor Limits
     nb_max         :65535  nb_min          :    0  nb_align      :    1
     nb_seg_max     :65535  nb_mtu_seg_max  :65535
  Tx: descriptor Limits
     nb_max         :65535  nb_min          :    0  nb_align      :    1
     nb_seg_max     :65535  nb_mtu_seg_max  :65535
  Rx: Port Config
     burst_size     :   32  ring_size       :  512  nb_queues     :    1
  Tx: Port Config
     burst_size     :   32  ring_size       :  512  nb_queues     :    1
  Switch Info: (null)
     domain_id      :65535  port_id         :    0

    Create: Default RX  1:0  - Memory used (MBUFs  4096 x (size 9746 + Hdr 128)) + 192 =  39497 KB, headroom 128
      Set RX queue stats mapping pid 1, q 0, lcore 3


    Create: Default TX  1:0  - Memory used (MBUFs  4096 x (size 9746 + Hdr 128)) + 192 =  39497 KB, headroom 128
    Create: Range TX    1:0  - Memory used (MBUFs  4096 x (size 9746 + Hdr 128)) + 192 =  39497 KB, headroom 128
    Create: Rate TX     1:0  - Memory used (MBUFs  4096 x (size 9746 + Hdr 128)) + 192 =  39497 KB, headroom 128
    Create: Sequence TX 1:0  - Memory used (MBUFs  4096 x (size 9746 + Hdr 128)) + 192 =  39497 KB, headroom 128
    Create: Special TX  1:0  - Memory used (MBUFs    64 x (size 9746 + Hdr 128)) + 192 =    618 KB, headroom 128

                                                                       Port memory used = 198103 KB
Src MAC d0:63:b4:03:2a:b3
 <Promiscuous mode Enabled> <Promiscuous mode Enabled>
Initialize Port 2 -- TxQ 1, RxQ 1
** Device Info (dpni.3, if_index:0, flags 00000066) **
   min_rx_bufsize :  512  max_rx_pktlen     :10240  hash_key_size :    0
   max_rx_queues  :  128  max_tx_queues     :   16  max_vfs       :    0
   max_mac_addrs  :   16  max_hash_mac_addrs:    0  max_vmdq_pools:   16
   vmdq_queue_base:    0  vmdq_queue_num    :    0  vmdq_pool_base:    0
   nb_rx_queues   :    0  nb_tx_queues      :    0  speed_capa    : 00005d60

   flow_type_rss_offloads:000000029e03fffc  reta_size             :    0
   rx_offload_capa       :VLAN_STRIP IPV4_CKSUM UDP_CKSUM TCP_CKSUM OUTER_IPV4_CKSUM VLAN_FILTER JUMBO_FRAME SCATTER TIMESTAMP SCTP_CKSUM OUTER_UDP_CKSUM 
   tx_offload_capa       :VLAN_INSERT IPV4_CKSUM UDP_CKSUM TCP_CKSUM SCTP_CKSUM OUTER_IPV4_CKSUM MT_LOCKFREE MULTI_SEGS MBUF_FAST_FREE 
   rx_queue_offload_capa :0000000000000000  tx_queue_offload_capa :0000000000000000
   dev_capa              :0000000000000000

  RX Conf:
     pthresh        :    0 hthresh          :    0 wthresh        :    0
     Free Thresh    :    0 Drop Enable      :    0 Deferred Start :    0
     offloads       :0000000000000000
  TX Conf:
     pthresh        :    0 hthresh          :    0 wthresh        :    0
     Free Thresh    :    0 RS Thresh        :    0 Deferred Start :    0
     offloads       :0000000000000000
  Rx: descriptor Limits
     nb_max         :65535  nb_min          :    0  nb_align      :    1
     nb_seg_max     :65535  nb_mtu_seg_max  :65535
  Tx: descriptor Limits
     nb_max         :65535  nb_min          :    0  nb_align      :    1
     nb_seg_max     :65535  nb_mtu_seg_max  :65535
  Rx: Port Config
     burst_size     :   32  ring_size       :  512  nb_queues     :    1
  Tx: Port Config
     burst_size     :   32  ring_size       :  512  nb_queues     :    1
  Switch Info: (null)
     domain_id      :65535  port_id         :    0

    Create: Default RX  2:0  - Memory used (MBUFs  4096 x (size 9746 + Hdr 128)) + 192 =  39497 KB, headroom 128
      Set RX queue stats mapping pid 2, q 0, lcore 5


    Create: Default TX  2:0  - Memory used (MBUFs  4096 x (size 9746 + Hdr 128)) + 192 =  39497 KB, headroom 128
    Create: Range TX    2:0  - Memory used (MBUFs  4096 x (size 9746 + Hdr 128)) + 192 =  39497 KB, headroom 128
    Create: Rate TX     2:0  - Memory used (MBUFs  4096 x (size 9746 + Hdr 128)) + 192 =  39497 KB, headroom 128
    Create: Sequence TX 2:0  - Memory used (MBUFs  4096 x (size 9746 + Hdr 128)) + 192 =  39497 KB, headroom 128
    Create: Special TX  2:0  - Memory used (MBUFs    64 x (size 9746 + Hdr 128)) + 192 =    618 KB, headroom 128

                                                                       Port memory used = 198103 KB
Src MAC d0:63:b4:03:2a:b4
 <Promiscuous mode Enabled> <Promiscuous mode Enabled> <Promiscuous mode Enabled>
Initialize Port 3 -- TxQ 1, RxQ 1
** Device Info (dpni.4, if_index:0, flags 00000066) **
   min_rx_bufsize :  512  max_rx_pktlen     :10240  hash_key_size :    0
   max_rx_queues  :  128  max_tx_queues     :   16  max_vfs       :    0
   max_mac_addrs  :   16  max_hash_mac_addrs:    0  max_vmdq_pools:   16
   vmdq_queue_base:    0  vmdq_queue_num    :    0  vmdq_pool_base:    0
   nb_rx_queues   :    0  nb_tx_queues      :    0  speed_capa    : 00005d60

   flow_type_rss_offloads:000000029e03fffc  reta_size             :    0
   rx_offload_capa       :VLAN_STRIP IPV4_CKSUM UDP_CKSUM TCP_CKSUM OUTER_IPV4_CKSUM VLAN_FILTER JUMBO_FRAME SCATTER TIMESTAMP SCTP_CKSUM OUTER_UDP_CKSUM 
   tx_offload_capa       :VLAN_INSERT IPV4_CKSUM UDP_CKSUM TCP_CKSUM SCTP_CKSUM OUTER_IPV4_CKSUM MT_LOCKFREE MULTI_SEGS MBUF_FAST_FREE 
   rx_queue_offload_capa :0000000000000000  tx_queue_offload_capa :0000000000000000
   dev_capa              :0000000000000000

  RX Conf:
     pthresh        :    0 hthresh          :    0 wthresh        :    0
     Free Thresh    :    0 Drop Enable      :    0 Deferred Start :    0
     offloads       :0000000000000000
  TX Conf:
     pthresh        :    0 hthresh          :    0 wthresh        :    0
     Free Thresh    :    0 RS Thresh        :    0 Deferred Start :    0
     offloads       :0000000000000000
  Rx: descriptor Limits
     nb_max         :65535  nb_min          :    0  nb_align      :    1
     nb_seg_max     :65535  nb_mtu_seg_max  :65535
  Tx: descriptor Limits
     nb_max         :65535  nb_min          :    0  nb_align      :    1
     nb_seg_max     :65535  nb_mtu_seg_max  :65535
  Rx: Port Config
     burst_size     :   32  ring_size       :  512  nb_queues     :    1
  Tx: Port Config
     burst_size     :   32  ring_size       :  512  nb_queues     :    1
  Switch Info: (null)
     domain_id      :65535  port_id         :    0

    Create: Default RX  3:0  - Memory used (MBUFs  4096 x (size 9746 + Hdr 128)) + 192 =  39497 KB, headroom 128
      Set RX queue stats mapping pid 3, q 0, lcore 7


    Create: Default TX  3:0  - Memory used (MBUFs  4096 x (size 9746 + Hdr 128)) + 192 =  39497 KB, headroom 128
    Create: Range TX    3:0  - Memory used (MBUFs  4096 x (size 9746 + Hdr 128)) + 192 =  39497 KB, headroom 128
    Create: Rate TX     3:0  - Memory used (MBUFs  4096 x (size 9746 + Hdr 128)) + 192 =  39497 KB, headroom 128
    Create: Sequence TX 3:0  - Memory used (MBUFs  4096 x (size 9746 + Hdr 128)) + 192 =  39497 KB, headroom 128
    Create: Special TX  3:0  - Memory used (MBUFs    64 x (size 9746 + Hdr 128)) + 192 =    618 KB, headroom 128

                                                                       Port memory used = 198103 KB
Src MAC d0:63:b4:03:2a:b5
 <Promiscuous mode Enabled> <Promiscuous mode Enabled> <Promiscuous mode Enabled> <Promiscuous mode Enabled>
                                                                      Total memory used = 792412 KB


=== Display processing on lcore 0
WARNING: Nothing to do on lcore 9: exiting
  RX processing lcore:   1 rx:  1 tx:  0
For RX found 1 port(s) for lcore 1
  TX processing lcore:   2 rx:  0 tx:  1
For TX found 1 port(s) for lcore 2

  RX processing lcore:   3 rx:  1 tx:  0
For RX found 1 port(s) for lcore 3
  TX processing lcore:   4 rx:  0 tx:  1
For TX found 1 port(s) for lcore 4

  RX processing lcore:   5 rx:  1 tx:  0
For RX found 1 port(s) for lcore 5
  TX processing lcore:   6 rx:  0 tx:  1
For TX found 1 port(s) for lcore 6

  RX processing lcore:   7 rx:  1 tx:  0
For RX found 1 port(s) for lcore 7
  TX processing lcore:   8 rx:  0 tx:  1
For TX found 1 port(s) for lcore 8

- Ports 0-3 of 4   <Main Page>  Copyright(c) <2010-2021>, Intel Corporation
  Flags:Port        : P------Sngl       :0 P------Sngl       :1 P------Sngl       :2 P------Sngl       :3
Link State          :           <--Down-->           <--Down-->           <--Down-->           <--Down-->     ---Total Rate---
Pkts/s Rx           :                    0                    0                    0                    0                    0
       Tx           :                    0                    0                    0                    0                    0
MBits/s Rx/Tx       :                  0/0                  0/0                  0/0                  0/0                  0/0
Pkts/s Rx Max       :                    0                    0                    0                    0                    0
       Tx Max       :                    0                    0                    0                    0                    0
Broadcast           :                    0                    0                    0                    0
Multicast           :                    0                    0                    0                    0
Sizes 64            :                    0                    0                    0                    0
      65-127        :                    0                    0                    0                    0
      128-255       :                    0                    0                    0                    0
      256-511       :                    0                    0                    0                    0
      512-1023      :                    0                    0                    0                    0
      1024-1518     :                    0                    0                    0                    0
Runts/Jumbos        :                  0/0                  0/0                  0/0                  0/0
ARP/ICMP Pkts       :                  0/0                  0/0                  0/0                  0/0
Errors Rx/Tx        :                  0/0                  0/0                  0/0                  0/0
Total Rx Pkts       :                    0                    0                    0                    0
      Tx Pkts       :                    0                    0                    0                    0
      Rx/Tx MBs     :                  0/0                  0/0                  0/0                  0/0
Pattern Type        :              abcd...              abcd...              abcd...              abcd...
Tx Count/% Rate     :        Forever /100%        Forever /100%        Forever /100%        Forever /100%
Pkt Size/Tx Burst   :            64 /   32            64 /   32            64 /   32            64 /   32
TTL/Port Src/Dest   :       64/ 1234/ 5678       64/ 1234/ 5678       64/ 1234/ 5678       64/ 1234/ 5678
Pkt Type:VLAN ID    :      IPv4 / UDP:0001      IPv4 / UDP:0001      IPv4 / UDP:0001      IPv4 / UDP:0001
802.1p CoS/DSCP/IPP :            0/  0/  0            0/  0/  0            0/  0/  0            0/  0/  0
VxLAN Flg/Grp/vid   :     0000/    0/    0     0000/    0/    0     0000/    0/    0     0000/    0/    0
IP  Destination     :          192.168.1.1          192.168.0.1          192.168.3.1          192.168.2.1
    Source          :       192.168.0.1/24       192.168.1.1/24       192.168.2.1/24       192.168.3.1/24
MAC Destination     :    d0:63:b4:03:2a:b3    d0:63:b4:03:2a:b2    d0:63:b4:03:2a:b5    d0:63:b4:03:2a:b4
    Source          :    d0:63:b4:03:2a:b2    d0:63:b4:03:2a:b3    d0:63:b4:03:2a:b4    d0:63:b4:03:2a:b5
PCI Vendor/Addr     :    0000:0000/00:00.0    0000:0000/00:00.0    0000:0000/00:00.0    0000:0000/00:00.0
-- Pktgen 21.03.0 (DPDK 20.11.2)  Powered by DPDK  (pid:1660) -----------------
Pktgen:/> set 2 proto udp
Pktgen:/> set 3 proto udp
Pktgen:/> start 0
Pktgen:/> start 1
Pktgen:/> start 2
Pktgen:/> start 3

Topic		Replies	Views
ClearFog LX2162A 1G Ethernet not working NXP LX2160	6	84	April 14, 2025
LX2162A clearfog SFP+ optical transceivers not working NXP LX2160	1	285	July 9, 2023
Clearfog base : problem with eth2 OCTEON CN9130	1	352	June 11, 2023
10G problem within QSFP connector NXP LX2160	1	361	December 30, 2022
ClearFog CX LX2 progress - what works for me NXP LX2160 NXP-LX2160	18	821	May 25, 2022

Eth0 not working (ClearFog / LX2160)

Related topics