karim96
November 21, 2024, 10:50pm
1
Hi,
I have bought this box
I am trying to build the kernel and Ubuntu for it (using lx2160a_build git repo [1]), but I don’t know which SERDES configuration I should choose.
The default options build the kernel for HoneyComb, but I have ClearFog. So obviously I have to choose another TARGET.
Amongst the following the TARGETs|LX2160A_CEX7_CLEARFOG-CX_8_5_ |LX2160A_CEX7_CLEARFOG-CX_18_5_*)
Which one should I choose for my box?
[1] GitHub - SolidRun/lx2160a_build: Scripts to build basic images for LX2160A COM express type 7 modules
Thanks
well the configuration depends on how you intend to use the qsfp28 cage. Are you using a 4x25G breakout, or just as a single 100G connection? You can download the lx2160a reference manual from NXP and that has all the SERDES configurations listed there.
karim96
November 22, 2024, 8:57am
3
Hi @jnettlet . Thanks for your reply!
For reference, my board looks the same as the one shown at the top of this page [1]
Please forgive my ignorance. I might say something dumb, but I’m slightly new to this kind of technologies.
ATM, I want to use it as a storage server for another bigger machine. The bigger machine has a 40G NIC and I have two types of 40G cables. One is direct cable and another is a breakout cable 4x 10G cable.
I tried to create dpmac.1 (eth2 on qsfp28) and dpmac.2 (eth3 on 4x 10G SFP+). When I connect eth2 or eth3 to the bigger machine, the link on the bigger machine goes up, but on the Arm box it doesn’t.
so tldr, I want 40G connection. Is that possible? And what SERDES would work in this case?
Thanks again!
[1] https://solidrun.atlassian.net/wiki/spaces/developer/pages/197494288/HoneyComb+LX2+ClearFog+CX+LX2+Quick+Start+Guide
yes 40Gb on the qsfp28 is an option, but the configuration may require some slight changes. To just test the 40Gbps connection you can use protocol 20_5_2. From there to also support the 4x10Gbps it would require some RCW customizations.
karim96
November 22, 2024, 6:00pm
5
Hi @jnettlet
I choose 20_5_2 as per your suggestion. Resulting image islx2160acex7_2000_700_2900_20_5_2-37d25bc.img
However, upon connecting the 40G transceiver to the qsfp28 port, I get
root@localhost:~# [ 1390.338518] sfp qsfp-0: EEPROM base structure checksum failure: 0x13 != 0x00
[ 1390.345581] sfp EE: 00000000: 0d 00 06 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 1390.354302] sfp EE: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 1390.363008] sfp EE: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 1390.371710] sfp EE: 00000030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 1390.380411] sfp EE: 00000040: 18 b3 87 40 14 a3 ff ff 80 bd e8 14 00 80 ff ff ...@............
[ 1390.389111] sfp EE: 00000050: 68 8a 5b 40 14 a3 ff ff 90 bd e8 14 00 80 ff ff h.[@............
I tried with different cables, I get the same error. Do you have any clue why I’m getting this?
Thanks much!
can you run ethtool -m on that interface and post the output?
karim96
November 25, 2024, 8:48pm
7
Hi @jnettlet
No luck so far!
On the Arm box, I’m getting this
root@localhost:~# sudo ethtool -m eth1
Cannot get module EEPROM information: Operation not supported
dmesg from sfp module on the Arm box
root@localhost:~# sudo dmesg | grep sfp
[ 3.531590] sfp qsfp-0: Host maximum power 8.0W
[ 3.536222] sfp qsfp-0: No tx_disable pin: SFP modules will always be emitting.
[ 3.875370] sfp qsfp-0: EEPROM base structure checksum failure: 0x13 != 0x00
[ 3.886144] sfp EE: 00000000: 0d 00 06 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 3.886147] sfp EE: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 3.900831] sfp EE: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 3.900833] sfp EE: 00000030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 3.911346] sfp EE: 00000040: 18 b3 67 90 d3 c6 ff ff 80 3d 9b 11 00 80 ff ff ..g......=......
[ 3.911349] sfp EE: 00000050: 68 8a 3b 90 d3 c6 ff ff 90 3d 9b 11 00 80 ff ff h.;......=......
[ 5.988225] sfp sfp-0: Host maximum power 2.0W
[ 5.992807] sfp sfp-0: No tx_disable pin: SFP modules will always be emitting.
[ 6.004493] sfp sfp-1: Host maximum power 2.0W
[ 6.009066] sfp sfp-1: No tx_disable pin: SFP modules will always be emitting.
[ 6.020739] sfp sfp-2: Host maximum power 2.0W
[ 6.025306] sfp sfp-2: No tx_disable pin: SFP modules will always be emitting.
[ 6.036954] sfp sfp-3: Host maximum power 2.0W
[ 6.041526] sfp sfp-3: No tx_disable pin: SFP modules will always be emitting.
[ 274.693147] sfp qsfp-0: EEPROM base structure checksum failure: 0x13 != 0x00
[ 274.700220] sfp EE: 00000000: 0d 00 06 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
On the bigger machine, I have a Mellanox Connect X3 and ethtool -m returns
$ sudo ethtool -m ens31 [12/17787]
Identifier : 0x0d (QSFP+)
Extended identifier : 0x10
Extended identifier description : 1.5W max. Power consumption
Extended identifier description : No CDR in TX, No CDR in RX
Extended identifier description : High Power Class (> 3.5 W) not enabled
Power set : On
Power override : On
Connector : 0x23 (No separable connector)
Transceiver codes : 0x08 0x00 0x00 0x00 0x00 0x00 0x00 0x00
Transceiver type : 40G Ethernet: 40G Base-CR4
Encoding : 0x05 (64B/66B)
BR, Nominal : 10300Mbps
Rate identifier : 0x00
Length (SMF,km) : 0km
Length (OM3 50um) : 0m
Length (OM2 50um) : 0m
Length (OM1 62.5um) : 0m
Length (Copper or Active cable) : 2m
Transmitter technology : 0xa0 (Copper cable unequalized)
Attenuation at 2.5GHz : 4db
Attenuation at 5.0GHz : 6db
Attenuation at 7.0GHz : 0db
Attenuation at 12.9GHz : 0db
Vendor name : FS
Vendor OUI : 00:09:3a
Vendor PN : QSFP-PC02
Vendor rev : 03
Vendor SN : G1912189187-2
Date code : 20191227
Revision Compliance : Revision not specified
Module temperature : 0.00 degrees C / 32.00 degrees F
Module voltage : 0.0000 V
I am going to have to take a look at the device-tree fixups done for the SERDES configuration.
Hi @jnettlet how can I help with that?