SerDes configuration for SFP+ modules

NXP LX2160
1

I’ve been seeing some issues with use of the 4x SFP+ cages on the Clearfog board and have spent a fair bit of time getting these to work as expected with SFP+ (10Gbase-SR) transceivers. I thought I’d document what I’ve discovered here in case it helps someone or SolidRun want to modify the 10G SerDes bank 1 lane configuration to perform better on the ClearFog / Honeycomb carrier.

The issue I’ve seen is that the SFP+ cages perform poorly especially when used with optical transceivers. The two bottom ports (closer to the PCB) have worse performance than the top two and packet loss can often be seen when using these ports. We examined the signal eye diagram (a measure of signal integrity) using external equipment and could see little space in the middle of the eye when a transceiver is used. This indicates poor signal integrity. We’ve verified this poor performance across 4x Clearfog boards with 14x LX2160 CoMs.

When using protocol 8 for SerDes bank 1, the lanes are correctly configured for XFI/SFI by the SolidRun build environment. XFI is a MAC to phy protocol whereas SFI is used for a MAC to SFP+ protocol (i.e. what we need to drive an SFP module). They are very similar but electrically different. The LX2160 reference manual says the following RE operation using SFI:

“All default SerDes settings are for XFI/SFI. XFI and SFI may appear synonymous from a programming perspective, but they adhere to different electrical specifications and will need additional TX equalization tuning. Final design must consider hardware influences (trace length, trace materials, IBIS-AMI modeling, etc.) to meet specifications for proper operation.”

SerDes Lane m TX Equalization Register 0 (LNATECR0 - LNHTECR0) is particular interest to TX Equalization tuning. This register is configured at boot from the DPC (DataPath Configuration) device tree loaded by u-boot. The default DPC loaded by the lx2160a_build project contains the default register settings and does not attempt to tune SerDes TX equalization for the ClearFog carrier’s electrical characteristics.

I’ve tried changing the TX equalisation configuration experimentally and this fixes the issue for me and enables communication with SFP+ transceivers on all four cages without losing packets. There are better ways of achieving this and some soft of validation process should be followed for each SerDes lane in order to work out the correct equalization settings for that lane. That said, the following seems to work well for me and patching the DPC as follows for each of the SFP+ dpmacs results in improved performance (zero packet loss):

N.B. These settings are not optimal and were found experimentally rather than via a proper verification process. I hope to derive some better ones if I can get access to the required test equipment

            mac@** update mac in DPC as follows ** { 
                link_type = "MAC_LINK_TYPE_PHY";
                serdes_cfg = "custom";

                // EQ_TYPE
                // number of levels of TX Equalization
                serdes_eq = <1>;

                // EQ_PREQ
                // Drive strength of TX full swing transition
                // bit to precursor
                serdes_preq = <3>;

                // EQ_POST1Q
                // Drive strength of full swing transition bit
                // to first post-cursor
                serdes_post1q = <0x07>;

                // EQ_AMP_RED
                // Overall transmit amplitude reduction
                serdes_amp = <0x1f>;

                max_rate = <10000>;
            };

This can be applied by using dtc (device tree compiler) to decompile the DPC dtb:
sudo dd if=/dev/mmcblk0 bs=512 skip=28672 | dtc -I dtb -O dts > ./dpc.dts

then editing
vim ./dpc.dts # update the required macs as above

and re-compiling and dd’ing back to the appropriate location:
dtc -I dts -O dtb ./dpc.dts | dd of=/dev/mmcblk0 bs=512 seek=28672 conv=notrunc

1 Like