We have observed similar problems with all Marvell devices we own (CN9130 Base, CN9130 Pro, GT-8K). For that we made a table of various combinations.
One key insight we had is that DACs seem to work without issue and the problems with optics seem to disappear when the the other side is NOT connected to another Marvell chip.
The disconnects were partially caused by the enclosure, since it doesn’t allow the sfp modules to fully seat.
I attached a table (hopefully it renders correctly) with a couple of combinations we tested. Columns that indicate disconnects have the same dmesg messages as above. In addition to that we observed CRC errors that cause packets to drop (around 0.1-1%). The errors look like:
[ 7221.999308] mvpp2 f2000000.ethernet eth0: Link is Up - 10Gbps/Full - flow control rx
[ 7232.691563] mvpp2 f2000000.ethernet eth0: Link is Down
[ 7241.237238] mvpp2 f2000000.ethernet eth0: Link is Up - 10Gbps/Full - flow control rx
[ 7262.187663] mvpp2 f2000000.ethernet eth0: bad rx status 43048a10 (crc error), size=1516
[ 7262.839740] mvpp2 f2000000.ethernet eth0: bad rx status 43048a10 (crc error), size=790
[ 7263.548694] mvpp2 f2000000.ethernet eth0: bad rx status 43048a10 (crc error), size=1516
[ 7263.974951] mvpp2 f2000000.ethernet eth0: bad rx status 43048a10 (crc error), size=1516
[ 7265.089051] mvpp2 f2000000.ethernet eth0: bad rx status 43048a10 (crc error), size=1516
[ 7266.933865] mvpp2 f2000000.ethernet eth0: bad rx status 43048a10 (crc error), size=1516
Summary
| Clearfog | SFP Modul | Modul Firmware | Detected | Medium | SFP Modul | Modul Firmware | Detected | Peer | Link? | Disconnects? | CRC Error | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CN9130 Pro | Flexoptix P.1396.10 | Nvidia MFM1T02A-LR | Yes | SM | Flexoptix P.1396.10 | MikroTik S+31DLC10D | Yes | CR328-24P-4S+RM | Yes | Yes | ||
| CN9130 Pro | SFP-10GLRM-31 | MK | Yes | SM | SFP-10GLRM-31 | Generic | Yes | CR328-24P-4S+RM | Yes | Yes | ||
| CN9130 Pro | Finisar FTLX8571D3BCL | Generic | Yes | MM | Finisar FTLX8571D3BCL | Generic | Yes | CR328-24P-4S+RM | Yes | Yes | ||
| CN9130 Pro | Flexoptix P.C30.1 | Generic | Yes | DAC | Flexoptix P.C30.1 | Generic | Yes | CR328-24P-4S+RM | No | |||
| CN9130 Pro | Flexoptix P.1396.10 | Nvidia MFM1T02A-LR | Yes | SM | Flexoptix P.1396.10 | Aruba J9152A | Yes | Aruba 2930F JL258A | Yes | No | ||
| CN9130 Base | Flexoptix P.1396.10 | Nvidia MFM1T02A-LR | Yes | SM | Flexoptix P.1396.10 | MikroTik S+31DLC10D | Yes | CR328-24P-4S+RM | Yes | Yes | ||
| CN9130 Base | Flexoptix P.1396.10 | Intel E10GSFPLR | Yes | SM | Flexoptix P.1396.10 | MikroTik S+31DLC10D | Yes | CR328-24P-4S+RM | Yes | Yes | ||
| CN9130 Base | Flexoptix P.1396.10 | Intel E10GSFPSR | Yes | SM | Flexoptix P.1396.10 | MikroTik S+31DLC10D | Yes | CR328-24P-4S+RM | Yes | Yes | ||
| CN9130 Base | Flexoptix P.1396.10 | MSA Standard P.1396.10 | Yes | SM | Flexoptix P.1396.10 | MikroTik S+31DLC10D | Yes | CR328-24P-4S+RM | Yes | Yes | ||
| CN9130 Base | Flexoptix P.1396.10 | Penguin Computing SFP+ LR | Yes | SM | Flexoptix P.1396.10 | MikroTik S+31DLC10D | Yes | CR328-24P-4S+RM | Yes | Yes | ||
| CN9130 Base | Flexoptix P.1396.10 | Quanta SFP+ LR | Yes | SM | Flexoptix P.1396.10 | MikroTik S+31DLC10D | Yes | CR328-24P-4S+RM | Yes | Yes | ||
| CN9130 Base | Flexoptix P.1396.10 | Synology SFP+ LR | Yes | SM | Flexoptix P.1396.10 | MikroTik S+31DLC10D | Yes | CR328-24P-4S+RM | Yes | Yes | ||
| CN9130 Base | Flexoptix P.1396.10 | Zyxel SFP10G-LR | Yes | SM | Flexoptix P.1396.10 | MikroTik S+31DLC10D | Yes | CR328-24P-4S+RM | Yes | Yes | ||
| CN9130 Base | Finisar FTLX8571D3BCL | Generic | Yes | SM | Finisar FTLX8571D3BCL | Generic | Yes | CR328-24P-4S+RM | Yes | Yes | ||
| CN9130 Pro | Flexoptix P.C30.1 | Nvidia MC3309130-001 | DAC | Flexoptix P.C30.1 | Aruba J9281B | Aruba 2930F JL258A | ||||||
| CN9130 Base | Flexoptix P.1396.10 | Nvidia MFM1T02A-LR | SM | Flexoptix P.1396.10 | Aruba J9152A | Aruba 2930F JL258A | ||||||
| CN9130 Base | Flexoptix P.1396.10 | Ubiquiti | Yes | SMF | Flexoptix P.1396.10 | Ubiquiti | Yes | CN9130 Pro | Yes | No | Yes | |
| CN9130 Base | Flexoptix P.1396.10 | Mellanox | Yes | SMF | Flexoptix P.1396.10 | Mellanox | Yes | CN9130 Pro | Yes | No | Yes | |
| CN9130 Base | Flexoptix P.1396.10 | ZTE | Yes | SMF | Flexoptix P.1396.10 | ZTE | Yes | CN9130 Pro | Yes | No | Yes | |
| CN9130 Base | Flexoptix P.1396.10 | Mikrotik | Yes | SMF | Flexoptix P.1396.10 | Mikrotik | Yes | CN9130 Pro | No | |||
| CN9130 Base | Flexoptix P.1396.10 | Intel | Yes | SMF | Flexoptix P.1396.10 | Intel | Yes | CN9130 Pro | No | |||
| CN9130 Base | Flexoptix P.1396.10 | Ubiquiti 1G | Yes | SMF | Flexoptix P.1396.10 | Ubiquiti 1G | Yes | CN9130 Pro | No | |||
| CN9130 Base | FS SFPP-PC01 | Generic | Yes | DAC | FS SFPP-PC01 | Generic | Yes | CN9130 Pro | Yes | No | No | |
| CN9130 Base | SFP-10GLRM-31 | Generic | Yes | SMF | SFP-10GLRM-31 | Generic | Yes | CN9130 Pro | Yes | No | Yes | |
| CN9130 Base | SFP-10GLRM-31 | Generic | Yes | SMF | SFP-10GLRM-31 | Generic | Yes | Mellanox Connect-X 3 | Yes | No | No | |
| CN9130 Base | Flexoptix P.1396.10 | Ubiquiti | Yes | SMF | Flexoptix P.1396.10 | Ubiquiti | Yes | Mellanox Connect-X 3 | Yes | No | No | |
| CN9130 Base | Flexoptix P.1396.10 | Mellanox | Yes | SMF | Flexoptix P.1396.10 | Mellanox | Yes | Mellanox Connect-X 3 | Yes | No | No | |
| CN9130 Base | Flexoptix P.1396.10 | ZTE | Yes | SMF | Flexoptix P.1396.10 | ZTE | Yes | Mellanox Connect-X 3 | Yes | No | No |