Interesting. Was the MikroTik switch logging any errors?
Sorry, took me a bit:
dmesg
[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
[ 0.000000] Linux version 6.2.8-arm64+ (fbl@fbl-xps) (aarch64-linux-gnu-gcc (GCC) 12.2.0, GNU ld (GNU Binutils) 2.39) #5 SMP Fri Mar 31 21:32:59 CEST 2023
[ 0.000000] Machine model: SolidRun CN9130 based SOM Clearfog Pro
[ 0.000000] efi: UEFI not found.
[ 0.000000] NUMA: No NUMA configuration found
[ 0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x000000013fffffff]
[ 0.000000] NUMA: NODE_DATA [mem 0x13f7da900-0x13f7dcfff]
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x0000000000000000-0x00000000ffffffff]
[ 0.000000] DMA32 empty
[ 0.000000] Normal [mem 0x0000000100000000-0x000000013fffffff]
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000000000000-0x0000000003ffffff]
[ 0.000000] node 0: [mem 0x0000000004000000-0x00000000041fffff]
[ 0.000000] node 0: [mem 0x0000000004200000-0x00000000043fffff]
[ 0.000000] node 0: [mem 0x0000000004400000-0x00000000053fffff]
[ 0.000000] node 0: [mem 0x0000000005400000-0x00000000bfffffff]
[ 0.000000] node 0: [mem 0x0000000100000000-0x000000013fffffff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x000000013fffffff]
[ 0.000000] cma: Reserved 64 MiB at 0x00000000bc000000
[ 0.000000] psci: probing for conduit method from DT.
[ 0.000000] psci: PSCIv1.1 detected in firmware.
[ 0.000000] psci: Using standard PSCI v0.2 function IDs
[ 0.000000] psci: MIGRATE_INFO_TYPE not supported.
[ 0.000000] psci: SMC Calling Convention v1.2
[ 0.000000] percpu: Embedded 30 pages/cpu s84328 r8192 d30360 u122880
[ 0.000000] pcpu-alloc: s84328 r8192 d30360 u122880 alloc=30*4096
[ 0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3
[ 0.000000] Detected PIPT I-cache on CPU0
[ 0.000000] CPU features: detected: Spectre-v2
[ 0.000000] CPU features: detected: Spectre-v3a
[ 0.000000] CPU features: detected: Spectre-BHB
[ 0.000000] CPU features: detected: ARM erratum 1742098
[ 0.000000] CPU features: detected: ARM errata 1165522, 1319367, or 1530923
[ 0.000000] alternatives: applying boot alternatives
[ 0.000000] Fallback order for Node 0: 0
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1032192
[ 0.000000] Policy zone: Normal
[ 0.000000] Kernel command line: log_level=7 net.ifnames=0
[ 0.000000] Unknown kernel command line parameters "log_level=7", will be passed to user space.
[ 0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[ 0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.000000] mem auto-init: stack:off, heap alloc:on, heap free:off
[ 0.000000] software IO TLB: area num 4.
[ 0.000000] software IO TLB: mapped [mem 0x00000000b8000000-0x00000000bc000000] (64MB)
[ 0.000000] Memory: 3072824K/4194304K available (12672K kernel code, 2696K rwdata, 4372K rodata, 6208K init, 592K bss, 219856K reserved, 65536K cma-reserved)
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[ 0.000000] ftrace: allocating 42168 entries in 165 pages
[ 0.000000] ftrace: allocated 165 pages with 4 groups
[ 0.000000] trace event string verifier disabled
[ 0.000000] rcu: Hierarchical RCU implementation.
[ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=4.
[ 0.000000] Rude variant of Tasks RCU enabled.
[ 0.000000] Tracing variant of Tasks RCU enabled.
[ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[ 0.000000] GIC: Adjusting CPU interface base to 0x00000000f022f000
[ 0.000000] Root IRQ handler: gic_handle_irq
[ 0.000000] GIC: Using split EOI/Deactivate mode
[ 0.000000] GICv2m: DT overriding V2M MSI_TYPER (base:160, num:32)
[ 0.000000] GICv2m: range[mem 0xf0280000-0xf0280fff], SPI[160:191]
[ 0.000000] GICv2m: DT overriding V2M MSI_TYPER (base:192, num:32)
[ 0.000000] GICv2m: range[mem 0xf0290000-0xf0290fff], SPI[192:223]
[ 0.000000] GICv2m: DT overriding V2M MSI_TYPER (base:224, num:32)
[ 0.000000] GICv2m: range[mem 0xf02a0000-0xf02a0fff], SPI[224:255]
[ 0.000000] GICv2m: DT overriding V2M MSI_TYPER (base:256, num:32)
[ 0.000000] GICv2m: range[mem 0xf02b0000-0xf02b0fff], SPI[256:287]
[ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[ 0.000000] arch_timer: cp15 timer(s) running at 25.00MHz (phys).
[ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x5c40939b5, max_idle_ns: 440795202646 ns
[ 0.000000] sched_clock: 56 bits at 25MHz, resolution 40ns, wraps every 4398046511100ns
[ 0.000218] Console: colour dummy device 80x25
[ 0.000224] printk: console [tty0] enabled
[ 0.000437] Calibrating delay loop (skipped), value calculated using timer frequency.. 50.00 BogoMIPS (lpj=100000)
[ 0.000447] pid_max: default: 32768 minimum: 301
[ 0.000560] LSM: initializing lsm=lockdown,capability,landlock,yama,integrity,apparmor,tomoyo,bpf
[ 0.000586] landlock: Up and running.
[ 0.000590] Yama: becoming mindful.
[ 0.000659] AppArmor: AppArmor initialized
[ 0.000667] TOMOYO Linux initialized
[ 0.000689] LSM support for eBPF active
[ 0.000766] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.000779] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.001694] cblist_init_generic: Setting adjustable number of callback queues.
[ 0.001708] cblist_init_generic: Setting shift to 2 and lim to 1.
[ 0.001750] cblist_init_generic: Setting shift to 2 and lim to 1.
[ 0.001849] rcu: Hierarchical SRCU implementation.
[ 0.001855] rcu: Max phase no-delay instances is 1000.
[ 0.002616] EFI services will not be available.
[ 0.002838] smp: Bringing up secondary CPUs ...
[ 0.003341] Detected PIPT I-cache on CPU1
[ 0.003404] CPU1: Booted secondary processor 0x0000000001 [0x410fd083]
[ 0.003927] Detected PIPT I-cache on CPU2
[ 0.003984] CPU2: Booted secondary processor 0x0000000100 [0x410fd083]
[ 0.004500] Detected PIPT I-cache on CPU3
[ 0.004541] CPU3: Booted secondary processor 0x0000000101 [0x410fd083]
[ 0.004580] smp: Brought up 1 node, 4 CPUs
[ 0.004604] SMP: Total of 4 processors activated.
[ 0.004609] CPU features: detected: 32-bit EL0 Support
[ 0.004613] CPU features: detected: 32-bit EL1 Support
[ 0.004618] CPU features: detected: CRC32 instructions
[ 0.004658] CPU: All CPU(s) started at EL2
[ 0.004674] alternatives: applying system-wide alternatives
[ 0.012617] node 0 deferred pages initialised in 8ms
[ 0.013346] devtmpfs: initialized
[ 0.016286] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 0.016309] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
[ 0.016889] pinctrl core: initialized pinctrl subsystem
[ 0.017199] DMI not present or invalid.
[ 0.017509] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[ 0.018086] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations
[ 0.018257] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[ 0.018483] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[ 0.018532] audit: initializing netlink subsys (disabled)
[ 0.018638] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1
[ 0.018996] thermal_sys: Registered thermal governor 'fair_share'
[ 0.019000] thermal_sys: Registered thermal governor 'bang_bang'
[ 0.019007] thermal_sys: Registered thermal governor 'step_wise'
[ 0.019011] thermal_sys: Registered thermal governor 'user_space'
[ 0.019015] thermal_sys: Registered thermal governor 'power_allocator'
[ 0.019039] cpuidle: using governor ladder
[ 0.019052] cpuidle: using governor menu
[ 0.019109] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[ 0.019159] ASID allocator initialised with 65536 entries
[ 0.019323] Serial: AMBA PL011 UART driver
[ 0.025015] KASLR disabled due to lack of seed
[ 0.025459] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
[ 0.025468] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page
[ 0.025474] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages
[ 0.025478] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page
[ 0.025483] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
[ 0.025487] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page
[ 0.025492] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages
[ 0.025496] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page
[ 0.033834] ACPI: Interpreter disabled.
[ 0.034016] iommu: Default domain type: Translated
[ 0.034024] iommu: DMA domain TLB invalidation policy: strict mode
[ 0.034135] EDAC MC: Ver: 3.0.0
[ 0.034800] NetLabel: Initializing
[ 0.034807] NetLabel: domain hash size = 128
[ 0.034811] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO
[ 0.034840] NetLabel: unlabeled traffic allowed by default
[ 0.034941] vgaarb: loaded
[ 0.035078] clocksource: Switched to clocksource arch_sys_counter
[ 0.035314] VFS: Disk quotas dquot_6.6.0
[ 0.035346] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.035707] AppArmor: AppArmor Filesystem Enabled
[ 0.035744] pnp: PnP ACPI: disabled
[ 0.039208] NET: Registered PF_INET protocol family
[ 0.039356] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear)
[ 0.040853] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[ 0.040912] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[ 0.040979] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[ 0.041182] TCP bind hash table entries: 32768 (order: 8, 1048576 bytes, linear)
[ 0.041578] TCP: Hash tables configured (established 32768 bind 32768)
[ 0.041728] MPTCP token hash table entries: 4096 (order: 4, 98304 bytes, linear)
[ 0.041917] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 0.041973] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 0.042116] NET: Registered PF_UNIX/PF_LOCAL protocol family
[ 0.042139] NET: Registered PF_XDP protocol family
[ 0.042147] PCI: CLS 0 bytes, default 64
[ 0.042423] Trying to unpack rootfs image as initramfs...
[ 0.047702] kvm [1]: IPA Size Limit: 44 bits
[ 0.055297] kvm [1]: vgic interrupt IRQ9
[ 0.055670] kvm [1]: Hyp mode initialized successfully
[ 0.056393] Initialise system trusted keyrings
[ 0.056425] Key type blacklist registered
[ 0.056559] workingset: timestamp_bits=42 max_order=20 bucket_order=0
[ 0.056594] zbud: loaded
[ 0.056891] integrity: Platform Keyring initialized
[ 0.056900] Key type asymmetric registered
[ 0.056905] Asymmetric key parser 'x509' registered
[ 0.680536] Freeing initrd memory: 27768K
[ 0.689231] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 249)
[ 0.689330] io scheduler mq-deadline registered
[ 0.693108] armada-ap806-pinctrl f06f4000.system-controller:pinctrl: registered pinctrl driver
[ 0.693490] armada-cp110-pinctrl f2440000.system-controller:pinctrl: registered pinctrl driver
[ 0.694501] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[ 0.697088] mv_xor_v2 f0400000.xor: Marvell Version 2 XOR driver
[ 0.697380] mv_xor_v2 f0420000.xor: Marvell Version 2 XOR driver
[ 0.697686] mv_xor_v2 f0440000.xor: Marvell Version 2 XOR driver
[ 0.697964] mv_xor_v2 f0460000.xor: Marvell Version 2 XOR driver
[ 0.698555] mv_xor_v2 f26a0000.xor: Marvell Version 2 XOR driver
[ 0.698859] mv_xor_v2 f26c0000.xor: Marvell Version 2 XOR driver
[ 0.699982] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[ 0.720949] f0512000.serial: ttyS0 at MMIO 0xf0512000 (irq = 22, base_baud = 12500000) is a 16550A
[ 0.720981] printk: console [ttyS0] enabled
[ 1.801045] Serial: AMBA driver
[ 1.804349] msm_serial: driver initialized
[ 1.809406] mousedev: PS/2 mouse device common for all mice
[ 1.815867] ledtrig-cpu: registered to indicate activity on CPUs
[ 1.822037] SMCCC: SOC_ID: ARCH_SOC_ID not implemented, skipping ....
[ 1.829251] NET: Registered PF_INET6 protocol family
[ 1.841517] Segment Routing with IPv6
[ 1.845247] In-situ OAM (IOAM) with IPv6
[ 1.849225] mip6: Mobile IPv6
[ 1.852214] NET: Registered PF_PACKET protocol family
[ 1.857356] mpls_gso: MPLS GSO support
[ 1.863424] registered taskstats version 1
[ 1.867625] Loading compiled-in X.509 certificates
[ 1.900453] Loaded X.509 cert 'Build time autogenerated kernel key: 6c9fc8e59db3b67dca1ad08066f15ab7c231ebd2'
[ 1.911155] zswap: loaded using pool lzo/zbud
[ 1.918377] Key type .fscrypt registered
[ 1.922333] Key type fscrypt-provisioning registered
[ 1.934139] Key type encrypted registered
[ 1.938185] AppArmor: AppArmor sha1 policy hashing enabled
[ 1.943715] ima: No TPM chip found, activating TPM-bypass!
[ 1.949240] ima: Allocated hash algorithm: sha256
[ 1.953990] ima: No architecture policies found
[ 1.958565] evm: Initialising EVM extended attributes:
[ 1.963728] evm: security.selinux
[ 1.967056] evm: security.SMACK64 (disabled)
[ 1.971347] evm: security.SMACK64EXEC (disabled)
[ 1.975985] evm: security.SMACK64TRANSMUTE (disabled)
[ 1.981060] evm: security.SMACK64MMAP (disabled)
[ 1.985699] evm: security.apparmor
[ 1.989117] evm: security.ima
[ 1.992100] evm: security.capability
[ 1.995693] evm: HMAC attrs: 0x1
[ 2.150310] hw perfevents: enabled with armv8_cortex_a72 PMU driver, 7 counters available
[ 2.158779] gpio gpiochip0: Static allocation of GPIO base is deprecated, use dynamic allocation.
[ 2.168061] gpio gpiochip1: Static allocation of GPIO base is deprecated, use dynamic allocation.
[ 2.177007] gpio-59 (phy-reset): hogged as output/high
[ 2.182541] gpio gpiochip2: Static allocation of GPIO base is deprecated, use dynamic allocation.
[ 2.194830] Freeing unused kernel memory: 6208K
[ 2.232121] Checked W+X mappings: passed, no W+X pages found
[ 2.237830] Run /init as init process
[ 2.241513] with arguments:
[ 2.241516] /init
[ 2.241519] with environment:
[ 2.241521] HOME=/
[ 2.241523] TERM=linux
[ 2.241526] log_level=7
[ 2.372753] ap0_sd_vccq: Bringing 3300000uV into 1800000-1800000uV
[ 2.384696] sdhci: Secure Digital Host Controller Interface driver
[ 2.390955] sdhci: Copyright(c) Pierre Ossman
[ 2.396936] sdhci-pltfm: SDHCI platform and OF driver helper
[ 2.406830] usb_phy_generic cp0-usb3-phy@0: dummy supplies not allowed for exclusive requests
[ 2.415923] usb_phy_generic cp0-usb3-phy@1: dummy supplies not allowed for exclusive requests
[ 2.427866] xenon-sdhci f2780000.mmc: Got CD GPIO
[ 2.433951] pca953x 0-0020: supply vcc not found, using dummy regulator
[ 2.435136] armada38x-rtc f2284000.rtc: registered as rtc0
[ 2.443920] pca953x 0-0020: using no AI
[ 2.447442] armada38x-rtc f2284000.rtc: setting system clock to 2105-07-21T01:33:39 UTC (4277583219)
[ 2.450013] mmc0: SDHCI controller on f06e0000.mmc [f06e0000.mmc] using ADMA 64-bit
[ 2.452463] gpio-95 (pcie1.0-clkreq): hogged as input
[ 2.453501] gpio-98 (pcie1.0-w-disable): hogged as output/low
[ 2.453812] gpio-99 (pcie2.0-clkreq): hogged as input
[ 2.454432] gpio-102 (pcie2.0-w-disable): hogged as output/low
[ 2.454743] gpio-100 (usb3-current-limit): hogged as input
[ 2.455363] gpio-101 (usb3-power): hogged as output/high
[ 2.456821] gpio-106 (m.2 devslp): hogged as output/low
[ 2.459467] armada8k-pcie f2620000.pcie: host bridge /cp0/pcie@f2620000 ranges:
[ 2.475274] mmc1: SDHCI controller on f2780000.mmc [f2780000.mmc] using ADMA 64-bit
[ 2.477843] armada8k-pcie f2620000.pcie: MEM 0x00e0000000..0x00e0efffff -> 0x00e0000000
[ 2.521297] SCSI subsystem initialized
[ 2.524086] mmc1: new high speed SDHC card at address 5048
[ 2.528592] armada8k-pcie f2620000.pcie: iATU: unroll F, 8 ob, 8 ib, align 64K, limit 4G
[ 2.528776] mmcblk1: mmc1:5048 SD16G 14.4 GiB
[ 2.531032] usbcore: registered new interface driver usbfs
[ 2.531306] usbcore: registered new interface driver hub
[ 2.531348] usbcore: registered new device driver usb
[ 2.532283] mmcblk1: p1
[ 2.550444] sfp sfp-eth@0: Host maximum power 2.0W
[ 2.556551] xhci-hcd f2500000.usb: xHCI Host Controller
[ 2.558093] libata version 3.00 loaded.
[ 2.569283] ahci f2540000.sata: supply ahci not found, using dummy regulator
[ 2.573899] xhci-hcd f2500000.usb: new USB bus registered, assigned bus number 1
[ 2.579188] ahci f2540000.sata: supply phy not found, using dummy regulator
[ 2.586396] xhci-hcd f2500000.usb: hcc params 0x0a000990 hci version 0x100 quirks 0x0000000000010010
[ 2.586880] mmc0: new HS200 MMC card at address 0001
[ 2.587548] mmcblk0: mmc0:0001 8GTF4R 7.28 GiB
[ 2.589979] mmcblk0: p1
[ 2.590346] mmcblk0boot0: mmc0:0001 8GTF4R 4.00 MiB
[ 2.591255] mmcblk0boot1: mmc0:0001 8GTF4R 4.00 MiB
[ 2.592840] mmcblk0rpmb: mmc0:0001 8GTF4R 512 KiB, chardev (242:0)
[ 2.595143] hwmon hwmon0: temp1_input not attached to any thermal zone
[ 2.595728] platform f2540000.sata:sata-port@0: supply target not found, using dummy regulator
[ 2.595966] platform f2540000.sata:sata-port@1: supply target not found, using dummy regulator
[ 2.596114] ahci f2540000.sata: masking port_map 0x3 -> 0x3
[ 2.596144] ahci f2540000.sata: AHCI 0001.0000 32 slots 2 ports 6 Gbps 0x3 impl platform mode
[ 2.596149] ahci f2540000.sata: flags: 64bit ncq sntf led only pmp fbs pio slum part sxs
[ 2.600636] xhci-hcd f2500000.usb: irq 39, io mem 0xf2500000
[ 2.632974] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
[ 2.638210] xhci-hcd f2500000.usb: xHCI Host Controller
[ 2.685488] mv88e6085 f212a200.mdio-mii:04: switch 0x1760 detected: Marvell 88E6176, revision 1
[ 2.690106] xhci-hcd f2500000.usb: new USB bus registered, assigned bus number 2
[ 2.724613] xhci-hcd f2500000.usb: Host supports USB 3.0 SuperSpeed
[ 2.725416] scsi host0: ahci
[ 2.731037] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.02
[ 2.734276] scsi host1: ahci
[ 2.742160] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.745198] ata1: SATA max UDMA/133 mmio [mem 0xf2540000-0xf256ffff] port 0x100 irq 40
[ 2.752315] usb usb1: Product: xHCI Host Controller
[ 2.752319] usb usb1: Manufacturer: Linux 6.2.8-arm64+ xhci-hcd
[ 2.752322] usb usb1: SerialNumber: f2500000.usb
[ 2.760274] ata2: SATA max UDMA/133 mmio [mem 0xf2540000-0xf256ffff] port 0x180 irq 40
[ 2.765280] mvpp2 f2000000.ethernet: using 8 per-cpu buffers
[ 2.765923] hub 1-0:1.0: USB hub found
[ 2.765942] hub 1-0:1.0: 1 port detected
[ 2.766387] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
[ 2.766435] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 6.02
[ 2.766439] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.766442] usb usb2: Product: xHCI Host Controller
[ 2.766445] usb usb2: Manufacturer: Linux 6.2.8-arm64+ xhci-hcd
[ 2.766447] usb usb2: SerialNumber: f2500000.usb
[ 2.766899] hub 2-0:1.0: USB hub found
[ 2.766923] hub 2-0:1.0: 1 port detected
[ 2.878106] mvpp2 f2000000.ethernet eth0: Using random mac address 2e:1d:ed:43:1f:27
[ 2.898281] sfp sfp-eth@0: module FiberStore SFP-10GSR-85 rev sn F171CO31012 dc 170109
[ 2.908572] mvpp2 f2000000.ethernet eth1: Using random mac address 16:01:24:aa:45:8e
[ 2.918058] mvpp2 f2000000.ethernet eth2: Using random mac address b6:eb:3e:56:30:66
[ 2.935177] hwmon hwmon1: temp1_input not attached to any thermal zone
[ 3.085269] ata1: SATA link down (SStatus 0 SControl 300)
[ 3.090729] ata2: SATA link down (SStatus 0 SControl 300)
[ 3.348957] EXT4-fs (mmcblk1p1): mounted filesystem f08ddc42-9533-4f8d-b672-9a6816266c22 with ordered data mode. Quota mode: none.
[ 3.477251] Not activating Mandatory Access Control as /sbin/tomoyo-init does not exist.
[ 3.531104] armada8k-pcie f2620000.pcie: Phy link never came up
[ 3.537212] armada8k-pcie f2620000.pcie: PCI host bridge to bus 0000:00
[ 3.543866] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 3.549382] pci_bus 0000:00: root bus resource [mem 0xe0000000-0xe0efffff]
[ 3.556309] pci 0000:00:00.0: [11ab:0110] type 01 class 0x060400
[ 3.562353] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[ 3.568697] pci 0000:00:00.0: supports D1 D2
[ 3.572989] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[ 3.579416] pci 0000:00:00.0: BAR 0: assigned [mem 0xe0000000-0xe00fffff]
[ 3.586246] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[ 4.206156] pcieport 0000:00:00.0: PME: Signaling with IRQ 37
[ 4.212089] pcieport 0000:00:00.0: AER: enabled with IRQ 37
[ 4.218067] armada8k-pcie f2640000.pcie: host bridge /cp0/pcie@f2640000 ranges:
[ 4.225442] armada8k-pcie f2640000.pcie: MEM 0x00e1000000..0x00e1efffff -> 0x00e1000000
[ 4.233956] armada8k-pcie f2640000.pcie: iATU: unroll F, 8 ob, 8 ib, align 64K, limit 4G
[ 4.624162] systemd[1]: Inserted module 'autofs4'
[ 4.691020] systemd[1]: systemd 247.3-6 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +ZSTD +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=unified)
[ 4.713645] systemd[1]: Detected architecture arm64.
[ 4.740065] systemd[1]: Set hostname to <sr-cn913x>.
[ 5.067503] systemd[1]: Queued start job for default target Multi-User System.
[ 5.239083] armada8k-pcie f2640000.pcie: Phy link never came up
[ 5.245144] armada8k-pcie f2640000.pcie: PCI host bridge to bus 0001:00
[ 5.251797] pci_bus 0001:00: root bus resource [bus 00-ff]
[ 5.257313] pci_bus 0001:00: root bus resource [mem 0xe1000000-0xe1efffff]
[ 5.264241] pci 0001:00:00.0: [11ab:0110] type 01 class 0x060400
[ 5.270285] pci 0001:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[ 5.276629] pci 0001:00:00.0: supports D1 D2
[ 5.280922] pci 0001:00:00.0: PME# supported from D0 D1 D3hot
[ 5.287377] pci 0001:00:00.0: BAR 0: assigned [mem 0xe1000000-0xe10fffff]
[ 5.294205] pci 0001:00:00.0: PCI bridge to [bus 01-ff]
[ 5.914172] pcieport 0001:00:00.0: PME: Signaling with IRQ 56
[ 5.920100] pcieport 0001:00:00.0: AER: enabled with IRQ 56
[ 5.926081] mv88e6085 f212a200.mdio-mii:04: switch 0x1760 detected: Marvell 88E6176, revision 1
[ 6.691420] mv88e6085 f212a200.mdio-mii:04: OF node /cp0/config-space@f2000000/mdio@12a200/switch@4/ports/port@5 of CPU port 5 lacks the required "phy-mode" property
[ 6.714663] mv88e6085 f212a200.mdio-mii:04: configuring for fixed/ link mode
[ 6.725597] mv88e6085 f212a200.mdio-mii:04: Link is Up - 1Gbps/Full - flow control off
[ 6.765845] mv88e6085 f212a200.mdio-mii:04 lan5 (uninitialized): PHY [mv88e6xxx-1:00] driver [Marvell 88E1540] (irq=66)
[ 6.778388] mvpp2 f2000000.ethernet: all ports have a low MTU, switching to per-cpu buffers
[ 6.803192] mv88e6085 f212a200.mdio-mii:04: Link is Down
[ 6.808195] mvpp2 f2000000.ethernet: using 8 per-cpu buffers
[ 6.812388] mv88e6085 f212a200.mdio-mii:04: Link is Up - 1Gbps/Full - flow control off
[ 6.906599] mv88e6085 f212a200.mdio-mii:04 lan4 (uninitialized): PHY [mv88e6xxx-1:01] driver [Marvell 88E1540] (irq=67)
[ 6.986171] mv88e6085 f212a200.mdio-mii:04 lan3 (uninitialized): PHY [mv88e6xxx-1:02] driver [Marvell 88E1540] (irq=68)
[ 7.065847] mv88e6085 f212a200.mdio-mii:04 lan2 (uninitialized): PHY [mv88e6xxx-1:03] driver [Marvell 88E1540] (irq=69)
[ 7.147085] mv88e6085 f212a200.mdio-mii:04 lan1 (uninitialized): PHY [mv88e6xxx-1:04] driver [Marvell 88E1540] (irq=70)
[ 7.161410] DSA: tree 0 setup
[ 8.107083] random: crng init done
[ 8.124016] systemd[1]: Created slice system-getty.slice.
[ 8.147672] systemd[1]: Created slice system-modprobe.slice.
[ 8.167595] systemd[1]: Created slice system-serial\x2dgetty.slice.
[ 8.191268] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.
[ 8.215219] systemd[1]: Started Forward Password Requests to Wall Directory Watch.
[ 8.239436] systemd[1]: Set up automount Arbitrary Executable File Formats File System Automount Point.
[ 8.267184] systemd[1]: Reached target Local Encrypted Volumes.
[ 8.287158] systemd[1]: Reached target Paths.
[ 8.307134] systemd[1]: Reached target Remote File Systems.
[ 8.327123] systemd[1]: Reached target Slices.
[ 8.347133] systemd[1]: Reached target Swap.
[ 8.367388] systemd[1]: Listening on fsck to fsckd communication Socket.
[ 8.391249] systemd[1]: Listening on initctl Compatibility Named Pipe.
[ 8.415416] systemd[1]: Listening on Journal Audit Socket.
[ 8.435332] systemd[1]: Listening on Journal Socket (/dev/log).
[ 8.455354] systemd[1]: Listening on Journal Socket.
[ 8.475402] systemd[1]: Listening on udev Control Socket.
[ 8.495294] systemd[1]: Listening on udev Kernel Socket.
[ 8.539254] systemd[1]: Mounting Huge Pages File System...
[ 8.560846] systemd[1]: Mounting POSIX Message Queue File System...
[ 8.584943] systemd[1]: Mounting Kernel Debug File System...
[ 8.604935] systemd[1]: Mounting Kernel Trace File System...
[ 8.624981] systemd[1]: Starting Create list of static device nodes for the current kernel...
[ 8.671373] systemd[1]: Starting Load Kernel Module configfs...
[ 8.692990] systemd[1]: Starting Load Kernel Module fuse...
[ 8.713070] systemd[1]: Condition check resulted in Set Up Additional Binary Formats being skipped.
[ 8.722280] systemd[1]: Condition check resulted in File System Check on Root Device being skipped.
[ 8.722803] fuse: init (API version 7.38)
[ 8.751387] systemd[1]: Starting Journal Service...
[ 8.780205] systemd[1]: Starting Load Kernel Modules...
[ 8.800921] systemd[1]: Starting Remount Root and Kernel File Systems...
[ 8.823127] EXT4-fs (mmcblk1p1): re-mounted f08ddc42-9533-4f8d-b672-9a6816266c22. Quota mode: none.
[ 8.825002] systemd[1]: Starting Coldplug All udev Devices...
[ 8.854471] systemd[1]: Started Journal Service.
[ 9.118358] systemd-journald[259]: Received client request to flush runtime journal.
[ 10.104778] sbsa-gwdt f0610000.watchdog: Initialized with 10s timeout @ 25000000 Hz, action=0.
[ 10.268194] omap_rng f2760000.trng: Random Number Generator ver. 203b34c
[ 10.279239] at24 0-0053: supply vcc not found, using dummy regulator
[ 10.290680] at24 0-0053: 256 byte 24c02 EEPROM, writable, 16 bytes/write
[ 10.756143] mvpp2 f2000000.ethernet eth1: configuring for fixed/sgmii link mode
[ 10.763619] mvpp2 f2000000.ethernet eth1: Link is Up - 1Gbps/Full - flow control off
[ 10.848535] mvpp2 f2000000.ethernet eth0: configuring for inband/10gbase-r link mode
[ 10.858608] mv88e6085 f212a200.mdio-mii:04 lan1: configuring for phy/gmii link mode
[ 10.940083] mvpp2 f2000000.ethernet eth2: PHY [f212a200.mdio-mii:00] driver [Marvell 88E1510] (irq=POLL)
[ 10.949693] mvpp2 f2000000.ethernet eth2: configuring for phy/rgmii-id link mode
[ 11.097755] mv88e6085 f212a200.mdio-mii:04: Link is Down
[ 11.107378] mv88e6085 f212a200.mdio-mii:04: Link is Up - 1Gbps/Full - flow control off
[ 15.039662] mvpp2 f2000000.ethernet eth2: Link is Up - 1Gbps/Full - flow control rx/tx
[ 15.047648] IPv6: ADDRCONF(NETDEV_CHANGE): eth2: link becomes ready
[ 146.168361] mvpp2 f2000000.ethernet eth0: Link is Up - 10Gbps/Full - flow control rx
[ 146.176174] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 146.274737] mvpp2 f2000000.ethernet eth0: Link is Down
[ 146.279938] mvpp2 f2000000.ethernet eth0: Link is Up - 10Gbps/Full - flow control rx
[ 146.392931] mvpp2 f2000000.ethernet eth0: Link is Down
[ 146.398126] mvpp2 f2000000.ethernet eth0: Link is Up - 10Gbps/Full - flow control rx
...
But like @wteiken, I also doubt this has anything to do with the modules. These kind of issues and the CRC errors seem to only happen when the link partner is a Marvel chip. In this case it was a CN9130 base connected to a CN9130 Pro and the issues went away the moment one side was replaced with a different switch or a DAC. The same issues also happen with the ClearFog-GT8K.
We also checked the rx_los value and it stayed at 0.
root@sr-cn913x:/home/debian# cat /sys/kernel/debug/sfp-eth\@0/state
Module state: present
Module probe attempts: 0 0
Device state: up
Main state: link_up
Fault recovery remaining retries: 5
PHY probe remaining retries: 12
moddef0: 1
rx_los: 0
tx_fault: 0
tx_disable: 0
Here are some details of a Mikrotik netPower 16P connected to a ClearFog-GT8K. According to the block diagram, the switch is using a Marvel 98DX3236:
/interface/ethernet/print stats where name=sfp-sfpplus2
name: sfp-sfpplus2
driver-rx-byte: 264 694 619
driver-rx-packet: 1 190 510
driver-tx-byte: 308 692 535
driver-tx-packet: 3 400 053
rx-bytes: 687 370 606 208
rx-too-short: 0
rx-too-long: 0
rx-unicast: 653 254 823
rx-broadcast: 24 218
rx-pause: 0
rx-multicast: 150 231 239
rx-error-events: 453
rx-fcs-error: 175
rx-fragment: 0
rx-overflow: 0
rx-jabber: 0
tx-bytes: 637 895 630 895
tx-unicast: 657 638 718
tx-broadcast: 331 178
tx-pause: 0
tx-multicast: 110 944 251
tx-underrun: 0
tx-collision: 0
tx-late-collision: 0
tx-drop: 1 105
tx-rx-64: 45 648 222
tx-rx-65-127: 517 469 004
tx-rx-128-255: 51 087 927
tx-rx-256-511: 27 399 154
tx-rx-512-1023: 30 351 371
tx-rx-1024-max: 900 469 349
tx-queue0-packet: 768 914 044
tx-queue1-packet: 0
tx-queue2-packet: 0
tx-queue3-packet: 0
tx-queue4-packet: 0
tx-queue5-packet: 0
tx-queue6-packet: 0
tx-queue7-packet: 0
The GT8K reports link losses about 1-2 every 10s and occationally crc errors proportionally to the throughput.
[499500.012673] mvpp2 f2000000.ethernet eth0: bad rx status 12048510 (crc error), size=678
The number of errors reported by the switch are orders of magnitude lower than those reported by the GT8K.
Currently I have one CN9130 Base, one CN9130 Pro and a GT8K on my table where I can easily run different firmwares if needed. I have generic 10G single-mode fiber modules from FS that work as long as the link partner is not a Marvel chip and in addition to that a generic DAC, also from FS, with which I have yet to see any errors.
Is this happening if they are directy connected, or only when running through a switch? I am trying to replicate the behavior locally.
This is also happening when they are directly connected. So a minimal setup is to directly connect two ClearFog CN9130 devices with a 10G fiber optic cable and modules. 1G fiber modules work, 1G RJ45 modules work and 10G DACs work without problems.
Forgot: The CRC error is something I also see on another CN913X based system (QNAP nas), same effect that it stopped with the changed switch. They usually only happen under heavy traffic towards the CN913X based systems (i.e., not really for the ACKs during traffic then copying data from the CN913X based system).
The link flaps are not happening on the other systems (i.e., only on my CN9130 base), despite running the same modules and changing them to make sure it’s not an issue with the specific module.
So only some of the issues seem to be specific to the SolidRun systems. Have not looked in detail which other chips are used on the QNAP for networking…
I am having exactly the same issue with two directly connected ClearFog CN9130 (one Base, one Pro) systems. I am using the following transceivers (which do work flawlessly in other systems):
- Fiberstore SFP-10GSR-85
- Digitus DN-81200
A 10G DAC connection does work flawlessly.
I have tested various OS and Kernel combinations.
I have got the least worse results with an Ubuntu 20.04.5 LTS build based on linux: upgrade to v6.1.29 current LTS · SolidRun/cn913x_build@8a00da5 · GitHub
With the mentioned Ubuntu build the Link drops about every 1-60 seconds.
This is a problem with the comphy tuning of the SFP port in Arm Trusted Firmware.
The serdes line needs to be tuned to the board and a phy-porting-layer.h needs to be added to the ATF code. The final bootloader needs to be updated on the device.
It is not that easy to find the correct settings, basically you need to do some try and error with the registers while running some scenario that fails.
That sounds very involved. Thank you for updating us with the steps needed to diagnose and fix the problems. It sounds like this is something that an enthusiastic end user like myself wouldn’t be able to do in a home environment. That said I would like to assist as I am able. I can test images as they are generated, if that helps. Otherwise, thank you for the update, and I hope a correct tuning is found.
@lblyth sorry I am not working on this. I do not have a board to do any testing. It was a general comment from our custom project where we had to tune the serdes lines for the CN913X.
Is there any movement on this topic from SolidRun? I am asking cause I purchased this board planning to use the SFP port with a fiber optic cable. I gather but have not tested that DACs will work, but I have already made the investment in SFPs and fiber. SolidRun said they tested this board with SFPs. There seems to be a good number of users experiencing this problem so a resolution will be appreciated by many customers. I am hoping SolidRun can resolve this via software update. If that is not possible they could issue some amount of refunds so affected users can either purchase DACs or different gear.
We are trying to determine if the method we used for calibrating the serdes on the Armada 8040 systems is still available for the CN9130. The issue seems to only exist when connecting between Marvell chipsets, and doesn’t seem to be an issue when connecting from one of our boards to say a broadcom, or intel based chipset.
Thank you for the reply. Are you able to share a timeline or a priority level for this? If the method that was used is not available is there a plan b?
I am hoping to make plans around the timeline of this bug being fixed.
So I stumbled upon this patch net: dsa: mv88e6xxx: Add erratum 3.14 for 88E6390X and 88E6190X. I know it’s a completely different chip, but this still looks like a similar problem.
In any case: Take this as a friendly reminder that this is still an issue and I very much would like to see a solution or at the very least some concrete hints on how to solve this.
I guess @Sven was referring to phy-port-layer.h here. But without access to proper documentation, I wouldn’t even know where to begin.
I was looking to go optical, so I picked up a few cables/modules, and I’m having similar results to others here. I figured I’d list the hardware I’ve tried.
Peer 1: CN9130 Pro
Peer 2: Trendnet TEG-7124ws
uname -a
Linux router 6.8.1-vanilla #5 SMP PREEMPT Sat Mar 30 08:33:53 EDT 2024 aarch64 GNU/Linux
| model | type | link? | flaps? |
|---|---|---|---|
| Trendnet TEG-10GBSR | 10-gbase sr | yes | yes |
when flapping:
[538955.938213] mvpp2 f2000000.ethernet sfp0: Link is Up - 10Gbps/Full - flow control rx
[538956.027671] mvpp2 f2000000.ethernet sfp0: bad rx status 12048510 (crc error), size=742
[538956.838784] mvpp2 f2000000.ethernet sfp0: Link is Down
[538956.844078] mvpp2 f2000000.ethernet sfp0: Link is Up - 10Gbps/Full - flow control rx
[538957.145150] mvpp2 f2000000.ethernet sfp0: bad rx status 12048510 (crc error), size=592
[538957.676684] mvpp2 f2000000.ethernet sfp0: Link is Down
[538957.682035] mvpp2 f2000000.ethernet sfp0: Link is Up - 10Gbps/Full - flow control rx
[538958.379405] mvpp2 f2000000.ethernet sfp0: bad rx status 12008510 (crc error), size=68
[538958.502233] mvpp2 f2000000.ethernet sfp0: bad rx status 12048510 (crc error), size=592
[538958.768211] mvpp2 f2000000.ethernet sfp0: bad rx status 12408510 (crc error), size=68
[538959.162732] mvpp2 f2000000.ethernet sfp0: Link is Down
[538959.168103] mvpp2 f2000000.ethernet sfp0: Link is Up - 10Gbps/Full - flow control rx
[538959.976552] mvpp2 f2000000.ethernet sfp0: Link is Down
[538959.981850] mvpp2 f2000000.ethernet sfp0: Link is Up - 10Gbps/Full - flow control rx
[538961.283664] mvpp2 f2000000.ethernet sfp0: bad rx status 12048510 (crc error), size=786
ethtool && ethtool -m
Settings for sfp0:
Supported ports: [ FIBRE ]
Supported link modes: 10000baseSR/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 10000baseSR/Full
Advertised pause frame use: Symmetric Receive-only
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Speed: 10000Mb/s
Duplex: Full
Auto-negotiation: on
Port: FIBRE
PHYAD: 0
Transceiver: internal
Link detected: yes
Identifier : 0x03 (SFP)
Extended identifier : 0x04 (GBIC/SFP defined by 2-wire interface ID)
Connector : 0x07 (LC)
Transceiver codes : 0x10 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
Transceiver type : 10G Ethernet: 10G Base-SR
Encoding : 0x00 (unspecified)
BR, Nominal : 10300MBd
Rate identifier : 0x00 (unspecified)
Length (SMF,km) : 0km
Length (SMF) : 0m
Length (50um) : 80m
Length (62.5um) : 30m
Length (Copper) : 0m
Length (OM3) : 300m
Laser wavelength : 850nm
Vendor name : TRENDnet
Vendor OUI : 00:00:00
Vendor PN : TEG-10GBSR
Vendor rev : V3.0
Option values : 0x00 0x3a
Option : RX_LOS implemented
Option : TX_FAULT implemented
Option : TX_DISABLE implemented
Option : RATE_SELECT implemented
BR margin, max : 0%
BR margin, min : 0%
Vendor SN : RA3ISR3000023
Date code : 230918
And, I’ve tried the following DAC cable:
| model | type | link? | flaps? |
|---|---|---|---|
| HiFiber CAB-10SFP | 10-gbase cu | no | n/a |
Also, the port works fine with 1g rj45 sfp modules (tp link and a no-name brand).