Ubuntu 22.04 cannot recognize NVMe M.2 on boot (LX2160a)

The Ubuntu 22.04 installer recognizes the NMVe M.2 on installation. We use ‘user-data’ and create ESP partitions and other partitions without issue. I can mount that drive from another Ubuntu 20.04 instance that is an eMMC boot and navigate around the file systems on that drive and everything looks normal.

When we then try to boot off of the NVMe0 drive, we get these errors and drop into ‘initramfs’:

4.521321] nvme 0000:01:00.0: Adding to iommu group 2

[ 4.537405] nvme nvme0: pci function 0000:01:00.0
[ 65.061228] nvme nvme0: I/O 28 QID 0 timeout, disable controller
[ 65.173219] nvme nvme0: Device shutdown incomplete; abort shutdown
[ 65.193361] nvme nvme0: Identify Controller failed (-4)
[ 65.198585] nvme nvme0: Removing after probe failure status: -5

We have tried a number of power management commands, iommu.passthrough, etc. without any change.

The real question is why the booting kernel, 22.04, cannot seem to detect/discover the NVMe drive.

Thanks,

Dave

This is most likely due to the kernel version in the newer image. Relatively recently Linux has enabled more PCIe power management features by default, where they used to be disabled. This has caused some NVMe devices to start having issues, not just on Aarch64 but also x86_64 platforms. Please try booting with nvme_core.default_ps_max_latency_us=0 set on the kernel commandline.

Thank you for the suggestion. It did not make any difference. The NVMe drive is clearly recognized initially, but any subsequent commands to it by the booting Ubuntu 22.04 kernel are ignored.

Any other suggestions are welcome. We’ve been languishing on this issue far longer than I care to admit.

Thanks,

Dave