HoneyComb Distro-Installation Part 1 and 2

SolidRun DevCuts
, , ,
1

HoneyComb and ClearFog systems booting with UEFI and ACPI are almost fully supported on any distribution that is using the 5.14 kernel and newer. There are a few quirks that are still needed while we work to mainline the patches needed for full support of the board. Current distros that work with a minimal set of quirks are, Fedora 35 and newer, Debian Bookworm, Ubuntu 22.04 Jammy JellyFish, Centos Stream 9, OpenSuse Leap 15.3.

Until the latest IORT RMR patches are accepted into the mainline kernel (this has been an ongoing process for over 14 months as of this writing) You will need to add the following bootargs to your linux kernel commandline, arm-smmu.disable_bypass=0 iommu.passthrough=1 This is especially important for installation as without it the LX2160a networking driver will produce excessive SMMU errors in the logs and slow the entire distribution down. With this option set networking will be fully functional.

If you are planning to use a GPU with your board we recommend an AMD Radeon GPU that is Polaris generation or older. For Nvidia GPUs only the GT 7xx series is supported well enough with OSS drivers to be useable. Following are useful GPU quirks.

AMD:

NVIDIA

  • Currently the Nvidia binary Aarch64 driver does not function with HoneyComb. The kernel module will build and load, however when you try to initialize the driver the binary code errors out.
  • The in kernel nouveau driver does function properly if the edk2 firmware source is built with x86emu option. This is not built by default as the SystemReady spec does not allow driver emulation in the firmware.
  • The downside of the nouveau driver is that it only supports clock control for the GT 7xx series graphics cards. Anything newer than that you will only get the base clock speed which tend to be quite slow.
  • With the Nvidia GPUs we do also recommend to use the above cortex_a72_memcpy library.

Networking support

With kernel versions 5.14 and newer the 1Gbps RJ45 interface will work out of the box. If you would like to use the 10Gbps networking interfaces then you will need to install NXP’s restool utility to intialize the SFP+ interfaces. Fedora has this utility packaged and can be dnf installed, otherwise the source can be downloaded and compiled here, qoriq-components/restool - DPAA2 Resource Management Tool You can follow NXP’s documentation regarding configuration.

What doesn’t have mainline ACPI support yet

  • SMMU support. The fsl-mc packet processor requires host DRAM to function so needs the SMMU configurated to allow it to access this address space. The spec for this is done in ACPI using the IORT table and specifically RMR’s (Reserved Memory Regions). Linux maintainer’s rejected Arm’s spec for this and over the past year the spec has been reworked so that hopefully this ACPI feature will be accepted into mainline and the above workarounds will no longer be needed.
  • I2C clocking. While I2C is functional it is running at a very slow speed due to hard-coded values in the Linux kernel. Rather than an immediate fix Arm has again worked on another spec change for specifying simple device clocking via methods in AML. This spec has been approved however there is no code merged to take advantage of it yet.
  • SDHC/MMC. This has the same issue as I2C and should be resolved when the same infrastructure code is merged.

Please feel free to post any distro specific questions, or hints below. We will be updating this thread as bugs are resolved, or new features are added.

4 Likes
2

Thanks, this is helpful. I would encourage a bit more information to be added in the official getting started guide as I’ve been banging my head trying to get the SFP+ modules to work. I am not using one of the pre-built images as we want control over our software baseline. I have successfully flashed the xspi with the uefi image and using the ACPI table, not the device tree blob. I installed a nvme drive in the m.2 slot and it seems to work without any issue.

  1. I found it rather annoying that there are no link lights on the SFP+ cage
  2. If you switch between ACPI and DTB in the UEFI after the OS is installed the system will probably kernel panic.
  3. Using the restool command compiled from github source was problematic. I had to extract restool from one of the downloadable images in order to create the network interfaces. As far as i can tell this had something to do with a mismatch between the MC firmware version and restool. It would be good to describe how I could upgrade my MC firmware.
  4. Network interfaces do not persist across a reboot.
  5. There was no mapping of which interface (numbered on the SFP cage) correspond to which SERDES or with eth device for that matter. It seems to correlate to the order the dpmacs are registered with ls-addni
  6. Fiber Modules don’t seem to work well. I had success with some 10G Active Optical Cables.
  7. Do the SFP+ ports negotiate down with slower modules installed? I couldn’t find this answer in any documentation.
  8. How can i troubleshoot connections without link lights? ethtool --show-module doesn’t report anything about the module plugged into the slot.
  9. The I had a hard time trying to get the copper ethernet port to show up. I think the thing that worked was modprobe at803x. For some reason once I did that eth0 persisted across a reboot and I didn’t make any changes to modprobe.d.

Mabye this is all obvious to someone at SolidRun but as someone who deals with x86_64 based server class machines, this is a lot of trouble to go through just to get an ARM hypervisor cluster. Hopefully this will help someone else.

1 Like
3

device-tree in the EFI firmware is purely for debugging during development work. If you prefer to use device-tree we recommend using u-boot as the bootloader.

In general EFI is a proof of concept and because a dead end production bootloader since the mainline Linux kernel wouldn’t accept some patches needed for full hardware functionality when booting with ACPI.

In general most of the issues you are having are fixed under u-boot and I recommend you test using the latest development work done here, GitHub - SolidRun/lx2160a_build at develop-ls-5.15.71-2.2.0

4

Seconding @boyceke1 's comment here about possibly updating the developer documentation! I had been following the ‘DevCuts’ video on installing Fedora, but that is going to NVMe which I am already using for something else. The installer doesn’t see SDHC/MMC at all (as per my other question).

So I really would like to get Ubuntu 22 installed on eMMC. If the way to do this is through u-boot, then my problem is that I can’t figure out how to boot into the Ubuntu installer (which is EFI based). I’ve tried ‘bootefi’ one of the files from the installer image that I loaded into memory; but the installer doesn’t find the installer media after that.

There’s nothing else on the Ubuntu installer image; no Device Tree file, just three ‘.EFI’ applications.

5

The issue with no eMMC / uSDHC is due to mainline not accepting a couple of small patches needed to properly support the DMA coherency of the devices with ACPI. I will look into updating the device-tree support in EFI so the hardware can be better supported by mainline distros out of the box.

6

Thanks for the quick reply and confirmation. For the time being I’ve just installed onto SATA; whenever eMMC is supported I’ll move it there and get my drive back. Nice to have the mainline kernel finally.

In case it helps anyone, I’ve documented my installation steps here.

7

I am on Debian Bookworm stock kernel 6.1.0-35-arm64 with iommu workaround to boot parameters

# lsmod | grep qoriq
qoriq_thermal          16384  0
qoriq_cpufreq          16384  0
ahci_qoriq             16384  0
libahci_platform       20480  2 ahci_platform,ahci_qoriq
libahci                40960  3 libahci_platform,ahci_platform,ahci_qoriq
libata                299008  4 libahci,libahci_platform,ahci_platform,ahci_qoriq

Should cpufreq scaling/governor be working on that kernel?

8

With UEFI firmwares we have cpufrequency scaling disabled. This was tested via standard ACPI methods however based on the design of the chip we found that CPU scaling had minimal benefits to the platform. It was designed to mainly be an always on full powered networking device.

9

Ok thank you for fast response. If I build EDK2 firmware myself anyway is there a toggle in code to re-enable it? Minimal difference may be a huge difference for my setup.

Later on I will grep the code maybe I will find it myself.

10

There is not because the original standard it was designed for has been deprecated by the ACPI standards and most distributions do not enable the kernel module because they only want to support CPPC.
Generally if you need tuning at this level it is much better to use u-boot and device-tree since there are specific platform modules that can be used.

11

Hmmm. Thank you. For now its not “production” yet so I will consider that. Would need to reinstall GRUB2 unless U-Boot supports EFI boot but I think upstream it does.