HoneyComb Distro-Installation Part 1 and 2

HoneyComb and ClearFog systems booting with UEFI and ACPI are almost fully supported on any distribution that is using the 5.14 kernel and newer. There are a few quirks that are still needed while we work to mainline the patches needed for full support of the board. Current distros that work with a minimal set of quirks are, Fedora 35 and newer, Debian Bookworm, Ubuntu 22.04 Jammy JellyFish, Centos Stream 9, OpenSuse Leap 15.3.

Until the latest IORT RMR patches are accepted into the mainline kernel (this has been an ongoing process for over 14 months as of this writing) You will need to add the following bootargs to your linux kernel commandline, arm-smmu.disable_bypass=0 iommu.passthrough=1 This is especially important for installation as without it the LX2160a networking driver will produce excessive SMMU errors in the logs and slow the entire distribution down. With this option set networking will be fully functional.

If you are planning to use a GPU with your board we recommend an AMD Radeon GPU that is Polaris generation or older. For Nvidia GPUs only the GT 7xx series is supported well enough with OSS drivers to be useable. Following are useful GPU quirks.

AMD:

NVIDIA

  • Currently the Nvidia binary Aarch64 driver does not function with HoneyComb. The kernel module will build and load, however when you try to initialize the driver the binary code errors out.
  • The in kernel nouveau driver does function properly if the edk2 firmware source is built with x86emu option. This is not built by default as the SystemReady spec does not allow driver emulation in the firmware.
  • The downside of the nouveau driver is that it only supports clock control for the GT 7xx series graphics cards. Anything newer than that you will only get the base clock speed which tend to be quite slow.
  • With the Nvidia GPUs we do also recommend to use the above cortex_a72_memcpy library.

Networking support

With kernel versions 5.14 and newer the 1Gbps RJ45 interface will work out of the box. If you would like to use the 10Gbps networking interfaces then you will need to install NXP’s restool utility to intialize the SFP+ interfaces. Fedora has this utility packaged and can be dnf installed, otherwise the source can be downloaded and compiled here, qoriq-components/restool - DPAA2 Resource Management Tool You can follow NXP’s documentation regarding configuration.

What doesn’t have mainline ACPI support yet

  • SMMU support. The fsl-mc packet processor requires host DRAM to function so needs the SMMU configurated to allow it to access this address space. The spec for this is done in ACPI using the IORT table and specifically RMR’s (Reserved Memory Regions). Linux maintainer’s rejected Arm’s spec for this and over the past year the spec has been reworked so that hopefully this ACPI feature will be accepted into mainline and the above workarounds will no longer be needed.
  • I2C clocking. While I2C is functional it is running at a very slow speed due to hard-coded values in the Linux kernel. Rather than an immediate fix Arm has again worked on another spec change for specifying simple device clocking via methods in AML. This spec has been approved however there is no code merged to take advantage of it yet.
  • SDHC/MMC. This has the same issue as I2C and should be resolved when the same infrastructure code is merged.

Please feel free to post any distro specific questions, or hints below. We will be updating this thread as bugs are resolved, or new features are added.

4 Likes

Thanks, this is helpful. I would encourage a bit more information to be added in the official getting started guide as I’ve been banging my head trying to get the SFP+ modules to work. I am not using one of the pre-built images as we want control over our software baseline. I have successfully flashed the xspi with the uefi image and using the ACPI table, not the device tree blob. I installed a nvme drive in the m.2 slot and it seems to work without any issue.

  1. I found it rather annoying that there are no link lights on the SFP+ cage
  2. If you switch between ACPI and DTB in the UEFI after the OS is installed the system will probably kernel panic.
  3. Using the restool command compiled from github source was problematic. I had to extract restool from one of the downloadable images in order to create the network interfaces. As far as i can tell this had something to do with a mismatch between the MC firmware version and restool. It would be good to describe how I could upgrade my MC firmware.
  4. Network interfaces do not persist across a reboot.
  5. There was no mapping of which interface (numbered on the SFP cage) correspond to which SERDES or with eth device for that matter. It seems to correlate to the order the dpmacs are registered with ls-addni
  6. Fiber Modules don’t seem to work well. I had success with some 10G Active Optical Cables.
  7. Do the SFP+ ports negotiate down with slower modules installed? I couldn’t find this answer in any documentation.
  8. How can i troubleshoot connections without link lights? ethtool --show-module doesn’t report anything about the module plugged into the slot.
  9. The I had a hard time trying to get the copper ethernet port to show up. I think the thing that worked was modprobe at803x. For some reason once I did that eth0 persisted across a reboot and I didn’t make any changes to modprobe.d.

Mabye this is all obvious to someone at SolidRun but as someone who deals with x86_64 based server class machines, this is a lot of trouble to go through just to get an ARM hypervisor cluster. Hopefully this will help someone else.

device-tree in the EFI firmware is purely for debugging during development work. If you prefer to use device-tree we recommend using u-boot as the bootloader.

In general EFI is a proof of concept and because a dead end production bootloader since the mainline Linux kernel wouldn’t accept some patches needed for full hardware functionality when booting with ACPI.

In general most of the issues you are having are fixed under u-boot and I recommend you test using the latest development work done here, GitHub - SolidRun/lx2160a_build at develop-ls-5.15.71-2.2.0