op5_tftp_nfs.md

TFTP + NFS LKDev Environment Guideline (NixOS Host)

This setup lets you compile the linux kernel on your development workstation and have the OrangePi 5 automatically boot it over the network using TFTP (to load the kernel from u-boot) and NFS (to load the rootfs). The goal is simply to speed up debugging/testing iterations while developing/fixing the linux kernel.

Prerequisites

• Orange Pi 5: With a working distro (likely from here) installed on SD card.

• Network (local) Connection: Ethernet cable connecting the board to your local network.

• Custom U-Boot Files: The stock U-Boot (from the official linux images) lacks Ethernet support. Download these modified versions (or build yourself one):

  • http://onurozkan.dev/lkdev-artifacts/rk3588s/idbloader.img
  • http://onurozkan.dev/lkdev-artifacts/rk3588s/u-boot.itb

• Serial Port Access: UART-to-USB adapter/converter (be aware, only TX, RX and GND should be connected. Do not connect the VCC/power pin). This is only needed if bootloader is broken and we have to boot manually from u-boot.

• Host Packages: Host system should have the related compilers and other packages installed on the system. You can use this shell if you don't want to waste time on this.

0- Flash the bootloader components

Flash the downloaded U-Boot files to your SD card. Replace /dev/sdX with your actual SD card device (e.g., /dev/sda).

sudo dd if=idbloader.img of=/dev/sdX bs=512 seek=64 conv=fsync
sudo dd if=u-boot.itb of=/dev/sdX bs=512 seek=16384 conv=fsync

1- Prepare workstation directories

On your NixOS workstation, create the directories that will host the TFTP (for boot files) and NFS (for rootfs).

sudo mkdir -p /srv/tftp/rk3588s
sudo mkdir -p /srv/nfs/opi5

2) Extract rootfs from the board

Boot your board normally from the SD card. Once it's online and reachable via SSH, run the following command from your workstation to clone the board's rootfs (don't forget to replace 192.168.1.169 with your board's IP):

sudo rsync -aHAX --numeric-ids \
  --rsync-path="sudo rsync" \
  root@192.168.1.169:/ /srv/nfs/opi5/ \
  --exclude={"/dev/*","/proc/*","/sys/*","/run/*","/tmp/*","/mnt/*","/lost+found"}

3) Configure TFTP and NFS

Add the following to your workstation's configuration.nix to enable TFTP and NFS.

services.tftpd = {
  enable = true;
  path = "/srv/tftp";
};

services.nfs.server = {
  enable = true;
};

services.nfs.server.exports = ''
  /srv/nfs *(rw,sync,no_subtree_check,fsid=0,no_root_squash)
  /srv/nfs/opi5 *(rw,sync,no_subtree_check,no_root_squash)
'';

networking.firewall = {
  enable = true;
  allowedUDPPorts = [69];
  allowedTCPPorts = [2049];
};

After updating your config, apply the changes:

sudo nixos-rebuild switch
sudo exportfs -rav

4) Build the kernel

From your kernel source tree on your workstation, configure and build the kernel.

export ARCH="arm64"
export CROSS_COMPILE="aarch64-unknown-linux-gnu-"
    
make defconfig
./scripts/config \
  --set-str LOCALVERSION "-lkdev" \
  --enable NFS_FS \
  --enable NFS_V4 \
  --enable ROOT_NFS \
  --enable IP_PNP \
  --enable IP_PNP_DHCP \
  --enable STMMAC_ETH \
  --enable STMMAC_PLATFORM \
  --enable DWMAC_ROCKCHIP \
  --enable PHYLIB \
  --enable MDIO_BUS \
  --enable REALTEK_PHY

make olddefconfig

make -j$(nproc) Image dtbs modules
sudo make modules_install INSTALL_MOD_PATH=/srv/nfs/opi5
sudo cp arch/arm64/boot/dts/rockchip/rk3588s-orangepi-5.dtb /srv/tftp/rk3588s
sudo cp arch/arm64/boot/Image /srv/tftp/rk3588s

5) Configure netboot on the board

Log in to the board's shell and back up the existing boot scripts. We will create a new script to tell U-Boot to look at your workstation (the "server") for the kernel.

sudo cp -a /boot/boot.scr /boot/boot.scr.bak
sudo cp -a /boot/boot.cmd /boot/boot.cmd.bak

Create a new /boot/boot.cmd with the following content (don't forget to replace 192.168.1.116 with your workstation's IP):

setenv autoload no
setenv serverip 192.168.1.116
dhcp

tftpboot ${loadaddr} rk3588s/Image
tftpboot ${fdt_addr_r} rk3588s/rk3588s-orangepi-5.dtb

setenv bootargs "console=ttyS2,1500000 earlycon ip=dhcp root=/dev/nfs rw nfsroot=192.168.1.116:/opi5,vers=4,proto=tcp rootwait"
booti ${loadaddr} - ${fdt_addr_r}

Now compile the script into a format U-Boot understands:

mkimage -A arm -T script -C none -n "op5 netboot" -d /boot/boot.cmd /boot/boot.scr

6- Boot and verify

Open a serial terminal on your workstation to watch the boot process. Replace ttyUSB0 with your actual UART device.

picocom -b 1500000 /dev/ttyUSB0

Now power the board. It should now fetch the Image via TFTP and mount rootfs via NFS.

Once logged in, verify the kernel and rootfs:

• Kernel Version: Run uname -r.
• NFS: Run findmnt /, the FS should point to your workstation's IP (e.g., 192.168.1.116:/opi5).