instructions on how to configure a CM4 to boot the root filesystem over pcie/sata. This is geared to work with a turingpi2, but would probably work with other CM4 carrier boards

instructions.md

Boot from SD / EMMC with root filesystem on SATA SSD

CM4s will not boot directly from an SSD over SATA, but you can boot from SD / EMMC and mount the root filesystem from an external disk.

• This does not apply to you if you are using NVME
• This does not apply to you if you are using USB

Requirements

• A Turingpi2
• A CM4 module (lite or with EMMC)
• A SATA Drive with cables etc
• A mini pcie to sata connection. I bought two of these
• Another computer to image the OS with using rpiboot

On the last point:

• I'm using a Pi 400 running the 64 bit desktop version of Raspberry Pi OS
• A cheap 50 cm usb-a to usb-a cable for connecting it to the turingpi2.

Prior to that I was trying to use an aging Microsoft Surface Pro Laptop to peform the imaging. It was a fustrating nightmare.

• Wouldn't work with the node-1 slot
• Would perform 99% of the OS then fail
• If there is already an OS on the CM4 rpiboot would become temperamental
• Windows would complain about unrecognized devices and corrupt media

Literally hours of rebooting and plugging things in and out until it would randomly worked. All these problems went away when I switched to RPIOS / Pi400.

The only issue I encounter is that after I perform a usb device mount, I need to do a full power cycle on the turingpi2 or the CM4 module will not boot.

For this procedure I will reference the pi400 as the PC used to run rpiboot and the raspberry pi imager.

Flash the OS onto the CM4 from the pi400

• Not going to explain this process in detail.
• Follow the documentaion for rpiboot.
• Install the OS, reboot it. Let it do its thing.
• Once it's done remount the CM4 filkesystem again with ./rpiboot

Tar the CM4 rootfs from the pi400

There are other ways of doing this, depending on your individual scenario / hardware etc.

• dd
• rsync
• gparted

The long and the short of it is you need to get the root filesystem onto the external drive.

cd /media/admin
tar -cvf ~/rootfs.tar rootfs/

• This will take a few minutes. Once it's done you'll have an archive around the 1.1 GB mark.
• Disable the device usb mounting and reboot the CM4 node.
• you may need to power cycle the turingpi

Prepare the CM4 node

• SSH into the CM4 node
• SCP the rootfs.tar from the other machine

scp root@pi400:~/rootfs.tar .

Format the SSD Drive on the CM4

• Connect the stata drive to the CM4 using a mini pcie adapter
• Create a partition with the following commands
• Change the size of the partition to suit your needs

parted /dev/sda mklabel gpt
parted /dev/sda mkpart primary ext4 0G 256G
mkfs.ext4 /dev/sda1

• Once thats done mount it

mkdir /mnt/external
mount /dev/sda1 /mnt/external

• Now unarchive the rootfs.tar

cd /mnt/external
tar -xf ~/rootfs.tar --strip-components=1

Update cmdline.txt and fstab on the CM4

• Run blkid /dev/sda1 to get the PARTUUID

/dev/sda1: UUID="3df573bc-5f2c-48f0-b782-4360d40142c2" BLOCK_SIZE="4096" TYPE="ext4" PARTLABEL="primary" PARTUUID="95707ce3-e866-430c-8133-3c014be6207b"

• Update /boot/cmdline.txt with the new PARTUUID so that it look like this

console=serial0,115200 console=tty1 root=PARTUUID=95707ce3-e866-430c-8133-3c014be6207b rootfstype=ext4 fsck.repair=yes rootwait

• Update fstab on the external drive the one at /mnt/external/etc/fstab NOT /etc/fstab

proc            /proc           proc    defaults          0       0
PARTUUID=58a29364-01  /boot           vfat    defaults          0       2
# PARTUUID=58a29364-02  /               ext4    defaults,noatime  0       1
PARTUUID=95707ce3-e866-430c-8133-3c014be6207b  /               ext4    defaults,noatime  0       1
# a swapfile is not a swap partition, no line here
#   use  dphys-swapfile swap[on|off]  for that

Create an initrd

• This is the most important part of the whole operation. Read this thread. I'll go through the steps anyway as there is a few typos in the commands
• Update /etc/initramfs-tools/modules by adding ahci to the end

# List of modules that you want to include in your initramfs.
# They will be loaded at boot time in the order below.
#
# Syntax:  module_name [args ...]
#
# You must run update-initramfs(8) to effect this change.
#
# Examples:
#
# raid1
# sd_mod
ahci

• Create /etc/initramfs-tools/conf.d/resume with the contents

RESUME=none

• Run the following command to generate and initrd.img

update-initramfs -c -k `uname -r`

• It will spit out a result which contains something like /boot/initrd.img-5.15.84-v8+. Make a note of this.

Update config.txt

• Open /boot/config.txt and insert the following initramfs initrd.img-5.15.84-v8+ followkernel
• Change initrd.img-5.15.84-v8+ to the output of the previous step
• I put it at the top of the file just because

# For more options and information see
# http://rpf.io/configtxt
# Some settings may impact device functionality. See link above for details

initramfs initrd.img-5.15.84-v8+ followkernel
# uncomment if you get no picture on HDMI for a default "safe" mode
#hdmi_safe=1
....

Profit

• Unmount the external drive umount /mnt/external
• Reboot the CM4
• Run checks

df -h

Filesystem      Size  Used Avail Use% Mounted on
udev            3.7G     0  3.7G   0% /dev
tmpfs           782M  1.1M  781M   1% /run
/dev/sda1       234G  1.5G  221G   1% /
tmpfs           3.9G     0  3.9G   0% /dev/shm
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
/dev/mmcblk0p1  255M   40M  216M  16% /boot
tmpfs           782M     0  782M   0% /run/user/1000

I'm probably going to write some Ansible to automate this, as it's pretty laborious for three nodes.