Increase the BAR memory address space for PCIe devices on the Raspberry Pi Compute Module 4

increase-pci-bar-space.sh

#!/bin/bash

# The default BAR address space available on the CM4 may be too small to allow
# some devices to initialize correctly. To avoid 'failed to assign memory'
# errors on boot, you can increase the range of the PCIe bus in the Raspberry
# Pi's Device Tree (a .dtb file specific to each Pi model).
#
# You should probably read up on Device Trees if you don't know what they are:
# https://www.raspberrypi.org/documentation/configuration/device-tree.md
#
# NOTE: The default BAR allocation was increased to 1 GB in this commit:
# https://github.com/raspberrypi/linux/commit/54db4b2fa4d17251c2f6e639f849b27c3b553939

# Download the trial firmware from this post:
# https://www.raspberrypi.org/forums/viewtopic.php?p=1761834#p1761834
# and replace the corresponding files in the boot volume.

# Back up current Device Tree for the CM4.
sudo cp /boot/bcm2711-rpi-cm4.dtb /boot/bcm2711-rpi-cm4.dtb.bak

# Decompile the current Device Tree to a dts (source) file.
dtc -I dtb -O dts /boot/bcm2711-rpi-cm4.dtb -o ~/test.dts

# Edit the file and change the PCIe bus range as mentioned in:
# https://www.raspberrypi.org/forums/viewtopic.php?p=1746665#p1746665
nano ~/test.dts

# Replace the line that allocates 1 GB of RAM in the pci section:
#   ranges = <0x02000000 0x0 0xc0000000 0x6 0x00000000 0x0 0x40000000>;
#
# with the following line, which provides up to 8 GB of RAM:
#   ranges = <0x02000000 0x0 0x00000000 0x6 0x00000000 0x2 0x00000000>;

# Also replace the scb ranges to:
#    ranges = <0x0 0x7c000000  0x0 0xfc000000  0x0 0x03800000>,
#             <0x0 0x40000000  0x0 0xff800000  0x0 0x00800000>,
#             <0x6 0x00000000  0x6 0x00000000  0x2 0x00000000>,
#             <0x0 0x00000000  0x0 0x00000000  0x0 0xfc000000>;

# Recompile the Device Tree from the dts (source) file.
dtc -I dts -O dtb ~/test.dts -o ~/test.dtb

# Copy the new Device Tree into place.
sudo mv ~/test.dtb /boot/bcm2711-rpi-cm4.dtb

# Reboot.
sudo reboot

I'd never heard of this before until it came up at https://jamesachambers.com/pcie-1x-nvme-on-raspberry-pi-compute-module-4-guide/#comment-12952. John tried a Xawai adapter that I had previously tested with and had no trouble with. He is encountering the same error as RoboTakoh. Since I've used this adapter successfully before it can't be impossible like may be the case with the firewire card.

I can't explain why I never encountered it with the same adapter yet. Hopefully John will test some suggestions I had and some suggestions here. I may try doing it again with the latest Pi OS (I used 64 bit previously, one test I suggested was he try the 64 bit flavor to see if it makes any difference) to see if I can reproduce it on my end. I also linked him here to try some of the tweaks in the gist.

Most likely it just needs a firmware update as it sounds like this should be resolved and would explain why I never encountered it (I was late to the CM4 party).

Very strange issue, great work on this Jeff!

Thanks for the script @geerlingguy !
Do you know how to modify this on a PI5?

This may be useful on the Pi 5 as well (to your question @5erv3), since some larger graphics cards are setting down to 256 MB of BAR space as they have 12+ GB of VRAM: geerlingguy/raspberry-pi-pcie-devices#680