#!/bin/bash | |
# The default BAR address space available on the CM4 is too small to allow | |
# devices like GPUs to initialize correctly. To avoid 'failed to assign memory' | |
# errors on boot, we need to increase the range of the PCIe bus in the Raspberry | |
# Pi's Device Tree (a .dtb file specific to each Pi model). | |
# | |
# You should probably read up on Device Trees if you don't know what they are: | |
# https://www.raspberrypi.org/documentation/configuration/device-tree.md | |
# | |
# NOTE: The default BAR allocation was increased to 1 GB in this commit: | |
# https://github.com/raspberrypi/linux/commit/54db4b2fa4d17251c2f6e639f849b27c3b553939 | |
# | |
# Once that hits the stable Raspberry Pi OS image (or you can install via the | |
# beta firmware channel), the BAR change instructions below will not be needed. | |
# Back up current Device Tree for the CM4. | |
sudo cp /boot/bcm2711-rpi-cm4.dtb /boot/bcm2711-rpi-cm4.dtb.bak | |
# Decompile the current Device Tree to a dts (source) file. | |
dtc -I dtb -O dts /boot/bcm2711-rpi-cm4.dtb -o ~/test.dts | |
# Edit the file and change the PCIe bus range as mentioned in: | |
# https://www.raspberrypi.org/forums/viewtopic.php?p=1746665#p1746665 | |
nano ~/test.dts | |
# Replace the line that allocates ~64 MB of RAM in the pci section: | |
# ranges = < 0x2000000 0x00 0xf8000000 0x06 0x00 0x00 0x4000000 >; | |
# | |
# with the following line, which provides up to 1 GB of RAM: | |
# ranges = <0x02000000 0x0 0xc0000000 0x6 0x00000000 0x0 0x40000000>; | |
# | |
# alternatively, use the following line, for ~256 MB of RAM: | |
# ranges = <0x02000000 0x0 0xe0000000 0x6 0x00000000 0x0 0x10000000>; | |
# Recompile the Device Tree from the dts (source) file. | |
dtc -I dts -O dtb ~/test.dts -o ~/test.dtb | |
# Copy the new Device Tree into place. | |
sudo mv ~/test.dtb /boot/bcm2711-rpi-cm4.dtb | |
# Reboot. | |
sudo reboot |
This comment has been minimized.
This comment has been minimized.
See related gist: Set up the Nvidia GeForce GT 710 on Raspberry Pi Compute Module 4 |
This comment has been minimized.
This comment has been minimized.
It's worth noting that the outbound window isn't allocating RAM, it's reserving a chunk of the PCI address space for devices and mapping it into the ARM's physical address space. No RAM is lost in the process. The default outbound window on rpi-5.9.y (available from our "next" firmware branch) and rpi-5.10.y has now been expanded to 1GB, so in the future the dtb surgery described above won't be necessary. |
This comment has been minimized.
This comment has been minimized.
@pelwell - ah, thanks for clearing up my misunderstanding, and also for helping get this change into the updated firmware! That'll make it easier and I can avoid patching the DTB. And the commit: raspberrypi/linux@54db4b2 |
This comment has been minimized.
This comment has been minimized.
Also can do Note: |
This comment has been minimized.
This comment has been minimized.
Another update: this post in the Pi Forums contains some trial firmware that allows the BAR space to take up the maximum possible space on the Pi (8GB of PCIe space for any Pi with >= 4 GB of RAM). To use it:
More details on the entire process used to try to get a GPU working are in this comment: geerlingguy/raspberry-pi-pcie-devices#6 (comment) |
This comment has been minimized.
This comment has been minimized.
Is the "ranges = /bits/ 64 <0x0 0x600000000 0x00000000c0000000>;" syntax applicable to this version of the Device Tree Compiler (dtc)? |
This comment has been minimized.
This change is necessary to make the Raspberry Pi Compute Module 4 handle many 3rd party PCIe adapters (e.g. GPUs, SATA cards, etc.), as they require more BAR (Base Address Register) PCI configuration space than is provided by default (~64 MB). The default amount was adequate for simpler devices, like the VL805 USB 3.0 controller used in the Raspberry Pi 4, but many PCIe devices require at least hundreds of MB of BAR space, so this script increases the limit to 1 GB.
The default limit may be increased in future versions of Raspberry Pi OS, since this change is fairly innocuous.
For some examples of devices that may need larger BAR space, see my Raspberry Pi PCIe device compatibility database