Install Coral AI PCIe Edge TPU on Raspberry Pi 5

coral-ai-pcie-edge-tpu-raspberrypi-5-setup

#!/bin/bash

cd /

sudo apt update

echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" | sudo tee /etc/apt/sources.list.d/coral-edgetpu.list

curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

sudo apt-get update


sudo apt-get install libedgetpu1-std

sudo apt install -y devscripts debhelper

sudo apt install dkms

sudo apt-get install dh-dkms

# Clone the Gasket driver repository
sudo git clone https://github.com/google/gasket-driver.git

# Change directory to the cloned repository
cd gasket-driver

# Build the Gasket driver package
sudo debuild -us -uc -tc -b

# Go back to the parent directory
cd ..

# Install the built Gasket driver package
sudo dpkg -i gasket-dkms_1.0-18_all.deb


sudo sh -c "echo 'SUBSYSTEM==\"apex\", MODE=\"0660\", GROUP=\"apex\"' >> /etc/udev/rules.d/65-apex.rules"

sudo groupadd apex

sudo adduser $USER apex


# Update the boot configuration for Raspberry Pi
echo "kernel=kernel8.img" | sudo tee -a /boot/firmware/config.txt

# Back up the Device Tree Blob (DTB) 
sudo cp /boot/firmware/bcm2712-rpi-5-b.dtb /boot/firmware/bcm2712-rpi-5-b.dtb.bak 

# Decompile the DTB into a DTS file 
sudo dtc -I dtb -O dts /boot/firmware/bcm2712-rpi-5-b.dtb -o ~/test.dts 

# Modify the Device Tree Source (DTS) 
sudo sed -i '/pcie@110000 {/,/};/{/msi-parent = <[^>]*>;/{s/msi-parent = <[^>]*>;/msi-parent = <0x67>;/}}' ~/test.dts

# Recompile the DTS back into a DTB 
sudo dtc -I dts -O dtb ~/test.dts -o ~/test.dtb 

# Replace the old DTB with the new one 
sudo mv ~/test.dtb /boot/firmware/bcm2712-rpi-5-b.dtb

sudo reboot now

@Reddimus I followed your guide for the pi 5 and can see /dev/apex_0 but can't run the pycoral example. I followed the troubleshooting steps here -- when I run sudo lspci -vvv|grep -i MSI-X I can see the MSI-X isn't enabled. Do I need to rewrite the device tree as the author here does?

Also note the orientation of the FPC ribbon, the white arrows must align or the coral won't be detected.

@dataslayermedia Thank you for the image. The cable was correct, but the clip on the TPU Hat end wasn't closed. I closed it and powered up, but MSI-X was still showing as disabled. I re-imaged the SD card and ran your script, it enabled MSI-X correctly. I was then able to install python 3.9.16 with pyenv and run the pycoral example (after opening permissions to /dev/apex_0).

@jenniferlianne I was working on your issue yesterday. I am glad you fixed your issue. However, may I see your output once you run sudo lspci -vvv|grep -i MSI-X now that your hardware works?

@Reddimus Certainly. When the output looks like this there is no delegation error when running the example:

Capabilities: [d0] MSI-X: Enable+ Count=128 Masked-
Capabilities: [b0] MSI-X: Enable+ Count=61 Masked-

In my first attempt, both lines showed 'Enable-'. After I wrote in yesterday I installed more software on the pi and noticed later the connection had stopped working when I brought up Frigate. It was possible the ribbon had been jostled. At that point the d0 line showed 'Enable-' while the b0 line showed Enable+. I took the ribbon out and put it back together. Both lines then showed Enable+ and I was able to run Frigate and detect a person in frame.

If there was a power-on-self-test that shows the ribbon is connected properly, that would help troubleshoot the build. Alternately I'm wondering if there's a boot sequence issue. My set up is new, purchased for this purpose, the only difference is I have an ActiveCooler, though it isn't plugged in.

@Reddimus I followed your guide as well and had the same issue as jenniferlianne when trying to run the example. I get this when running it

ValueError: Failed to load delegate from libedgetpu.so.1 <

And when running lspci -vvv|grep -i MSI-X I get

Capabilities: [d0] MSI-X: Enable- Count=128 Masked- Capabilities: [b0] MSI-X: Enable+ Count=61 Masked-

I'm sure I put the cables on right but I'll double check tomorrow and probably reimage the sd card like Jennifer did as well.

@jenniferlianne @swalker23 I have updated the Raspberry Pi 5 - Google Coral Edge M.2 TPU installation guide. I believe some update or dependency made the old guide unusable. However, we can fix those issues by running @dataslayermedia 's script first, then adding a udev rule for /dev/apex_0, and installing Pyenv (python 3.9.16).

My raspberry pi has upgraded to version 6.6 of the kernel, which is great because my old cameras now work. The only issue is that the Coral device no longer works. I've deployed the updated gasket drivers which appear to compile and work correctly with the new kernel tree. The issue appears to be with the PCIE device itself. If I leave /boot/firmware/bcm2712-rpi-5-b.dtb unchanged the apex devices come up but we have the interupt issue with the actual coral code. If I make the change to the device tree with the line sudo sed -i '/pcie@110000 {/,/msi-parent = <0x2[fc]>;/{s/<0x2f>/<0x66>/; s/<0x2c>/<0x66>/}' the apex driver no longer loads. The lspci command does not show the coral device and I see the following error in dmesg:

[ 0.383054] brcm-pcie 1000110000.pcie: host bridge /axi/pcie@110000 ranges:
[ 0.383060] brcm-pcie 1000110000.pcie: No bus range found for /axi/pcie@110000, using [bus 00-ff]
[ 0.383071] brcm-pcie 1000110000.pcie: MEM 0x1b00000000..0x1bfffffffb -> 0x0000000000
[ 0.383077] brcm-pcie 1000110000.pcie: MEM 0x1800000000..0x1affffffff -> 0x0400000000
[ 0.383083] brcm-pcie 1000110000.pcie: IB MEM 0x0000000000..0x0fffffffff -> 0x1000000000
[ 0.384430] brcm-pcie 1000110000.pcie: Forcing gen 2
[ 0.384440] brcm-pcie 1000110000.pcie: Unable to find MSI PCI address
[ 0.390913] brcm-pcie: probe of 1000110000.pcie failed with error -22

I'm guessing that changing the msi-parent value to 0x66 is no longer correct. Does anyone have any insight on what the new value should be, or where we could even look to find it?

Hello! I have the same problem after updating to 6.6.21-v8-16k+ the patch no longer works, I just had to change the

# Change the line: msi-parent = <0x2f>; (under `pcie@110000`)
# To: msi-parent = <0x66>;

Does anyone have an idea how to fix this?

SATA controller: ASMedia Technology Inc. Device 1064 (rev 02) (prog-if 01 [AHCI 1.0]

HatDrive! Bottom

 pcie@100000 {
                        compatible = "brcm,bcm2712-pcie";
                        reg = <0x10 0x100000 0x00 0x9310>;
                        device_type = "pci";
                        max-link-speed = <0x02>;
                        #address-cells = <0x03>;
                        #interrupt-cells = <0x01>;
                        #size-cells = <0x02>;
                        interrupt-parent = <0x01>;
                        interrupts = <0x00 0xd5 0x04 0x00 0xd6 0x04>;
                        interrupt-names = "pcie\0msi";
                        interrupt-map-mask = <0x00 0x00 0x00 0x07>;
                        interrupt-map = <0x00 0x00 0x00 0x01 0x01 0x00 0xd1 0x04 0x00 0x00 0x00 0x02 0x01 0x00 0xd2 0x04 0x00 0x00 0x00 0x03 0x01 0x00 0xd3 0x04 0x00 0x00 0x00 0x04 0x01 0x00 0xd4 0x04>;
                        resets = <0x29 0x05 0x29 0x2a 0x2a>;
                        reset-names = "swinit\0bridge\0rescal";
                        msi-controller;
                        msi-parent = <0x2b>;
                        ranges = <0x2000000 0x00 0x00 0x17 0x00 0x00 0xfffffffc 0x43000000 0x04 0x00 0x14 0x00 0x03 0x00>;
                        dma-ranges = <0x43000000 0x10 0x00 0x00 0x00 0x10 0x00>;
                        status = "disabled";
                        phandle = <0x2b>;
                };

                pcie@110000 {
                        compatible = "brcm,bcm2712-pcie";
                        reg = <0x10 0x110000 0x00 0x9310>;
                        device_type = "pci";
                        max-link-speed = <0x02>;
                        #address-cells = <0x03>;
                        #interrupt-cells = <0x01>;
                        #size-cells = <0x02>;
                        interrupt-parent = <0x01>;
                        interrupts = <0x00 0xdf 0x04 0x00 0xe0 0x04>;
                        interrupt-names = "pcie\0msi";
                        interrupt-map-mask = <0x00 0x00 0x00 0x07>;
                        interrupt-map = <0x00 0x00 0x00 0x01 0x01 0x00 0xdb 0x04 0x00 0x00 0x00 0x02 0x01 0x00 0xdc 0x04 0x00 0x00 0x00 0x03 0x01 0x00 0xdd 0x04 0x00 0x00 0x00 0x04 0x01 0x00 0xde 0x04>;
                        resets = <0x29 0x07 0x29 0x2b 0x2a>;
                        reset-names = "swinit\0bridge\0rescal";
                        msi-controller;
                        msi-parent = <0x2c>;
                        ranges = <0x2000000 0x00 0x00 0x1b 0x00 0x00 0xfffffffc 0x43000000 0x04 0x00 0x18 0x00 0x03 0x00>;
                        dma-ranges = <0x3000000 0x10 0x00 0x00 0x00 0x10 0x00>;
                        brcm,enable-l1ss;
                        status = "disabled";
                        phandle = <0x67>;
                };
 reset-controller@119500 {
                        compatible = "brcm,bcm7216-pcie-sata-rescal";
                        reg = <0x10 0x119500 0x00 0x10>;
                        #reset-cells = <0x00>;
                        phandle = <0x2a>;
                };

                pcie@120000 {
                        compatible = "brcm,bcm2712-pcie";
                        reg = <0x10 0x120000 0x00 0x9310>;
                        device_type = "pci";
                        max-link-speed = <0x02>;
                        #address-cells = <0x03>;
                        #interrupt-cells = <0x01>;
                        #size-cells = <0x02>;
                        interrupt-parent = <0x01>;
                        interrupts = <0x00 0xe9 0x04 0x00 0xea 0x04>;
                        interrupt-names = "pcie\0msi";
                        interrupt-map-mask = <0x00 0x00 0x00 0x07>;
                        interrupt-map = <0x00 0x00 0x00 0x01 0x01 0x00 0xe5 0x04 0x00 0x00 0x00 0x02 0x01 0x00 0xe6 0x04 0x00 0x00 0x00 0x03 0x01 0x00 0xe7 0x04 0x00 0x00 0x00 0x04 0x01 0x00 0xe8 0x04>;
                        resets = <0x29 0x20 0x29 0x2c 0x2a>;
                        reset-names = "swinit\0bridge\0rescal";
                        msi-controller;
                        msi-parent = <0x2d>;
                        ranges = <0x2000000 0x00 0x00 0x1f 0x00 0x00 0xfffffffc 0x43000000 0x04 0x00 0x1c 0x00 0x03 0x00>;
                        dma-ranges = <0x2000000 0x00 0x00 0x1f 0x00 0x00 0x400000 0x43000000 0x10 0x00 0x00 0x00 0x10 0x00>;
                        brcm,enable-l1ss;
                        status = "okay";
                        brcm,enable-mps-rcb;
                        brcm,vdm-qos-map = <0xbbaa9888>;
                        aspm-no-l0s;
                        phandle = <0xaf>;

root@Pi-Nas:~# lspci
0000:00:00.0 PCI bridge: Broadcom Inc. and subsidiaries Device 2712 (rev 21)
0000:01:00.0 SATA controller: ASMedia Technology Inc. Device 1064 (rev 02)
0001:00:00.0 PCI bridge: Broadcom Inc. and subsidiaries Device 2712 (rev 21)
0001:01:00.0 Ethernet controller: Device 1de4:0001

root@Pi-Nas:~# lspci -vvv | grep -i MSI-X
        Capabilities: [b0] MSI-X: Enable+ Count=61 Masked-

root@Pi-Nas:~# lspci -k
0000:00:00.0 PCI bridge: Broadcom Inc. and subsidiaries Device 2712 (rev 21)
        Kernel driver in use: pcieport
0000:01:00.0 SATA controller: ASMedia Technology Inc. Device 1064 (rev 02)
        Subsystem: ASMedia Technology Inc. Device 2116
        Kernel driver in use: ahci
        Kernel modules: ahci
0001:00:00.0 PCI bridge: Broadcom Inc. and subsidiaries Device 2712 (rev 21)
        Kernel driver in use: pcieport
0001:01:00.0 Ethernet controller: Device 1de4:0001
        Kernel driver in use: rp1

root@Pi-Nas:~# lspci -v 
0000:01:00.0 SATA controller: ASMedia Technology Inc. Device 1064 (rev 02) (prog-if 01 [AHCI 1.0])
        Subsystem: ASMedia Technology Inc. Device 2116
        !!! Unknown header type 7f
        Memory at 1b00080000 (32-bit, non-prefetchable) [size=8K]
        Memory at 1b00082000 (32-bit, non-prefetchable) [size=8K]
        Expansion ROM at 1b00000000 [disabled] [size=512K]
        Kernel driver in use: ahci
        Kernel modules: ahci

I've been fiddling and managed to get this working on the Linux pi5 6.6.20+rpt-rpi-v8 kernel. Instead of using the value <0x66> I used <0x67> which matches the phandle value (which was previsouly <0x66> and now it works. So to be clear for the 6.6 kernal the change is:
msi-parent=<0x2c> should be changed to msi-parent=<0x67>

I believe the Linux kernel version 6.6 made the script incompatible. The following kernel version was working for me 1 week ago:

user@raspberrypi5: $ uname -r
6.1.0-rpi7-rpi-v8

If anyone knows how to downgrade the Raspberry Pi 5 kernel please let me know at the forked gist.
Thank you.

Hello, does anyone know of any rpi HAT that can provide two pcie slots. One for the coral M2 and the other for a NVME? thanks

I've been fiddling and managed to get this working on the Linux pi5 6.6.20+rpt-rpi-v8 kernel. Instead of using the value <0x66> I used <0x67> which matches the phandle value (which was previsouly <0x66> and now it works. So to be clear for the 6.6 kernal the change is:
msi-parent=<0x2c> should be changed to msi-parent=<0x67>

Thank you very much It worked for me too...

@ezaul did you install the drivers (especially Gasket) before your Pi upgraded the kernel?

I'm struggling with gasket-dkms installation failures.
@Jumble0470 you mention updated versions, can you provide a link.

Setting up gasket-dkms (1.0-18) ...
Loading new gasket-1.0 DKMS files...
Deprecated feature: REMAKE_INITRD (/usr/src/gasket-1.0/dkms.conf)
Building for 6.6.20+rpt-rpi-2712 6.6.20+rpt-rpi-v8
Building initial module for 6.6.20+rpt-rpi-2712
Deprecated feature: REMAKE_INITRD (/var/lib/dkms/gasket/1.0/source/dkms.conf)
Error! Bad return status for module build on kernel: 6.6.20+rpt-rpi-2712 (aarch64)
Consult /var/lib/dkms/gasket/1.0/build/make.log for more information.
dpkg: error processing package gasket-dkms (--configure):
 installed gasket-dkms package post-installation script subprocess returned error exit status 10
Processing triggers for man-db (2.11.2-2) ...
Errors were encountered while processing:
 gasket-dkms
E: Sub-process /usr/bin/dpkg returned an error code (1)

askpatrickw

Hi askpatrickw

Sorry friend, I don't use gasket-dkms, I ended up here on Google search with the same problem that you have to change rasp's dtb

What I use is
HatDrive!
SATA Controller: ASMedia Technology Inc. Device 1064 (rev 02) (prog-if 01 [AHCI 1.0]

that to work it needs to change

Change the line: msi-parent = <0x2c>; (under pcie@110000)
To: msi-parent = <0x67>;
Then save the file.

https://forums.raspberrypi.com/viewtopic.php?t=363682

Quick update - The issue was with the MPW7 hat. I switched to a Pineberry Pi base and I now have the apex_0 device.

Hi, have you managed to make the MPW7 work?

Hello, does anyone know of any rpi HAT that can provide two pcie slots. One for the coral M2 and the other for a NVME? thanks

https://pineberrypi.com/pages/products

Confirming that the MPW7 hat does not work. The TPU appears at first but after installing the driver it disappears. The Pineberry Pi AI hat works.

This one
https://pineberrypi.com/products/hatdrive-ai-nvme-2230-2242-gen-2-coral-edge-tpu-for-raspberry-pi-5

@Gus1994 @dataslayermedia @askpatrickw @ezaul i am getting this error
mv: failed to preserve ownership for '/boot/firmware/bcm2712-rpi-5-b.dtb': Operation not permitted
sudo mv ~/test.dtb /boot/firmware/bcm2712-rpi-5-b.dtb
can someone please help me i have a project and my Raspberry pi is not able to recognize the peripheral device

mv: failed to preserve ownership for '/boot/firmware/bcm2712-rpi-5-b.dtb': Operation not permitted sudo mv ~/test.dtb /boot/firmware/bcm2712-rpi-5-b.dtb

It's just saying it had to change permissions while moving the file. If you check the destination folder it should be there.

@mjforan but the problem is that i cannot check the destination folder as it is not even a folder rather its a device tree file in which made changes
these are the commands that i followed
sudo cp /boot/firmware/bcm2712-rpi-5-b.dtb /boot/firmware/bcm2712-rpi-5-b.dtb.bak

Decompile the DTB into a Device Tree Source (DTS) file for editing

- Ignore any warnings during decompilation

sudo dtc -I dtb -O dts /boot/firmware/bcm2712-rpi-5-b.dtb -o ~/test.dts

Modify the Device Tree Source

Replace 'msi-parent' value to change the Message Signaled Interrupts (MSI) parent setting

sudo sed -i '/pcie@110000 {/,/msi-parent = <0x2[fc]>;/{s/<0x2f>/<0x66>/; s/<0x2c>/<0x66>/}' ~/test.dts

Recompile the DTS back into a DTB

sudo dtc -I dts -O dtb ~/test.dts -o ~/test.dtb

Replace the old DTB with the new one

sudo mv ~/test.dtb /boot/firmware/bcm2712-rpi-5-b.dtb

now the problem is that it is still not recognizing my TPU
and i have made changes of 67 instead of 66 as said above
the script is also outdated if you can tell me your way of recognizing if you can for pineberry
as one of my friends had a NVME base and a M B/E key it worked fine for him just using the offical coral documentation but for me who has the "official pineberry stuff" it is not working i have even contacted the pineberry customer care they gave me link to this git repo
if someone can please HELPPPPP!!!!!!!!!!!!!!!!!!

@ChetanKukreja

the problem is that i cannot check the destination folder as it is not even a folder rather its a device tree file in which made changes

You are using the mv command, which takes two arguments. The first argument is the source path and the second is the destination. You can use the ls command on the destination path to verify the file was moved correctly.

Did you reboot after applying these changes? Did you verify you are running the correct kernel? Have you installed the driver packages? Does the device show at all in lspci?

@mjforan can you please guide me through this i have given you the image of the ls command also given you the reply when i lspci chetan@raspberrypi:~ $ lspci 00:00.0 PCI bridge: Broadcom Inc. and subsidiaries Device 2712 (rev 21) 01:00.0 Ethernet controller: Device 1de4:0001 (can you also tell me what do you mean by correct kernel)?

@ChetanKukreja

You must think for yourself a little my friend. You can see in that image that the file you moved is there in the correct place.

Line 27 of the gist shows how to set the correct kernel version, and it can be verified by uname -r , making sure -v8 is at the end.

The TPU device does not show up in your lspci output. Make sure the hardware is connected properly.

@mjforan i will tell you in detail what i did till now and what is my terminal i am on
Linux raspberrypi 6.6.20+rpt-rpi-v8 #1 SMP PREEMPT Debian 1:6.6.20-1+rpt1 (2024-03-07) aarch64 GNU/Linux
i followed the following commands
sudo apt-get install vim curl wget -y

Add the Google Coral Edge TPU repository to the system's source list

This repository contains packages specifically for the Coral Edge TPU

echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" | sudo tee /etc/apt/sources.list.d/coral-edgetpu.list

Download and add the Google GPG key to the system

This key is used to ensure the integrity and authenticity of the packages

curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

Update the package lists for packages that need upgrading, as well as new packages that have just come to the repositories

sudo apt-get update
and then when it came to gasket it showed error so i installed gasket from a forum i found https://forum.proxmox.com/threads/update-error-with-coral-tpu-drivers.136888/#post-608975
using this i did install the gasket file also the libedgetpu1

now i followed this command
echo "kernel=kernel8.img" | sudo tee -a /boot/config.txt
but on opening found that /boot/config.txt is moved to /boot/firmware/config.txt
so made changes in that also
i added this from the jeffgerling blog
dtparam=pciex1
dtparam=pciex1_gen=2

now i executed the following commands sudo cp /boot/firmware/bcm2712-rpi-5-b.dtb /boot/firmware/bcm2712-rpi-5-b.dtb.bak

Decompile the current dtb (ignore warnings)

dtc -I dtb -O dts /boot/firmware/bcm2712-rpi-5-b.dtb -o ~/test.dts
and made changes to test.dtb
instead of 66 made it 67
then followed the following command
dtc -I dts -O dtb ~/test.dts -o ~/test.dtb
sudo mv ~/test.dtb /boot/firmware/bcm2712-rpi-5-b.dtb

after that i dont get a coral device

@ChetanKukreja
How is the coral device connected to your raspberry pi?

@mjforan

as you see the hat is getting power

@ChetanKukreja Looks ok to me. I suggest creating a help post somewhere else (stack overflow, raspberry pi forum, etc.) because this gist is not an appropriate place for extensive debugging help.

For any others experiencing the same issues I had, this helped:
google-coral/edgetpu#808 (comment)

For reference:
I followed Reddimus' guide.
I am on kernel 6.6.20 using the PineberryPi HatDrive! AI.
My issue was that my accelerator wasn't found (solved this using the aforementioned msi-parent = <0x67>).
My next issue was that gasket-dkms package would not correctly install.
What finally worked for me was the following from google-coral/edgetpu issue #808 as follows:

@themana

sudo apt update
sudo apt upgrade
sudo apt install devscripts debhelper -y

Then clone and the rest

❯ git clone https://github.com/google/gasket-driver.git
Cloning into 'gasket-driver'...
...
❯ cd gasket-driver; debuild -us -uc -tc -b; cd ..
...
dpkg-deb: building package 'gasket-dkms' in '../gasket-dkms_1.0-18_all.deb'.
...
❯ ls -l gasket-dkms*
-rw-r--r-- 1 dewet dewet 49000 Jan 17 13:17 gasket-dkms_1.0-18_all.deb
-rw-r--r-- 1 dewet dewet  1788 Jan 17 13:18 gasket-dkms_1.0-18_amd64.build
-rw-r--r-- 1 dewet dewet  5642 Jan 17 13:17 gasket-dkms_1.0-18_amd64.buildinfo
-rw-r--r-- 1 dewet dewet  1017 Jan 17 13:17 gasket-dkms_1.0-18_amd64.changes

You can install that .deb on any systems that need to build the kernel module, and it will be rebuilt automatically with newer kernel packages being installed:

❯ sudo dpkg -i gasket-dkms_1.0-18_all.deb
...
Setting up gasket-dkms (1.0-18) ...
...
Building for 6.5.0-14-generic
Building initial module for 6.5.0-14-generic
...
depmod...
Time: 0h:00m:10s
❯ sudo modprobe apex
❯ lsmod | grep apex
apex                   28672  0
gasket                135168  1 apex

Hope this also helps others in the same boat!