-
-
Save dataslayermedia/714ec5a9601249d9ee754919dea49c7e to your computer and use it in GitHub Desktop.
| #!/bin/bash | |
| cd / | |
| sudo apt update | |
| echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" | sudo tee /etc/apt/sources.list.d/coral-edgetpu.list | |
| curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - | |
| sudo apt-get update | |
| sudo apt-get install libedgetpu1-std | |
| sudo apt install -y devscripts debhelper | |
| sudo apt install dkms | |
| sudo apt-get install dh-dkms | |
| # Clone the Gasket driver repository | |
| sudo git clone https://github.com/google/gasket-driver.git | |
| # Change directory to the cloned repository | |
| cd gasket-driver | |
| # Build the Gasket driver package | |
| sudo debuild -us -uc -tc -b | |
| # Go back to the parent directory | |
| cd .. | |
| # Install the built Gasket driver package | |
| sudo dpkg -i gasket-dkms_1.0-18_all.deb | |
| sudo sh -c "echo 'SUBSYSTEM==\"apex\", MODE=\"0660\", GROUP=\"apex\"' >> /etc/udev/rules.d/65-apex.rules" | |
| sudo groupadd apex | |
| sudo adduser $USER apex | |
| # Update the boot configuration for Raspberry Pi | |
| echo "kernel=kernel8.img" | sudo tee -a /boot/firmware/config.txt | |
| # Back up the Device Tree Blob (DTB) | |
| sudo cp /boot/firmware/bcm2712-rpi-5-b.dtb /boot/firmware/bcm2712-rpi-5-b.dtb.bak | |
| # Decompile the DTB into a DTS file | |
| sudo dtc -I dtb -O dts /boot/firmware/bcm2712-rpi-5-b.dtb -o ~/test.dts | |
| # Modify the Device Tree Source (DTS) | |
| sudo sed -i '/pcie@110000 {/,/};/{/msi-parent = <[^>]*>;/{s/msi-parent = <[^>]*>;/msi-parent = <0x67>;/}}' ~/test.dts | |
| # Recompile the DTS back into a DTB | |
| sudo dtc -I dts -O dtb ~/test.dts -o ~/test.dtb | |
| # Replace the old DTB with the new one | |
| sudo mv ~/test.dtb /boot/firmware/bcm2712-rpi-5-b.dtb | |
| sudo reboot now |
I believe the Linux kernel version 6.6 made the script incompatible. The following kernel version was working for me 1 week ago:
user@raspberrypi5: $ uname -r
6.1.0-rpi7-rpi-v8If anyone knows how to downgrade the Raspberry Pi 5 kernel please let me know at the forked gist.
Thank you.
Hello, does anyone know of any rpi HAT that can provide two pcie slots. One for the coral M2 and the other for a NVME? thanks
I've been fiddling and managed to get this working on the Linux pi5 6.6.20+rpt-rpi-v8 kernel. Instead of using the value <0x66> I used <0x67> which matches the phandle value (which was previsouly <0x66> and now it works. So to be clear for the 6.6 kernal the change is:
msi-parent=<0x2c> should be changed to msi-parent=<0x67>
Thank you very much It worked for me too...
@ezaul did you install the drivers (especially Gasket) before your Pi upgraded the kernel?
I'm struggling with gasket-dkms installation failures.
@Jumble0470 you mention updated versions, can you provide a link.
Setting up gasket-dkms (1.0-18) ...
Loading new gasket-1.0 DKMS files...
Deprecated feature: REMAKE_INITRD (/usr/src/gasket-1.0/dkms.conf)
Building for 6.6.20+rpt-rpi-2712 6.6.20+rpt-rpi-v8
Building initial module for 6.6.20+rpt-rpi-2712
Deprecated feature: REMAKE_INITRD (/var/lib/dkms/gasket/1.0/source/dkms.conf)
Error! Bad return status for module build on kernel: 6.6.20+rpt-rpi-2712 (aarch64)
Consult /var/lib/dkms/gasket/1.0/build/make.log for more information.
dpkg: error processing package gasket-dkms (--configure):
installed gasket-dkms package post-installation script subprocess returned error exit status 10
Processing triggers for man-db (2.11.2-2) ...
Errors were encountered while processing:
gasket-dkms
E: Sub-process /usr/bin/dpkg returned an error code (1)
askpatrickw
Hi askpatrickw
Sorry friend, I don't use gasket-dkms, I ended up here on Google search with the same problem that you have to change rasp's dtb
What I use is
HatDrive!
SATA Controller: ASMedia Technology Inc. Device 1064 (rev 02) (prog-if 01 [AHCI 1.0]
that to work it needs to change
Change the line: msi-parent = <0x2c>; (under pcie@110000)
To: msi-parent = <0x67>;
Then save the file.
Quick update - The issue was with the MPW7 hat. I switched to a Pineberry Pi base and I now have the apex_0 device.
Hi, have you managed to make the MPW7 work?
Hello, does anyone know of any rpi HAT that can provide two pcie slots. One for the coral M2 and the other for a NVME? thanks
Confirming that the MPW7 hat does not work. The TPU appears at first but after installing the driver it disappears. The Pineberry Pi AI hat works.
@Gus1994 @dataslayermedia @askpatrickw @ezaul i am getting this error
mv: failed to preserve ownership for '/boot/firmware/bcm2712-rpi-5-b.dtb': Operation not permitted
sudo mv ~/test.dtb /boot/firmware/bcm2712-rpi-5-b.dtb
can someone please help me i have a project and my Raspberry pi is not able to recognize the peripheral device
mv: failed to preserve ownership for '/boot/firmware/bcm2712-rpi-5-b.dtb': Operation not permitted sudo mv ~/test.dtb /boot/firmware/bcm2712-rpi-5-b.dtb
It's just saying it had to change permissions while moving the file. If you check the destination folder it should be there.
@mjforan but the problem is that i cannot check the destination folder as it is not even a folder rather its a device tree file in which made changes
these are the commands that i followed
sudo cp /boot/firmware/bcm2712-rpi-5-b.dtb /boot/firmware/bcm2712-rpi-5-b.dtb.bak
Decompile the DTB into a Device Tree Source (DTS) file for editing
- Ignore any warnings during decompilation
sudo dtc -I dtb -O dts /boot/firmware/bcm2712-rpi-5-b.dtb -o ~/test.dts
Modify the Device Tree Source
Replace 'msi-parent' value to change the Message Signaled Interrupts (MSI) parent setting
sudo sed -i '/pcie@110000 {/,/msi-parent = <0x2[fc]>;/{s/<0x2f>/<0x66>/; s/<0x2c>/<0x66>/}' ~/test.dts
Recompile the DTS back into a DTB
sudo dtc -I dts -O dtb ~/test.dts -o ~/test.dtb
Replace the old DTB with the new one
sudo mv ~/test.dtb /boot/firmware/bcm2712-rpi-5-b.dtb
now the problem is that it is still not recognizing my TPU
and i have made changes of 67 instead of 66 as said above
the script is also outdated if you can tell me your way of recognizing if you can for pineberry
as one of my friends had a NVME base and a M B/E key it worked fine for him just using the offical coral documentation but for me who has the "official pineberry stuff" it is not working i have even contacted the pineberry customer care they gave me link to this git repo
if someone can please HELPPPPP!!!!!!!!!!!!!!!!!!
the problem is that i cannot check the destination folder as it is not even a folder rather its a device tree file in which made changes
You are using the mv command, which takes two arguments. The first argument is the source path and the second is the destination. You can use the ls command on the destination path to verify the file was moved correctly.
Did you reboot after applying these changes? Did you verify you are running the correct kernel? Have you installed the driver packages? Does the device show at all in lspci?
@mjforan can you please guide me through this
i have given you the image of the ls command
also given you the reply when i lspci
chetan@raspberrypi:~ $ lspci
00:00.0 PCI bridge: Broadcom Inc. and subsidiaries Device 2712 (rev 21)
01:00.0 Ethernet controller: Device 1de4:0001
(can you also tell me what do you mean by correct kernel)?
You must think for yourself a little my friend. You can see in that image that the file you moved is there in the correct place.
Line 27 of the gist shows how to set the correct kernel version, and it can be verified by uname -r , making sure -v8 is at the end.
The TPU device does not show up in your lspci output. Make sure the hardware is connected properly.
@mjforan i will tell you in detail what i did till now and what is my terminal i am on
Linux raspberrypi 6.6.20+rpt-rpi-v8 #1 SMP PREEMPT Debian 1:6.6.20-1+rpt1 (2024-03-07) aarch64 GNU/Linux
i followed the following commands
sudo apt-get install vim curl wget -y
Add the Google Coral Edge TPU repository to the system's source list
This repository contains packages specifically for the Coral Edge TPU
echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" | sudo tee /etc/apt/sources.list.d/coral-edgetpu.list
Download and add the Google GPG key to the system
This key is used to ensure the integrity and authenticity of the packages
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
Update the package lists for packages that need upgrading, as well as new packages that have just come to the repositories
sudo apt-get update
and then when it came to gasket it showed error so i installed gasket from a forum i found https://forum.proxmox.com/threads/update-error-with-coral-tpu-drivers.136888/#post-608975
using this i did install the gasket file also the libedgetpu1
now i followed this command
echo "kernel=kernel8.img" | sudo tee -a /boot/config.txt
but on opening found that /boot/config.txt is moved to /boot/firmware/config.txt
so made changes in that also
i added this from the jeffgerling blog
dtparam=pciex1
dtparam=pciex1_gen=2
now i executed the following commands sudo cp /boot/firmware/bcm2712-rpi-5-b.dtb /boot/firmware/bcm2712-rpi-5-b.dtb.bak
Decompile the current dtb (ignore warnings)
dtc -I dtb -O dts /boot/firmware/bcm2712-rpi-5-b.dtb -o ~/test.dts
and made changes to test.dtb
instead of 66 made it 67
then followed the following command
dtc -I dts -O dtb ~/test.dts -o ~/test.dtb
sudo mv ~/test.dtb /boot/firmware/bcm2712-rpi-5-b.dtb
after that i dont get a coral device
@ChetanKukreja
How is the coral device connected to your raspberry pi?
@mjforan
as you see the hat is getting power
@ChetanKukreja Looks ok to me. I suggest creating a help post somewhere else (stack overflow, raspberry pi forum, etc.) because this gist is not an appropriate place for extensive debugging help.
For any others experiencing the same issues I had, this helped:
google-coral/edgetpu#808 (comment)
For reference:
I followed Reddimus' guide.
I am on kernel 6.6.20 using the PineberryPi HatDrive! AI.
My issue was that my accelerator wasn't found (solved this using the aforementioned msi-parent = <0x67>).
My next issue was that gasket-dkms package would not correctly install.
What finally worked for me was the following from google-coral/edgetpu issue #808 as follows:
sudo apt update sudo apt upgrade sudo apt install devscripts debhelper -yThen clone and the rest
❯ git clone https://github.com/google/gasket-driver.git Cloning into 'gasket-driver'... ... ❯ cd gasket-driver; debuild -us -uc -tc -b; cd .. ... dpkg-deb: building package 'gasket-dkms' in '../gasket-dkms_1.0-18_all.deb'. ... ❯ ls -l gasket-dkms* -rw-r--r-- 1 dewet dewet 49000 Jan 17 13:17 gasket-dkms_1.0-18_all.deb -rw-r--r-- 1 dewet dewet 1788 Jan 17 13:18 gasket-dkms_1.0-18_amd64.build -rw-r--r-- 1 dewet dewet 5642 Jan 17 13:17 gasket-dkms_1.0-18_amd64.buildinfo -rw-r--r-- 1 dewet dewet 1017 Jan 17 13:17 gasket-dkms_1.0-18_amd64.changesYou can install that
.debon any systems that need to build the kernel module, and it will be rebuilt automatically with newer kernel packages being installed:❯ sudo dpkg -i gasket-dkms_1.0-18_all.deb ... Setting up gasket-dkms (1.0-18) ... ... Building for 6.5.0-14-generic Building initial module for 6.5.0-14-generic ... depmod... Time: 0h:00m:10s ❯ sudo modprobe apex ❯ lsmod | grep apex apex 28672 0 gasket 135168 1 apex
Hope this also helps others in the same boat!
Had to change line #29 to:
echo "kernel=kernel8.img" | sudo tee -a /boot/firmware/config.txt
Had to change line #29 to:
echo "kernel=kernel8.img" | sudo tee -a /boot/firmware/config.txt
Good catch for Raspbian OS latest on the RPI 5.
cat /boot/config.txt
DO NOT EDIT THIS FILE
The file you are looking for has moved to /boot/firmware/config.txt
Yeaha! I have the Dual TPU Running at the AI Hat with bigs
thank you @feranick feranick GH Page and thank you @dataslayermedia for this script
a few changes at RPi5:
Linux escapepod 6.6.20+rpt-rpi-v8 #1 SMP PREEMPT Debian 1:6.6.20-1+rpt1 (2024-03-07) aarch64 GNU/Linux
echo "kernel=kernel8.img" | sudo tee -a /boot/firmware/config.txt
echo "dtparam=pciex1" | sudo tee -a /boot/firmware/config.txt
echo "pciex1_gen=3" | sudo tee -a /boot/firmware/config.txt
Maybe this is possible in an Onliner
change dts stuff, thanks at @ChetanKukreja
sudo sed -i '/pcie@110000 {/,/msi-parent = <0x2[fc]>;/{s/<0x2f>/<0x67>/; s/<0x2c>/<0x67>/}' ~/test.dts
Python Modules @ Python 3.11:
- numpy 1.26.4
- pillow 10.3.0
- pycoral 0.2.0 from Feranick/pycoral
python -c "import pycoral; print(pycoral.__version__)"
2.0.1
- FFLite-runtime from Feranick/TFlite-builds
python -c "import tflite_runtime; print(tflite_runtime.__version__)"
2.16.1
i ve init my project with rye my pyproject.toml is like this
[project]
name = "coral-t"
version = "0.1.0"
description = "Add your description here"
dependencies = [
"tflite-runtime @ https://github.com/feranick/TFlite-builds/releases/download/v.2.16.1/tflite_runtime-2.16.1-cp311-cp311-linux_aarch64.whl",
"pycoral @ https://github.com/feranick/pycoral/releases/download/v2.0.1TF2.16.1/pycoral-2.0.1-cp311-cp311-linux_aarch64.whl",
]
readme = "README.md"
requires-python = ">= 3.11"
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.rye]
managed = true
dev-dependencies = []
[tool.hatch.metadata]
allow-direct-references = true
[tool.hatch.build.targets.wheel]
packages = ["src/coral_t"]
Output:
(coral-t) ➜ coral_t git:(master) ✗ python3 examples/classify_image.py
--model test_data/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite
--labels test_data/inat_bird_labels.txt
--input test_data/parrot.jpg
----Debug2----
test_data/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite
----INFERENCE TIME----
Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory.
12.9ms
2.7ms
2.7ms
2.7ms
2.7ms
-------RESULTS--------
Ara macao (Scarlet Macaw): 0.75781
Next question is. How can i add the second tpu. I have an Dual TPU M2 Card but can only find /dev/apex_0
@dasTholo The Pi 5 only exposes one PCIe lane so you will need a HAT to multiplex both devices over the single lane. I don't think there are any dual-lane E HATs yet, but Pineberry Pi does make dual-slot HATs. You might be able to install two single-TPU devices, one A+E and one B+M key.
@mjforan
That is sad. I didn't know that. I had specially ordered the DualTPU. But I don't quite understand (I haven't found any other docs on this yet) how you can address both TPUs on an x86, for example. As they are both on the same PCIe port.
I have the Pineberry with M-Key and E-Key
https://pineberrypi.com/products/hatdrive-ai-nvme-2230-2242-gen-2-coral-edge-tpu-for-raspberry-pi-5
If I understand you correctly, then I could put another TPU in the other slot? This one here?
https://coral.ai/products/m2-accelerator-bm/
@dasTholo the m.2 specification is not just a single PCIe "port", but has many different configurations. The "key" is the shape of the connector. The A+E variant of the coral can plug in to the E key port on a computer which is commonly used for WiFi cards while the B+M variant can use the M key port typically used for SSDs. On the dual TPU version there are two separate PCIe devices, each of which requires a separate x1 PCIe interface (x1 means one lane). Even on larger computers, many E-key ports only have one PCIe interface - the coral page you linked to notes:
Although the M.2 Specification (section 5.1.2) declares E-key sockets provide two instances of PCIe x1, most manufacturers provide only one. To use both Edge TPUs, be sure your socket connects both instances to the host.
If I understand you correctly, then I could put another TPU in the other slot? This one here? https://coral.ai/products/m2-accelerator-bm/
Exactly. You will need to cut down the TPU to get it to fit (it is designed to do that). I can't personally guarantee that this will work - you might want to ask Pineberry Pi before buying another TPU. Let us know if it does work out for you!
@mjforan That is sad. I didn't know that. I had specially ordered the DualTPU. But I don't quite understand (I haven't found any other docs on this yet) how you can address both TPUs on an x86, for example. As they are both on the same PCIe port.
I have the Pineberry with M-Key and E-Key https://pineberrypi.com/products/hatdrive-ai-nvme-2230-2242-gen-2-coral-edge-tpu-for-raspberry-pi-5
If I understand you correctly, then I could put another TPU in the other slot? This one here? https://coral.ai/products/m2-accelerator-bm/
The dual TPU requires special considerations you need a specific adapter as well as a motherboard that will allow it. Magic Blue Smoke makes available a range of such adapters here.
But getting it to work on a Raspberry Pi 5 introduces yet another obstacle you would need a PCIe expansion slot and even then it's likely the motherboard will only make one TPU available. But I will try that soon to determine if it works.
I fixed the script as of April 7th, it looks like the gasket package now needs some finessing to work on the RPI. But the latest revision worked for me.
@dataslayermedia Thanks for the script. I set up this for a friend. I tested this against a fresh install and in addition to your script, I had to do following two steps in advance.
- "apt install dkms"
- Switching to X11 instead of Wayland using the raspi-config tool. Otherwise there was no GUI anymore, after reboot. Didn't found the root cause. Might be related to the HDMI Display type.
I ran into trouble with this script today with a fresh install of Rasp Pi OS on Rasp Pi 5. I believe the HAT is correctly installed considering the LED on it is green. Here's what I believe to be the relevant logs:
Setting up gasket-dkms (1.0-18) ...
Removing old gasket-1.0 DKMS files...
Deleting module gasket-1.0 completely from the DKMS tree.
Loading new gasket-1.0 DKMS files...
Deprecated feature: REMAKE_INITRD (/usr/src/gasket-1.0/dkms.conf)
Building for 6.6.20+rpt-rpi-2712 6.6.20+rpt-rpi-v8
Building initial module for 6.6.20+rpt-rpi-2712
Deprecated feature: REMAKE_INITRD (/var/lib/dkms/gasket/1.0/source/dkms.conf)
Error! Bad return status for module build on kernel: 6.6.20+rpt-rpi-2712 (aarch64)
Consult /var/lib/dkms/gasket/1.0/build/make.log for more information.
dpkg: error processing package gasket-dkms (--configure):
installed gasket-dkms package post-installation script subprocess returned error exit status 10
Errors were encountered while processing:
gasket-dkms
E: Sub-process /usr/bin/dpkg returned an error code (1)
DKMS make.log for gasket-1.0 for kernel 6.6.20+rpt-rpi-v8 (aarch64)
Tue 16 Apr 22:56:32 EDT 2024
make: Entering directory '/usr/src/linux-headers-6.6.20+rpt-rpi-v8'
/var/lib/dkms/gasket/1.0/build/gasket core.o
/var/lib/dkms/gasket/1.0/build/gasket_ioctl.o
/var/lib/dkms/gasket/1.0/build/gasket_interrupt.o
/var/lib/dkms/gasket/1.0/build/gasket_page_table.o
/var/lib/dkms/gasket/1.0/build/gasket_sysfs.o
/var/lib/dkms/gasket/1.0/build/apex_driver.o
/var/lib/dkms/gasket/1.0/build/apex_driver.c: 1143:12: warning: 'apex_pci_resume' defined but not used [-Wunused-function]
1143 | static int apex pci_resume (struct pci_dev *pci_dev)
/var/lib/dkms/gasket/1.0/build/apex_driver.c: 1128:12: warning: 'apex_pci_suspend' defined but not used [-Wunused-function]
1128 | static int apex pci_suspend (struct pci_dev *pci_dev, pm_message_t state) {
/var/lib/dkms/gasket/1.0/build/apex.o
/var/lib/dkms/gasket/1.0/build/gasket.o
MODPOST /var/lib/dkms/gasket/1.0/build/Module. symvers
/var/lib/dkms/gasket/1.0/build/gasket.mod.o
/var/lib/dkms/gasket/1.0/build/apex.mod. o
/var/lib/dkms/gasket/1.0/build/apex. ko
/var/lib/dkms/gasket/1.0/build/gasket. ko
make: Leaving directory '/usr/src/linux-headers-6.6.20+rpt-rpi-v8'
EDIT: I read the whole thread and used these commands before re-running the script and it seems to have worked!
sudo apt update
sudo apt upgrade
sudo apt install devscripts debhelper -y
Hi
Thankyou for creating this. I have been using it to install onto the Raspberry Pi OS Lite (64bit). I don't know if there are more things missing on the Lite version that the script relies on, but the build of gasket fails. When I repeat the build I got this:
sudo debuild -us -uc -tc -b
dpkg-buildpackage -us -uc -ui -tc -b
dpkg-buildpackage: info: source package gasket-dkms
dpkg-buildpackage: info: source version 1.0-18
dpkg-buildpackage: info: source distribution unstable
dpkg-buildpackage: info: source changed by Coral <coral-support@google.com>
dpkg-source --before-build .
dpkg-buildpackage: info: host architecture arm64
dpkg-checkbuilddeps: error: Unmet build dependencies: dkms
dpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting
dpkg-buildpackage: warning: (Use -d flag to override.)
debuild: fatal error at line 1182:
dpkg-buildpackage -us -uc -ui -tc -b failed
After a few attempts, I found that running before running your script meant it worked:
sudo apt-get install raspberrypi-kernel-headers
@pierslawson, are you seeing /dev/apex0 when you're done? I get successful builds but the Coral is not showing up as far as I can tell. Just looking for someone to confirm OS-Lite is working for them. ty!
Yes, I see /dev/apex_0 One thing to check is that you can see the device before you do anything else (i.e. on a fresh install of the OS. At first I wasn't seeing it and assumed that was normal if the driver had not yet been installed. However, after disconnecting and reconnecting the hat (whilst reformatting the SD card for about the 10th time) I happened to notice it was there before the script had been run. Then after updating the headers first, it all worked.
Note this is where I got the idea for updating the headers: https://gist.github.com/Reddimus/c6948d08a4f4b54ee9d075270bd79c3b
However use dataslayermedia's script as it is been modified since reddimus branched it.
I've been having trouble getting this working. I get nothing in lspci:
00:00.0 PCI bridge: Broadcom Inc. and subsidiaries Device 2712 (rev 21)
01:00.0 Ethernet controller: Device 1de4:0001
Firstly, I think the I think the msi-parent value has changed again, so just to clarify, I want msi-parent to match the phandle, right?
pcie@110000 {
compatible = "brcm,bcm2712-pcie";
reg = <0x10 0x110000 0x00 0x9310>;
device_type = "pci";
max-link-speed = <0x02>;
#address-cells = <0x03>;
#interrupt-cells = <0x01>;
#size-cells = <0x02>;
interrupt-parent = <0x01>;
interrupts = <0x00 0xdf 0x04 0x00 0xe0 0x04>;
interrupt-names = "pcie\0msi";
interrupt-map-mask = <0x00 0x00 0x00 0x07>;
interrupt-map = <0x00 0x00 0x00 0x01 0x01 0x00 0xdb 0x04 0x00 0x00 0x00 0x02 0x01 0x00 0xdc 0x04 0x00 0x00 0x00 0x03 0x01 0x00 0xdd 0x04 0x00 0x00 0x00 0x04 0x01 0x00 0xde 0x04>;
resets = <0x2c 0x07 0x2c 0x2b 0x2d>;
reset-names = "swinit\0bridge\0rescal";
msi-controller;
msi-parent = <0x6d>;
ranges = <0x2000000 0x00 0x00 0x1b 0x00 0x00 0xfffffffc 0x43000000 0x04 0x00 0x18 0x00 0x03 0x00>;
dma-ranges = <0x3000000 0x10 0x00 0x00 0x00 0x10 0x00>;
brcm,enable-l1ss;
status = "disabled";
phandle = <0x6d>;
};
But it's still not working - in dmesg I get nothing about apex, but I get this:
brcm-pcie 1000110000.pcie: link down
Just to clarify - should lspci show the device before starting? And if it doesn't show it's likely a hardware fault or physical installation issue - is that correct?
I've been having trouble getting this working. I get nothing in lspci:
00:00.0 PCI bridge: Broadcom Inc. and subsidiaries Device 2712 (rev 21) 01:00.0 Ethernet controller: Device 1de4:0001Firstly, I think the I think the msi-parent value has changed again, so just to clarify, I want msi-parent to match the phandle, right?
pcie@110000 { compatible = "brcm,bcm2712-pcie"; reg = <0x10 0x110000 0x00 0x9310>; device_type = "pci"; max-link-speed = <0x02>; #address-cells = <0x03>; #interrupt-cells = <0x01>; #size-cells = <0x02>; interrupt-parent = <0x01>; interrupts = <0x00 0xdf 0x04 0x00 0xe0 0x04>; interrupt-names = "pcie\0msi"; interrupt-map-mask = <0x00 0x00 0x00 0x07>; interrupt-map = <0x00 0x00 0x00 0x01 0x01 0x00 0xdb 0x04 0x00 0x00 0x00 0x02 0x01 0x00 0xdc 0x04 0x00 0x00 0x00 0x03 0x01 0x00 0xdd 0x04 0x00 0x00 0x00 0x04 0x01 0x00 0xde 0x04>; resets = <0x2c 0x07 0x2c 0x2b 0x2d>; reset-names = "swinit\0bridge\0rescal"; msi-controller; msi-parent = <0x6d>; ranges = <0x2000000 0x00 0x00 0x1b 0x00 0x00 0xfffffffc 0x43000000 0x04 0x00 0x18 0x00 0x03 0x00>; dma-ranges = <0x3000000 0x10 0x00 0x00 0x00 0x10 0x00>; brcm,enable-l1ss; status = "disabled"; phandle = <0x6d>; };But it's still not working - in dmesg I get nothing about apex, but I get this:
brcm-pcie 1000110000.pcie: link downJust to clarify - should lspci show the device before starting? And if it doesn't show it's likely a hardware fault or physical installation issue - is that correct?
Hello, what version of the kernel are you using?
uname -ra ?
I'm using 6.6.29-v8-16k+ #1760 SMP PREEMPT Mon Apr 29 14:44:20 BST 2024 aarch64 GNU/Linux and PCIe devices also stopped working
I can confirm that with the current kernel - 6.6.28+rpt-rpi-v8
[ 1.473214] brcm-pcie 1000110000.pcie: host bridge /axi/pcie@110000 ranges:
[ 1.480216] brcm-pcie 1000110000.pcie: No bus range found for /axi/pcie@110000, using [bus 00-ff]
[ 1.489758] brcm-pcie 1000110000.pcie: MEM 0x1b00000000..0x1bfffffffb -> 0x0000000000
[ 1.498061] brcm-pcie 1000110000.pcie: MEM 0x1800000000..0x1affffffff -> 0x0400000000
[ 1.506361] brcm-pcie 1000110000.pcie: IB MEM 0x0000000000..0x0fffffffff -> 0x1000000000
[ 1.515832] brcm-pcie 1000110000.pcie: Forcing gen 3
[ 1.520819] brcm-pcie 1000110000.pcie: Unable to find MSI PCI address
[ 1.527284] brcm-pcie: probe of 1000110000.pcie failed with error -22
[ 1.533802] brcm-pcie 1000120000.pcie: host bridge /axi/pcie@120000 ranges:
[ 1.540792] brcm-pcie 1000120000.pcie: No bus range found for /axi/pcie@120000, using [bus 00-ff]
[ 1.549879] brcm-pcie 1000120000.pcie: MEM 0x1f00000000..0x1ffffffffb -> 0x0000000000
[ 1.558176] brcm-pcie 1000120000.pcie: MEM 0x1c00000000..0x1effffffff -> 0x0400000000
[ 1.566477] brcm-pcie 1000120000.pcie: IB MEM 0x1f00000000..0x1f003fffff -> 0x0000000000
[ 1.574775] brcm-pcie 1000120000.pcie: IB MEM 0x0000000000..0x0fffffffff -> 0x1000000000
[ 1.584227] brcm-pcie 1000120000.pcie: Forcing gen 2
[ 1.589405] brcm-pcie 1000120000.pcie: PCI host bridge to bus 0000:00
Update #1: wget https://github.com/raspberrypi/firmware/raw/master/boot/bcm2712-rpi-5-b.dtb and moving the file mv bcm2712-rpi-5-b.dtb /boot/firmware/bcm2712-rpi-5-b.dtb then rebooting did the trick - /dev/apex_0 is an actual device now.
@lukaszraczylo @ezaul I think this is because the msi-parent now needs to be 0x6d in the latest updates
@scaredyfish Thank you so much! Updated the msi-parent, now getting:
script side: RuntimeError: Error in device opening (/dev/apex_0)!
dmesg: apex 0000:01:00.0: Couldn't reinit interrupts: -28
ls -lA /dev/apex_0
crw-rw-rw- 1 root apex 120, 0 May 6 22:30 /dev/apex_0
Still battling :)
Hi,
Every time I change msi-parent to
msi-parent = <0x66> or msi-parent = <0x67>
I lost my TPU card.
This is result when I use DTB default
$ uname -a
Linux smarthome 6.6.28+rpt-rpi-v8 #1 SMP PREEMPT Debian 1:6.6.28-1+rpt1 (2024-04-22) aarch64 GNU/Linux
List PCI devices
$ lspci
0000:00:00.0 PCI bridge: Broadcom Inc. and subsidiaries Device 2712 (rev 21)
0000:01:00.0 PCI bridge: ASMedia Technology Inc. ASM1182e 2-Port PCIe x1 Gen2 Packet Switch
0000:02:03.0 PCI bridge: ASMedia Technology Inc. ASM1182e 2-Port PCIe x1 Gen2 Packet Switch
0000:02:07.0 PCI bridge: ASMedia Technology Inc. ASM1182e 2-Port PCIe x1 Gen2 Packet Switch
0000:03:00.0 System peripheral: Global Unichip Corp. Coral Edge TPU
0000:04:00.0 Non-Volatile memory controller: Phison Electronics Corporation E8 PCIe3 NVMe Controller (rev 01)
0001:00:00.0 PCI bridge: Broadcom Inc. and subsidiaries Device 2712 (rev 21)
0001:01:00.0 Ethernet controller: Device 1de4:0001
Detect module
$ lspci -nn | grep 089a
0000:03:00.0 System peripheral [0880]: Global Unichip Corp. Coral Edge TPU [1ac1:089a]
$ ls /dev/apex_0
/dev/apex_0
The dmesg log
dmesg | grep brcm-pcie
[ 0.405470] brcm-pcie 1000110000.pcie: host bridge /axi/pcie@110000 ranges:
[ 0.405482] brcm-pcie 1000110000.pcie: No bus range found for /axi/pcie@110000, using [bus 00-ff]
[ 0.405493] brcm-pcie 1000110000.pcie: MEM 0x1b00000000..0x1bfffffffb -> 0x0000000000
[ 0.405500] brcm-pcie 1000110000.pcie: MEM 0x1800000000..0x1affffffff -> 0x0400000000
[ 0.405506] brcm-pcie 1000110000.pcie: IB MEM 0x0000000000..0x0fffffffff -> 0x1000000000
[ 0.407017] brcm-pcie 1000110000.pcie: Forcing gen 3
[ 0.407253] brcm-pcie 1000110000.pcie: PCI host bridge to bus 0000:00
[ 0.516181] brcm-pcie 1000110000.pcie: link up, 5.0 GT/s PCIe x1 (!SSC)
[ 0.546021] brcm-pcie 1000120000.pcie: host bridge /axi/pcie@120000 ranges:
[ 0.546029] brcm-pcie 1000120000.pcie: No bus range found for /axi/pcie@120000, using [bus 00-ff]
[ 0.546040] brcm-pcie 1000120000.pcie: MEM 0x1f00000000..0x1ffffffffb -> 0x0000000000
[ 0.546046] brcm-pcie 1000120000.pcie: MEM 0x1c00000000..0x1effffffff -> 0x0400000000
[ 0.546054] brcm-pcie 1000120000.pcie: IB MEM 0x1f00000000..0x1f003fffff -> 0x0000000000
[ 0.546059] brcm-pcie 1000120000.pcie: IB MEM 0x0000000000..0x0fffffffff -> 0x1000000000
[ 0.547144] brcm-pcie 1000120000.pcie: Forcing gen 2
[ 0.547196] brcm-pcie 1000120000.pcie: PCI host bridge to bus 0001:00
[ 0.656178] brcm-pcie 1000120000.pcie: link up, 5.0 GT/s PCIe x4 (!SSC)
I want to ask where the numbers for msi-parent like 0x66, 0x67 come from?
How do I know which number is correct?
@EnziinSystem I'm no expert, but in trying to diagnose my own problem, here's what I learned:
The device tree is compiled from the source files here - the dtsi files are included in the dts. The relevant portion is in bcm2712.dtsi
https://github.com/raspberrypi/linux/blob/rpi-6.6.y/arch/arm/boot/dts/broadcom/bcm2712-rpi-5-b.dts
https://github.com/raspberrypi/linux/blob/rpi-6.6.y/arch/arm/boot/dts/broadcom/bcm2712-rpi.dtsi
https://github.com/raspberrypi/linux/blob/rpi-6.6.y/arch/arm/boot/dts/broadcom/bcm2712.dtsi
The original source uses names, and when this is compiled, the names are replaced with phandles. These phandles are not fixed - any time Raspberry Pi makes an upstream change to the device tree, the phandle may change.
pcie1: pcie@110000 {
...
msi-parent = <&mip1>;
...
}
mip1: msi-controller@131000 {
...
}
becomes
pcie@110000 {
...
msi-parent = <0x2f>;
...
phandle = <0x6d>;
};
msi-controller@131000 {
...
phandle = <0x2f>;
}
We want to change msi-parent to pcie1 - i.e. its msi-parent is itself - in this case, 0x6d (I don't know what this means, or why it works, but someone more expert than me came up with it: https://forums.raspberrypi.com/viewtopic.php?p=2157674#p2157674)
You can now use dtoverlay=pineboards-hat-ai instead of having to change the msi-parent. It requires the most current kernel raspberrypi/linux@269a721
You can now use dtoverlay=pineboards-hat-ai instead of having to change the msi-parent. It requires the most current kernel raspberrypi/linux@269a721
My kernel:
$ uname -a
Linux smarthome 6.6.28+rpt-rpi-v8 #1 SMP PREEMPT Debian 1:6.6.28-1+rpt1 (2024-04-22) aarch64 GNU/Linux
Can I use dtoverlay=pineboards-hat-ai ?
Thanks.
you may need to use rpi-update to get the latest version https://www.raspberrypi.com/documentation/computers/os.html#rpi-update
It needs to be at 6.6.30 or higher
just tried dtoverlay=pineboards-hat-ai on Linux raspberrypi 6.6.30-v8+ #1761 SMP PREEMPT Thu May 2 16:54:52 BST 2024 aarch64 GNU/Linux
no, this doesn't work, I still have
ls -l /dev/apex_0
ls: cannot access '/dev/apex_0': No such file or directory
You need the gasket-dkms, not sure you have it installed. You can compile it following this info:
https://github.com/google/gasket-driver
or use prebuilt debs.
https://github.com/feranick/gasket-driver/releases/download/1.0-18.2/gasket-dkms_1.0-18.2_all.deb
It needs to be at 6.6.30 or higher
The config dtoverlay=pineboards-hat-ai doesn't work.
Here are the steps:
$ rpi-eeprom-update -a
$ sudo rpi-update
$ uname -a
Linux smarthome 6.6.30-v8+ #1761 SMP PREEMPT Thu May 2 16:54:52 BST 2024 aarch64 GNU/Linux
Configure
sudo nano /boot/firmware/config.txt
Add lines
[all]
# Enable the PCIe External connector.
dtparam=pciex1
dtparam=pciex1_gen=3
kernel=kernel8.img
# Enable Pineboard
dtoverlay=pineboards-hat-ai
Turn off PCI ASPM
sudo nano /boot/firmware/cmdline.txt
Add **pcie_aspm=off** before **rootwait**
sudo reboot
Install the PCIe driver and Edge TPU runtime
sudo apt update
echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" | sudo tee /etc/apt/sources.list.d/coral-edgetpu.list
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
sudo apt-get update
sudo apt-get install cmake
sudo apt-get install -y libedgetpu1-std
sudo apt install -y devscripts debhelper
sudo apt install -y dkms
sudo apt-get install -y dh-dkms
# Clone the Gasket driver repository
sudo git clone https://github.com/google/gasket-driver.git
cd gasket-driver
sudo debuild -us -uc -tc -b
# Go back to the parent directory
cd ..
# Install the built Gasket driver package
sudo dpkg -i gasket-dkms_1.0-18_all.deb
Selecting previously unselected package gasket-dkms.
(Reading database ... 175909 files and directories currently installed.)
Preparing to unpack gasket-dkms_1.0-18_all.deb ...
Unpacking gasket-dkms (1.0-18) ...
Setting up gasket-dkms (1.0-18) ...
Loading new gasket-1.0 DKMS files...
Deprecated feature: REMAKE_INITRD (/usr/src/gasket-1.0/dkms.conf)
Building for 6.6.30-v8+
Module build for kernel 6.6.30-v8+ was skipped since the
kernel headers for this kernel do not seem to be installed.
Note: Module build for kernel 6.6.30-v8+ was skipped since the kernel headers for this kernel do not seem to be installed
Set udev rule
sudo sh -c "echo 'SUBSYSTEM==\"apex\", MODE=\"0660\", GROUP=\"apex\"' >> /etc/udev/rules.d/65-apex.rules"
sudo groupadd apex
sudo adduser $USER apex
sudo reboot now
Result
$ lspci
0000:00:00.0 PCI bridge: Broadcom Inc. and subsidiaries Device 2712 (rev 21)
0000:01:00.0 PCI bridge: ASMedia Technology Inc. ASM1182e 2-Port PCIe x1 Gen2 Packet Switch
0000:02:03.0 PCI bridge: ASMedia Technology Inc. ASM1182e 2-Port PCIe x1 Gen2 Packet Switch
0000:02:07.0 PCI bridge: ASMedia Technology Inc. ASM1182e 2-Port PCIe x1 Gen2 Packet Switch
0000:03:00.0 System peripheral: Global Unichip Corp. Coral Edge TPU
0000:04:00.0 Non-Volatile memory controller: Phison Electronics Corporation E8 PCIe3 NVMe Controller (rev 01)
0001:00:00.0 PCI bridge: Broadcom Inc. and subsidiaries Device 2712 (rev 21)
0001:01:00.0 Ethernet controller: Device 1de4:0001
List TPU
$ lspci -nn | grep 089a
0000:03:00.0 System peripheral [0880]: Global Unichip Corp. Coral Edge TPU [1ac1:089a]
$ ls /dev/apex_0
ls: cannot access '/dev/apex_0': No such file or directory
$ sudo modprobe gasket
modprobe: FATAL: Module gasket not found in directory /lib/modules/6.6.30-v8+
$ sudo modprobe apex
modprobe: FATAL: Module apex not found in directory /lib/modules/6.6.30-v8+
Note: I did not make changes to MSI-parent and did not recompile the DTB
@EnziinSystem I concur and I followed those exact steps.
I have also tried installing the releases of gasket from Freneck's repo both of which do not seem to be aligned with the 6.6.30 headers.
@askpatrickw @EnziinSystem I'll get the tutorial for all AI boards uploaded to our docs website today
@askpatrickw @EnziinSystem I'll get the tutorial for all AI boards uploaded to our docs website today
When you're done, please give us the link here.
Thanks.
@EnziinSystem https://pineboards.io/blogs/tutorials/how-to-configure-the-google-coral-edge-tpu-on-the-raspberry-pi-5
This covers the DMKS drives as well as updating the kernel with rpi-update and installing the headers with rpi-source (since 6.6.30 is not live via apt yet.
sudo wget https://raw.githubusercontent.com/jgartrel/rpi-
--2024-05-09 12:24:03-- https://raw.githubusercontent.com/jgartrel/rpi-
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.
HTTP request sent, awaiting response... 400 Bad Request
2024-05-09 12:24:03 ERROR 400: Bad Request.
Check the URL:
mgapinski@coraltest:~ $ sudo wget https://raw.githubusercontent.com/jgartrel/rpi-source/master/rpi-source -O /usr/bin/rpi-source
--2024-05-09 11:52:17-- https://raw.githubusercontent.com/jgartrel/rpi-source/master/rpi-source
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.110.133, 185.199.109.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 16757 (16K) [text/plain]
Saving to: ‘/usr/bin/rpi-source’
/usr/bin/rpi-source 100%[=============================================================================================================================================>] 16.36K --.-KB/s in 0.004s
2024-05-09 11:52:17 (4.46 MB/s) - ‘/usr/bin/rpi-source’ saved [16757/16757]
mgapinski@coraltest:~ $ sudo chmod +x /usr/bin/rpi-source
mgapinski@coraltest:~ $ rpi-source --tag-update
*** SoC: BCM2712
*** Arch: 64-bit
*** Page Size: 4096
*** Set update tag: c1c78350ad097dff3289b07587120b84c3329878
mgapinski@coraltest:~ $ rpi-source --default-config
*** SoC: BCM2712
*** Arch: 64-bit
*** Page Size: 4096
*** rpi-update: https://github.com/Hexxeh/rpi-firmware
*** Firmware revision: 6bb99655a963e73937b981bbdf530da1a98e2b51
*** Linux source commit: ee713e438df8e09e01e4f7598382d3703db5d4c4
thank you so much @mikegapinski ! finally, it works for me
sudo lspci -v
0000:01:00.0 System peripheral: Global Unichip Corp. Coral Edge TPU (prog-if ff)
Subsystem: Global Unichip Corp. Coral Edge TPU
Flags: bus master, fast devsel, latency 0, IRQ 39
Memory at 1800100000 (64-bit, prefetchable) [size=16K]
Memory at 1800000000 (64-bit, prefetchable) [size=1M]
Capabilities: [80] Express Endpoint, MSI 00
Capabilities: [d0] MSI-X: Enable+ Count=128 Masked-
Capabilities: [e0] MSI: Enable- Count=1/32 Maskable- 64bit+
Capabilities: [f8] Power Management version 3
Capabilities: [100] Vendor Specific Information: ID=1556 Rev=1 Len=008 <?>
Capabilities: [108] Latency Tolerance Reporting
Capabilities: [110] L1 PM Substates
Capabilities: [200] Advanced Error Reporting
Kernel driver in use: apex
Kernel modules: apex
ls /dev/apex_*
/dev/apex_0
@mikegapinski, I'd post a comment on your post, but no way to do that.
The kernel update steps result in the following errors, for me anyway.
My Pi
cat /proc/cpuinfo | grep -i model
Model : Raspberry Pi 5 Model B Rev 1.0Make Fails
make libncurses5-dev
make: *** No rule to make target 'libncurses5-dev'. Stop.rpi-source fails
rpi-source --tag-update
ERROR:
Unexpected processor 4 (use --processor argument)I opened a bug on this last one as the docs are not helpful.
RPi-Distro/rpi-source#32
That is why I linked a fork of RPI-Distro in my tutorial sudo wget https://raw.githubusercontent.com/jgartrel/rpi-source/master/rpi-source -O /usr/bin/rpi-source
The command rpi-source --tag-update blocks it from being overwritten with the official one
I've been fiddling and managed to get this working on the Linux pi5 6.6.20+rpt-rpi-v8 kernel. Instead of using the value <0x66> I used <0x67> which matches the phandle value (which was previsouly <0x66> and now it works. So to be clear for the 6.6 kernal the change is:
msi-parent=<0x2c> should be changed to msi-parent=<0x67>