Skip to content

Instantly share code, notes, and snippets.

@cGandom
Last active July 18, 2024 16:00
Show Gist options
  • Save cGandom/23764ad5517c8ec1d7cd904b923ad863 to your computer and use it in GitHub Desktop.
Save cGandom/23764ad5517c8ec1d7cd904b923ad863 to your computer and use it in GitHub Desktop.
Emulating Raspberry Pi 4 with Qemu

Emulating Raspberry Pi 4 with Qemu

Just a quick update before we dive in: what we're actually doing here is running Raspberry Pi OS (64-bit) on a QEMU virtual ARM setup. This isn't full-blown hardware emulation of the Raspberry Pi 4, but more about creating a virtual environment for the OS. It doesn't mimic all the specific hardware features of the Pi 4, but it's pretty useful and great for general testing. I turned to this solution mainly to extract a modified sysroot from the Raspberry Pi OS, something not readily available in other resources. For those looking into detailed emulation of the actual Raspberry Pi 4's hardware in QEMU, check out this link for the latest updates: https://gitlab.com/qemu-project/qemu/-/issues/1208.

Hope it helps! :D

Shortcomings: No GUI yet, only console.

Steps

  1. Download Raspberry Pi OS (64-bit) from Raspberry Pi operating system images.
    Here we downloaded Raspberry Pi OS (64-bit) with desktop, Kernel version: 6.1, Debian version: 11 (bullseye), Release date: May 3rd 2023, named 2023-05-03-raspios-bullseye-arm64.img. We put it in /home/mydir.
  2. Install the required packages on your host system:
    $ # Cross compilers for arm64
    $ sudo apt install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu
    
    $ # Qemu itself
    $ sudo apt install qemu qemubuilder qemu-system-gui qemu-system-arm qemu-utils \
        qemu-system-data qemu-system
  3. Build the Linux kernel for qemu arm64 (You can download the kernel from Kernel.org):
    $ wget https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.1.34.tar.xz
    $ tar xvJf linux-6.1.34.tar.xz
    $ cd linux-6.1.34
    
    $ # create a .config file
    $ ARCH=arm64 CROSS_COMPILE=/bin/aarch64-linux-gnu- make defconfig
    $ # Use the kvm_guest config as the base defconfig, which is suitable for qemu
    $ ARCH=arm64 CROSS_COMPILE=/bin/aarch64-linux-gnu- make kvm_guest.config
    $ # Build the kernel
    $ ARCH=arm64 CROSS_COMPILE=/bin/aarch64-linux-gnu- make -j8
    
    $ cp arch/arm64/boot/Image /home/mydir
  4. Mount the image for enabling ssh and configuring username and password:
    1. Get the correct offset value with the help of fdisk utility:
      $ fdisk -l 2023-05-03-raspios-bullseye-arm64.img
      Disk 2023-05-03-raspios-bullseye-arm64.img: 4.11 GiB, 4412407808 bytes, 8617984 sectors
      Units: sectors of 1 * 512 = 512 bytes
      Sector size (logical/physical): 512 bytes / 512 bytes
      I/O size (minimum/optimal): 512 bytes / 512 bytes
      Disklabel type: dos
      Disk identifier: 0x3e247b30
      Device                                 Boot  Start     End Sectors  Size Id Type
      2023-05-03-raspios-bullseye-arm64.img1        8192  532479  524288  256M  c W95 FAT32 (LBA)
      2023-05-03-raspios-bullseye-arm64.img2      532480 8617983 8085504  3.9G 83 Linux
      As we can see, we have two partitions inside the downloaded image. The first device (partition) is the bootable partition, and the second one is the root filesystem. The first partition is what will be mounted as /boot in Raspberry Pi, and this is where we'll need to create some files.
      Obtain the correct offset of the first device by multiplying the start of the first partition (here 8192) by the sector size (here 512). Here it will be calculated as 8192 * 512 = 4194304
    2. Mount the image in /mnt/rpi directory:
      $ sudo mkdir /mnt/rpi
      $ sudo mount -o loop,offset=4194304 2023-05-03-raspios-bullseye-arm64.img /mnt/rpi
    3. Create a file named ssh to enable ssh:
      $ cd /mnt/rpi
      $ sudo touch ssh
    4. Additionally, create a file named userconf.txt in the same directory and put your desired username and password there, like <username>:<hashed-password> (might be better to leave the username as pi). This will be your default credentials:
      $ openssl passwd -6                                     # Generate the <hashed-password>
      $ echo 'pi:<hashed-password>' | sudo tee userconf.txt   # Put them inside `userconf.txt`
    5. Finally, unmount the image:
      $ sudo umount /mnt/rpi
  5. Run qemu emulator:
    $ cd /home/mydir
    $ qemu-system-aarch64 -machine virt -cpu cortex-a72 -smp 6 -m 4G \
        -kernel Image -append "root=/dev/vda2 rootfstype=ext4 rw panic=0 console=ttyAMA0" \
        -drive format=raw,file=2023-05-03-raspios-bullseye-arm64.img,if=none,id=hd0,cache=writeback \
        -device virtio-blk,drive=hd0,bootindex=0 \
        -netdev user,id=mynet,hostfwd=tcp::2222-:22 \
        -device virtio-net-pci,netdev=mynet \
        -monitor telnet:127.0.0.1:5555,server,nowait
    This machine will be able to access the internet.
  6. After the machine is completely booted up, you can login to it from your computer by using ssh and the username and password you specified:
    $ ssh -l pi localhost -p 2222
  7. Done!

Troubleshooting

  • If you had any problem with connecting to internet, it might be because of bad DNS configurations, and you should consider adding nameserver 8.8.8.8 to top of the file /etc/resolv.conf in the machine.
  • You can access the qemu monitor console with:
    $ telnet localhost 5555
@mmguero
Copy link

mmguero commented Jan 29, 2024

One thing I haven't been able to figure out, so maybe somebody watching this might know: I can see in the built kernel directory the .ko kernel module files that have been built, and I have even played around with trying to share this folder under /lib/modules, but I can't get modprobe foobar (whatever the kernel module is) to work to load any kernel modules in the emulated guest. Specifically I was trying to set up docker in the guest but was unable to load the overlay module.

@heqichen
Copy link

Nice post! I can even run RPI under wsl. But qemu need -nographic to avoid connecting graphic server.

@hatran3e
Copy link

hatran3e commented Apr 1, 2024

for wsl2, the command to run qemu should be:
qemu-system-aarch64 -machine virt -cpu cortex-a72 -smp 6 -m 4G -kernel Image -append "root=/dev/vda2 rootfstype=ext4 rw panic=0 console=ttyAMA0" -drive format=raw,file=2020-08-20-raspios-buster-arm64-lite.img,if=none,id=hd0,cache=writeback -device virtio-blk,drive=hd0,bootindex=0 -netdev user,id=mynet,hostfwd=tcp::2222-:22 -device virtio-net-pci,netdev=mynet -monitor telnet:127.0.0.1:5555,server,nowait -nographic
you should also install the package like @HarryCutts mentioned:
sudo apt install flex bison libssl-dev

@ZenoBogoni
Copy link

ZenoBogoni commented Apr 10, 2024

i got a error with user and passwd; pi and raspberry.

Hey, thanks for trying out the guide!

Did you make sure to put the hashed password value inside userconf.txt? You can generate the hashed password as I mentioned above, but be careful to avoid any typos:

$ openssl passwd -6  # enter your password here, for example 'raspberry'
$6$sbMZ.NX0yYjCn3JL$Wrdd9WKZfXlmpoCgS6O7JEibneWvkHlJwwtENCTssEQ6qt0pvbIciYdOUEortfyyx3WkQUVeXaHPc/ojFGxWv/

Now, use this hashed value like so:

$ echo 'pi:<hashed-password>' | sudo tee userconf.txt

# If you used `raspberry` as password
$ echo 'pi:$6$sbMZ.NX0yYjCn3JL$Wrdd9WKZfXlmpoCgS6O7JEibneWvkHlJwwtENCTssEQ6qt0pvbIciYdOUEortfyyx3WkQUVeXaHPc/ojFGxWv/' | sudo tee userconf.txt

Let me know if the issue continues, and provide any additional details if possible!

i'm sorry I am trying to do this, i set the password to raspberry using this last command you wrote here but still i cant login with

  • user: pi
  • passw: raspberry

am I missing something? I cant login neither from the console nor the ssh, in ssh it always says 'Permission denied, please try again.', in console it says 'login incorrect' thanks in advance

Solution

i mounted the root partition of the rpi like this

sudo mount -o loop,offset=272629760 2023-05-03-raspios-bullseye-arm64.img /mnt/rpi

then i modified the file /etc/shadow and replaced the hashed password manually, then i saved the changes and unmounted the partition

sudo nano etc/shadow
# find use and replace hashed password, then save and exit
cd 
sudo umount /mnt/rpi

then I started the vm and enjoyed it

@JPT77
Copy link

JPT77 commented Apr 17, 2024

Mounting a partition within an image is less error prone by using kpartx -av raspi.img and then linking the loop devices like this:
ln -s /dev/mapper/loop7p2 raspi.img2

don't forget to remove the loop devices using kpartx -dv raspi.img before deleting the image, else they will stick around till reboot.

you may increase the partition size (before creating the loop devices) by simply issuing truncate -s 4G raspi.img this will set the file size to 4 gig (sparse).

@mrdc
Copy link

mrdc commented Apr 17, 2024

I'm trying to use physical microSD from my RPi for debugging. At the moment I have an issue passing physical microSD to qemu using these additional arguments -drive file=/dev/disk13,if=none,format=raw and -append "root=/dev/mmcblk0p7 rootfstype=ext4"
disk13 is the physical microSD, /dev/mmcblk0p7 is root partition for RaspberryOS, qemu boots the kernel, but then I receive Kernel Panic, because it can't find root partition:

[    2.715128] VFS: Cannot open root device "mmcblk0p7" or unknown-block(0,0): error -6
[    2.715856] Please append a correct "root=" boot option; here are the available partitions:
[    2.716698] 0100            4096 ram0 
[    2.716746]  (driver?)
[    2.716996] 0101            4096 ram1 
[    2.717016]  (driver?)
[    2.718510] 0102            4096 ram2 
[    2.718532]  (driver?)
[    2.718744] 0103            4096 ram3 
[    2.718762]  (driver?)
[    2.719030] 0104            4096 ram4 
[    2.719048]  (driver?)
[    2.719880] 0105            4096 ram5 
[    2.719918]  (driver?)
[    2.720546] 0106            4096 ram6 
[    2.720562]  (driver?)
[    2.720752] 0107            4096 ram7 
[    2.720768]  (driver?)
[    2.720956] 0108            4096 ram8 
[    2.720972]  (driver?)
[    2.722830] 0109            4096 ram9 
[    2.722852]  (driver?)
[    2.723214] 010a            4096 ram10 
[    2.723294]  (driver?)
[    2.724030] 010b            4096 ram11 
[    2.724056]  (driver?)
[    2.724530] 010c            4096 ram12 
[    2.724550]  (driver?)
[    2.724744] 010d            4096 ram13 
[    2.724762]  (driver?)
[    2.724968] 010e            4096 ram14 
[    2.725448]  (driver?)
[    2.725838] 010f            4096 ram15 
[    2.725856]  (driver?)
[    2.726386] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)

I've tried -hda for qemu, different variants in -append "root=/dev/sda2" etc. Out of ideas what can be wrong - not a frequent qemu user for such scenarios. Qemu docs haven't helped, also for googling - everybody uses images, but not physical microSD.

@LiamFry
Copy link

LiamFry commented Jun 8, 2024

I'm following the instructions as provided and, while running Step 3 (ARCH=arm64 CROSS_COMPILE=/bin/aarch64-linux-gnu- make -j8) I get the following error:

  DTC     arch/arm64/boot/dts/qcom/sm8150-sony-xperia-kumano-griffin.dtb                                               
certs/extract-cert.c:21:10: fatal error: openssl/bio.h: No such file or directory                                      
   21 | #include <openssl/bio.h>                                                                                       
      |          ^~~~~~~~~~~~~~~                                                                                       
compilation terminated.                                    
make[2]: *** [scripts/Makefile.host:114: certs/extract-cert] Error 1                                                   
make[1]: *** [scripts/Makefile.build:503: certs] Error 2                                                               
make[1]: *** Waiting for unfinished jobs....                                                                           

This has the whole make process exist with an error:
make: *** [Makefile:2010: .] Error 2

If I re-run the command, I get a similar error:

  CC      fs/notify/dnotify/dnotify.o                      
certs/extract-cert.c:21:10: fatal error: openssl/bio.h: No such file or directory                                      
   21 | #include <openssl/bio.h>     
      |          ^~~~~~~~~~~~~~~ 
compilation terminated.                   
make[2]: *** [scripts/Makefile.host:114: certs/extract-cert] Error 1                                                   
make[1]: *** [scripts/Makefile.build:503: certs] Error 2
make[1]: *** Waiting for unfinished jobs....               

@JPT77
Copy link

JPT77 commented Jun 8, 2024

certs/extract-cert.c:21:10: fatal error: openssl/bio.h: No such file or directory

did you install - well - openssl-dev package or something like that?
Which platform are you on?
Usually each major distribution got an online package search that allows finding files in packages.

@LiamFry
Copy link

LiamFry commented Jun 10, 2024

Sorry for polluting this page with my issues. I'm fumbling in the dark here, being rather new to cross compiling and a total Qemu noob. I can only ask really silly questions.

I have OpenSSL installed for dev purposes but alas, it's for my host: x86_64; I suppose I need to download the OpenSSL source and build that for arm64 then figure out how to work it into the steps outlined at the very top of this page. A quick look at the OpenSSL build instructions is making this seem more daunting then expected.

All I want to do is build some "generic" software for the RPiZ2W, RPi3B+, and RPi4B - all aarch64/arm64 ... and by "generic" I mean I'm not making use of any special hardware or capabilities present on the Pi. I've had minimal success with straight-up cross compiling - "Hello World" is easy - but you throw 200+ source files with fancy makefiles and custom libraries at it and things fall down quickly. I thought emulating the RPi OS and migrating my build environment would be easier. It's turning out not to be.

In the past, I would have dedicated some actual hardware to be the "build machine" but instead of going out and buying yet another RPi4 just for building, I thought I'd be smarter about it and cross-compile or emulate.

@JPT77
Copy link

JPT77 commented Jun 10, 2024

Source (and dev-package) is per se platform independent. The dev package is the one that contains the header files .h and this is what you need to compile a software. You do not need the source. It asked for the .h file, remember?

again: which system do you use for building? then I can point you to which packages you need to install.

@sonicprod
Copy link

@LiamFry,

You simply need to install the libssl-dev package:

sudo apt-get install libssl-dev

@dojosgithub
Copy link

I cannot get the network to work on my M2, everything else seems perfect.

@CRT-HAO
Copy link

CRT-HAO commented Jul 13, 2024

How to run armhf version?

@emendir
Copy link

emendir commented Jul 18, 2024

Thank you so much for you work!
Based on it, I've figured out (through a lot of work!) a way to run a Raspberry Pi OS Desktop VM with a working interactive GUI!
See here, I've got a fully automated script for it:
https://gist.github.com/emendir/922d6914a1705ed2e8e4e96db726c422

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment