Skip to content

Instantly share code, notes, and snippets.

Last active November 24, 2023 05:32
Star You must be signed in to star a gist
Save Liryna/10710751 to your computer and use it in GitHub Desktop.
Emulating ARM on Debian/Ubuntu

You might want to read this to get an introduction to armel vs armhf.

If the below is too much, you can try Ubuntu-ARMv7-Qemu but note it contains non-free blobs.

Running ARM programs under linux (without starting QEMU VM!)

First, cross-compile user programs with GCC-ARM toolchain. Then install qemu-arm-static so that you can run ARM executables directly on linux

# armel packages also exist
sudo apt-get install gcc-arm-linux-gnueabihf libc6-dev-armhf-cross qemu-arm-static

Then compile your programs in amd64 directly:

cat > hello.c << EOF

#include <stdio.h>
int main(void) { return printf("Hello ARM!\n"); }

arm-linux-gnueabihf-gcc -static  -ohello hello.c

file hello
hello: ELF 32-bit LSB executable, ARM, version 1 (SYSV), statically linked,

Hello ARM!

If you want a dynamically-linked executable, you've to pass the linker path too:

arm-linux-gnueabihf-gcc -ohello hello.c
qemu-arm -L /usr/arm-linux-gnueabihf/ ./hello   # or qemu-arm-static

Debugging using GDB

Install QEMU

sudo apt-get install qemu

Create a hard disk

Create a hard disk for your virtual machine with required capacity.

qemu-img create -f raw armdisk.img 8G

You can then install Debian using an ISO CD or directly from vmlinuz

Netboot from vmlinuz

First, you should decide what CPU and machine type you want to emulate.

You can get a list of all supported CPUs (to be passed with -cpu option, see later below):

qemu-system-arm -cpu help

You can get a list of all supported machines (to be passed with -M option, see later below):

qemu-system-arm -machine help

In this example, I chose the cortex-a9 CPU and vexpress-a9 machine. This is an ARMv7 CPU which Debian calls as armhf (ARM hard float). You must download vmlinuz and initrd files for, say Wheezy armhf netboot. Cortex-A8, A9, A15 are all ARMv7 CPUs.

You can emulate ARMv6 which Debian calls as armel by downloading the corresponding files for Wheezy armel netboot. Note that you need armel for ARMv5, v6. Raspberry Pi uses ARMv6. In this case, the cpu is arm1176 and machine is versatilepb.

Create a virtual machine with 1024 MB RAM and a Cortex-A9 CPU. Note that we must -sd instead of -sda because vexpress kernel doesn't support PCI SCSI hard disks. You'll install Debian on on MMC/SD card, that's all it means.

qemu-system-arm -m 1024M -sd armdisk.img \
                -M vexpress-a9 -cpu cortex-a9 \
                -kernel vmlinuz-3.2.0-4-vexpress -initrd initrd.gz \
                -append "root=/dev/ram"  -no-reboot

Specifying -cpu is optional. It defaults to -cpu=any. However, -M is mandatory.

This will start a new QEMU window and the Debian installer will kick-in. Just proceed with the installation (takes maybe 3 hours or so). Make sure you install "ssh-server" in tasksel screen.

NOTE: For creating ARMv6, just pass versatilepb:

qemu-system-arm -m 1024M -M versatilepb \
                -kernel vmlinuz-3.2.0-4-versatile -initrd initrd.gz \
                -append "root=/dev/ram" -hda armdisk.img -no-reboot

Netboot from ISO

Download netboot ISO for armhf or armel as needed.

WAIT! Apparently, these Debian CD images are not bootable! But Ubuntu's ARM CD image works [2].

First boot from newly installed system

You need to copy vmlinuz from the installed disk image and pass it again to qemu-system-img [Qemu wiki] (").

For armel

sudo modprobe nbd max_part=16
sudo qemu-nbd -c /dev/nbd0 armel.img
mkdir ~/qemu-mounted
sudo mount /dev/nbd0p1 ~/qemu-mounted
mkdir after-copy

cp ~/qemu-mounted/boot/* after-copy/

sudo umount ~/qemu-mounted
sudo qemu-nbd -d /dev/nbd0
sudo killall qemu-nbd

Then pass the copied kernel and initrd to qemu-system-img. Also note that we are now booting from /dev/sda1 because that is where Linux was installed

qemu-system-arm -M versatilepb -m 1024M  \
                -kernel after-copy/vmlinuz-3.2.0-4-versatile \
                -initrd after-copy/initrd.img-3.2.0-4-versatile \
                -hda armel.img -append "root=/dev/sda1" 

And there you go, play with ARM to your heart's extent!

For armhf

Extract & copy the boot files exactly as before (but for armhf.img) and pass while invoking:

qemu-system-arm -m 1024M -M vexpress-a9  \
                -kernel armhf-extracted/vmlinuz-3.2.0-4-vexpress \
                -initrd armhf-extracted/initrd.img-3.2.0-4-vexpress \
                -append "root=/dev/mmcblk0p1" -sd armhf.img

Once again, note the device (mmcblk0p1) and partition (armhf.img) reflect SD-card usage.

Connecting to the SSH server

Login to the guest OS and create a private/public key pair: ssh-keygen -t rsa.

On the host, just redirect some random port from the host to guest's port 22 (or whichever port the SSH server is running on, see /etc/ssh/sshd_config)

qemu-system-arm ....  -redir tcp:5555::22 &

Then you can connect to SSH just like ssh -p 5555 localhost.

Chroot Voodoo your ARM VM (architectural chroot with QEMU)

After the install of your ARM, you will probably see that it is really slow. To speed up your arm, you can chroot it natively and let qemu-user-static interpret the ARM instruction. [5]

sudo apt-get install qemu-user-static kpartx

We mount the image using loopback

sudo kpartx -a -v armdisk.img
sudo mkdir /mnt/arm-vm
sudo mount /dev/mapper/loop0p2 /mnt/arm-vm

Copy the static binary

sudo cp /usr/bin/qemu-arm-static /mnt/arm-vm/usr/bin
sudo mount -o bind /proc /mnt/arm-vm/proc
sudo mount -o bind /dev /mnt/temp/dev
sudo mount -o bind /sys /mnt/temp/sys

We register qemu-arm-static as ARM interpreter to the kernel linux. [6]

#This can only be run as root (sudo don't work)
sudo su
echo ':arm:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-arm-static:' > /proc/sys/fs/binfmt_misc/register

Now we chroot to our VM.

sudo chroot /mnt/arm-vm

Let see if it work:

$ uname -a
Linux cis-linux-arm 2.6.32 #19-Ubuntu SMP Wed Oct 9 16:20:46 UTC 2013 armv7l GNU/Linux

N.B: After test, qemu 1.1 (Debian wheezy) had some strange behaviour but the 1.5 (Ubuntu saucy) was working perfectly !

When you finished your work you should unmount everything to avert bad behaviour. Do not forget to not start your VM with Qemu before unmount everything !

sudo umount /mnt/arm-vm/proc
sudo umount /mnt/arm-vm/dev
sudo umount /mnt/arm-vm/sys
sudo umount /mnt/arm-vm
sudo kpartx -d -v armdisk.img

[1] [2] [3] Differences between ARM926, ARM1136, A8 and A9 [4] [5] [6]

Copy link

foudfou commented Oct 2, 2016

As of qemu 2.7 at least, -redir tcp:5555::22 is deprecated: use -net user,hostfwd=tcp::5555-:22 instead. Thx for this great guide!

Copy link

Hello @Liryna !

I'm looking at emulating CentOS 7 and Debian 8 on aarch64 using a local build of qemu 2.7.0. I'm looking at an non EFI environment, which necessitates that I specify the kernel and its arguments.

Any ideas on how I could do that ?

Thank You !

Copy link

eblot commented Nov 16, 2016

There is an alternative way to store the installed kernel (vmlinuz*) and initrd to the host
Once the installation is complete, do not (yet) select "Finish installation" but "Start a shell", and:

chroot /target
cd boot/
scp * host:

where host is the IP of the host then resume with "Finish installation"

No need to use qemu-nbd, as long as a scp client is installed in the guest and sshd runs on the host.

Copy link

eblot commented Nov 16, 2016

I'm unable to boot a Jessie image, even with QEMU's -dtb vexpress-v2p-ca9.dtb option, with netboot-console images.

Any success with Jessie & an ARMv7 target?

Copy link

Finally, some qemu emulation steps that work. Thanks @Lirnya.

Copy link

jaytho commented Sep 14, 2017

I think you mean
apt-get install qemu-user-static
instead of
apt-get install qemu-arm-static.

first link is dead: is
what you mean?

Copy link

@jaytho thank you!

Copy link

After having installed Debian for armhf, to run qemu-system-arm, the partition to load root in the append command is actually partition 2 (p2) when using the suggested partitions (since partition 1 is /boot, from which we extract the kernel and the initial ram disk and partition 3 is /swap)
=> Use -append "root=/dev/mmcblk0p2" instead.
I guess the same thing happens with armel (/dev/sda2 instead of /dev/sda1)

Copy link

I am Running attifyos on virtualbox on which I emulated arm architecture using qemu . I have ssh the machine as well. But the ping is working the emulated machine.
Can u tell the probable reasons?

Copy link

rypz79 commented Dec 12, 2017

Hi @Liryna,

The QEMU Debian installer is not so straight forward , there are some selections to be made there.
Anyhow it doesn't finish the installation , I get the error message screen "Unable to install the selected kernel".
When I try again using "Install base system option" ( it always warns me to clean the target, how do I clean the target ?) I get the same results.
How to address these error messages ?

Thank you


Copy link

fcicq commented Feb 5, 2018

echo "xxx" | sudo tee (-a if append) can be used instead of sudo su -c echo.

Copy link

where I can get this file

Copy link

Here is an easier way to start an arm VM

$ sudo pip3 install arm_now
$ arm_now start armv5-eabi
Welcome to arm_now
buildroot login: root
# uname -m

Copy link

likan999 commented Jul 2, 2018

I tried to NetBoot and install Ubuntu 18.04 for arm on QEMU and I also saw the “continue without boot loader” and there is no initrd file on my installed boot partition. Anyone knows why and how to fix that? Thanks.

Copy link

ag88 commented Jul 7, 2019

Copy link

wtdcode commented Jul 28, 2021

Finish installation by offline installer in half an hour: .

Copy link

beroal commented Oct 4, 2021

Copy link

jaysonpowers-zz commented Oct 31, 2021

For wsl Ubuntu 20.04, if you have problems with installing qemu-arm-static see:
it says to use qemu-user-static for their purposes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment