Create Virtual Machines using QEMU on Silicon based Apple Macs

QEMU-Silicon-Mac-Virtualization.md

Install QEMU on Silicon based Apple Macs (June 2021)

Option 1 - Automatically

zsh -c "$(curl -fsSL https://raw.githubusercontent.com/nrjdalal/silicon-virtualizer/master/install-qemu.sh)"

Option 2 - Manually

• Install Xcode command line tools

xcode-select --install

• Install Homebrew arm64

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
echo 'eval "$(/opt/homebrew/bin/brew shellenv)"' >> /Users/$(logname)/.zprofile
eval "$(/opt/homebrew/bin/brew shellenv)"

• Install necessary packages for building

brew install libffi gettext glib pkg-config autoconf automake pixman ninja

• Clone qemu

git clone https://github.com/qemu/qemu

• Change directory to qemu repository

cd qemu

• Checkout to commit dated June 03, 2021 v6.0.0

git checkout 3c93dfa42c394fdd55684f2fbf24cf2f39b97d47

• Apply patch series v8 by Alexander Graf

curl https://patchwork.kernel.org/series/485309/mbox/ | git am

• Building qemu installer

mkdir build && cd build
../configure --target-list=aarch64-softmmu
make -j8

• Install qemu

sudo make install

Create Ubuntu Server using QEMU on Silicon based Apple Macs

First run, close manually after installation aka first reboot.

curl https://cdimage.ubuntu.com/releases/20.04/release/ubuntu-20.04.4-live-server-arm64.iso -o ubuntu-lts.iso

qemu-img create -f qcow2 virtual-disk.qcow2 8G

cp $(dirname $(which qemu-img))/../share/qemu/edk2-aarch64-code.fd .

dd if=/dev/zero conv=sync bs=1m count=64 of=ovmf_vars.fd

qemu-system-aarch64 \
  -machine virt,accel=hvf,highmem=off \
  -cpu cortex-a72 -smp 4 -m 4G \
  -device virtio-gpu-pci \
  -device virtio-keyboard-pci \
  -drive "format=raw,file=edk2-aarch64-code.fd,if=pflash,readonly=on" \
  -drive "format=raw,file=ovmf_vars.fd,if=pflash" \
  -drive "format=qcow2,file=virtual-disk.qcow2" \
  -cdrom ubuntu-lts.iso

Second run, enjoy your Ubuntu Server.

qemu-system-aarch64 \
  -machine virt,accel=hvf,highmem=off \
  -cpu cortex-a72 -smp 4 -m 4G \
  -device virtio-gpu-pci \
  -device virtio-keyboard-pci \
  -drive "format=raw,file=edk2-aarch64-code.fd,if=pflash,readonly=on" \
  -drive "format=raw,file=ovmf_vars.fd,if=pflash" \
  -drive "format=qcow2,file=virtual-disk.qcow2"

Bonus, connect via SSH and serve port 80 at localhost.

1. Start server using -

qemu-system-aarch64 \
  -machine virt,accel=hvf,highmem=off \
  -cpu cortex-a72 -smp 4 -m 4G \
  -device virtio-gpu-pci \
  -device virtio-keyboard-pci \
  -drive "format=raw,file=edk2-aarch64-code.fd,if=pflash,readonly=on" \
  -drive "format=raw,file=ovmf_vars.fd,if=pflash" \
  -drive "format=qcow2,file=virtual-disk.qcow2" \
  -nic hostfwd=tcp:127.0.0.1:9922-0.0.0.0:22,hostfwd=tcp:127.0.0.1:9980-0.0.0.0:80 &

2. Login via SSH using -

ssh <username>@127.0.0.1 -p 9922

3. Visit http content at -

open http://127.0.0.1:9980

So strange!

I am building qemu on Mac Mini Silicon M1 according to the above instructions, it successfully compiled, but the utility only supports the 'tcg' accelerator.

./qemu-system-aarch64 -accel help
Accelerators supported in QEMU binary:
tcg

I examined the configuration output, the compiler detected that the CPU is x86_64:

  Compilation
                         host CPU: x86_64
                  host endianness: little
  Targets and accelerators
                      KVM support: NO
                      HAX support: NO
                      HVF support: NO
                     WHPX support: NO
                     NVMM support: NO
                      Xen support: NO
                      TCG support: YES
                      TCG backend: native (x86_64)
                TCG debug enabled: NO
                      target list: aarch64-softmmu
                  default devices: YES
         out of process emulation: NO

Any hints?

Never mind, I figured it out.

I had Anaconda3 installed on my machine, it uses Python 3.8, which can't detect the CPU correctly.

After I removed Anaconda3 and switched to Python 3.9, everything works.

Getting invalid hcf error on m1 mac even after following the

zsh -c "$(curl -fsSL https://raw.githubusercontent.com/nrjdalal/silicon-virtualizer/master/install-qemu.sh)"

~ ❯ qemu-system-aarch64 \ 10:47:49 AM
-machine virt,accel=hvf,highmem=off
-cpu cortex-a72 -smp 4 -m 4G
-device virtio-gpu-pci
-device virtio-keyboard-pci
-drive "format=raw,file=edk2-aarch64-code.fd,if=pflash,readonly=on"
-drive "format=raw,file=ovmf_vars.fd,if=pflash"
-drive "format=qcow2,file=virtual-disk.qcow2"
-cdrom ubuntu-lts.iso
qemu-system-aarch64: invalid accelerator hvf

Wow thank you so much for this guide! :)

Qemu 6.1 is out and patchwork v12 is here : https://patchwork.kernel.org/series/548227/mbox/

is it worth the update? how do i know which commit to apply the patch on? thanks

@meshonline

After I removed Anaconda3 and switched to Python 3.9, everything works.

Is kvm working?

@meshonline

After I removed Anaconda3 and switched to Python 3.9, everything works.

Is kvm working?

No, but hvf works.

Not sure if my question was clear:
I mean inside of the Ubuntu guest... does kvm (nested virtualization) work there?

Unfortunately, the nested Ubuntu guest machine does not support any virtualization.

Is 4G upper limit or will it be possible to have more in the future ?

I use -M virt,highmem=off -m 6G -accel hvf -accel tcg -cpu cortex-a57 among a few other things and seems to work fine for me!

Thank you @nrjdalal - after trying multipass and utm, great to have a Linux VM running from the building blocks!

Qemu 6.1 is out and patchwork v12 is here : https://patchwork.kernel.org/series/548227/mbox/

is it worth the update? how do i know which commit to apply the patch on? thanks

@cattyhouse the updates are always worthy, feel free to update to latest, I'm not currently using virtualization, if it was for me, I'd always go with latest updates, commits you can test at different dates

running option 1 gives me:

ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

Definitely an M1 problem. I tried using lcurl flag but did not work. Any ideas? @nrjdalal

@KPhans the version installed via brew works great out of the box now. Here's a wrapper script I use, which helps get the qemu command line args right:

https://gitlab.com/CentOS/automotive/sample-images/-/blob/main/osbuild-manifests/runvm

@KPhans the version installed via brew works great out of the box now. Here's a wrapper script I use, which helps get the qemu command line args right:

https://gitlab.com/CentOS/automotive/sample-images/-/blob/main/osbuild-manifests/runvm

Thanks @ericcurtin ! How exactly do I use this script?

@KPhans the version installed via brew works great out of the box now. Here's a wrapper script I use, which helps get the qemu command line args right:
https://gitlab.com/CentOS/automotive/sample-images/-/blob/main/osbuild-manifests/runvm

Thanks @ericcurtin ! How exactly do I use this script?

Like so... Create a qcow2 file as a virtual hard disk:

qemu-img create fedora_silverblue.qcow2 512G

Run a vm with an installer .iso file to install your linux vm:

runvm --memory 4G --cdrom Fedora-Silverblue-ostree-aarch64-36-1.5.iso fedora_silverblue.qcow2

On every proceeding run:

runvm --memory 4G fedora_silverblue.qcow2

since you guys are using bash/zsh, you can always use array to have a better syntax

#!/bin/bash
#~/bin/qemu_ubuntu

options=(
# this is a comment
# enable hvf for the best performance
-machine virt,accel=hvf
# use a newer cpu, you can also use 'host' with qemu 7.0
#-cpu host
-cpu cortex-a72
# let's do 4core and 4GB RAM
-smp 4 -m 4G
# add a cd rom for the first time boot
-cdrom ubuntu-lts.iso
# let's go on, add anything you want
)
qemu-system-aarch64 "${options[@]}"

now move to your terminal and run

qemu_ubuntu

thanks for this howto, especially the magic SSH incantation appreciate it. Just got my M4 and I was able to just build the current qemu-9.2.0-rc0 source tree without any patches and it seems to work so far but I was having issue with my network working on alpine linux. Will try to verify/resolve that this morning.

@danmack hope it work's for you...