Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Create Virtual Machines using QEMU on Silicon based Apple Macs

Install QEMU on Silicon based Apple Macs (June 2021)

Option 1 - Automatically

zsh -c "$(curl -fsSL https://raw.githubusercontent.com/nrjdalal/silicon-virtualizer/master/install-qemu.sh)"

Option 2 - Manually

  • Install Xcode command line tools

xcode-select --install
  • Install Homebrew arm64

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
echo 'eval "$(/opt/homebrew/bin/brew shellenv)"' >> /Users/$(logname)/.zprofile
eval "$(/opt/homebrew/bin/brew shellenv)"
  • Install necessary packages for building

brew install libffi gettext glib pkg-config autoconf automake pixman ninja
  • Clone qemu

git clone https://github.com/qemu/qemu
  • Change directory to qemu repository

cd qemu
  • Checkout to commit dated June 03, 2021 v6.0.0

git checkout 3c93dfa42c394fdd55684f2fbf24cf2f39b97d47
  • Apply patch series v8 by Alexander Graf

curl https://patchwork.kernel.org/series/485309/mbox/ | git am
  • Building qemu installer

mkdir build && cd build
../configure --target-list=aarch64-softmmu
make -j8
  • Install qemu

sudo make install

Create Ubuntu Server using QEMU on Silicon based Apple Macs

First run, close manually after installation aka first reboot.

curl https://cdimage.ubuntu.com/releases/20.04/release/ubuntu-20.04.2-live-server-arm64.iso -o ubuntu-lts.iso

qemu-img create -f qcow2 virtual-disk.qcow2 8G

cp $(dirname $(which qemu-img))/../share/qemu/edk2-aarch64-code.fd .

dd if=/dev/zero conv=sync bs=1m count=64 of=ovmf_vars.fd

qemu-system-aarch64 \
  -machine virt,accel=hvf,highmem=off \
  -cpu cortex-a72 -smp 4 -m 4G \
  -device virtio-gpu-pci \
  -device virtio-keyboard-pci \
  -drive "format=raw,file=edk2-aarch64-code.fd,if=pflash,readonly=on" \
  -drive "format=raw,file=ovmf_vars.fd,if=pflash" \
  -drive "format=qcow2,file=virtual-disk.qcow2" \
  -cdrom ubuntu-lts.iso

Second run, enjoy your Ubuntu Server.

qemu-system-aarch64 \
  -machine virt,accel=hvf,highmem=off \
  -cpu cortex-a72 -smp 4 -m 4G \
  -device virtio-gpu-pci \
  -device virtio-keyboard-pci \
  -drive "format=raw,file=edk2-aarch64-code.fd,if=pflash,readonly=on" \
  -drive "format=raw,file=ovmf_vars.fd,if=pflash" \
  -drive "format=qcow2,file=virtual-disk.qcow2"

Bonus, connect via SSH and serve port 80 at localhost.

  1. Start server using -
qemu-system-aarch64 \
  -machine virt,accel=hvf,highmem=off \
  -cpu cortex-a72 -smp 4 -m 4G \
  -device virtio-gpu-pci \
  -device virtio-keyboard-pci \
  -drive "format=raw,file=edk2-aarch64-code.fd,if=pflash,readonly=on" \
  -drive "format=raw,file=ovmf_vars.fd,if=pflash" \
  -drive "format=qcow2,file=virtual-disk.qcow2" \
  -nic hostfwd=tcp:127.0.0.1:9922-0.0.0.0:22,hostfwd=tcp:127.0.0.1:9980-0.0.0.0:80 &
  1. Login via SSH using -
ssh <username>@127.0.0.1 -p 9922
  1. Visit http content at -
open http://127.0.0.1:9980
@corwin-of-amber

This comment has been minimized.

Copy link

@corwin-of-amber corwin-of-amber commented Jun 6, 2021

These things change so quickly! I was lucky to come across this just one hour after posting. Following the instructions, this has gotten me the closest so far to a working QEMU (except for the previous time when I built 5.2.0).

Now I get this error in one of the patched files:

../target/arm/hvf/hvf.c:67:7: error: use of undeclared identifier 'HV_REG_X0'
    { HV_REG_X0,   offsetof(CPUARMState, xregs[0]) },

and so forth up to HV_REG_X18. Different SDKs perhaps? I am still on Big Sur 11.3, will try later with 11.4.

@nrjdalal

This comment has been minimized.

Copy link
Owner Author

@nrjdalal nrjdalal commented Jun 6, 2021

@corwin-of-amber I've tried with Big Sur 11.4 myself, do update it. You can run the auto installer just for fun to see if everything is up to date or not.

zsh -c "$(curl -fsSL https://raw.githubusercontent.com/nrjdalal/silicon-virtualizer/master/install-qemu.sh)"

Do let me know, if everything goes alright.

@nrjdalal

This comment has been minimized.

Copy link
Owner Author

@nrjdalal nrjdalal commented Jun 6, 2021

I've also added a bonus script to create a Ubuntu Server once you successfully make and install QEMU.

@ovidiy1

This comment has been minimized.

Copy link

@ovidiy1 ovidiy1 commented Jun 6, 2021

I followed the post, all works except mouse pointer - mouse itself works, but no cursor visible, any clue?
"-show-cursor" is not supported anymore

@nrjdalal

This comment has been minimized.

Copy link
Owner Author

@nrjdalal nrjdalal commented Jun 6, 2021

@ovidiy1 try -display default,show-cursor=on

@ovidiy1

This comment has been minimized.

Copy link

@ovidiy1 ovidiy1 commented Jun 6, 2021

@nrjdalal thanks, works now!

@mateuszdrab

This comment has been minimized.

Copy link

@mateuszdrab mateuszdrab commented Jun 7, 2021

Finally been able to get it to build, but when I try to build with emulation support using

../configure --target-list=aarch64-softmmu,x86_64-softmmu,i386-softmmu --enable-cocoa

it just won't compile...

I was using the following article before: https://arstechnica.com/civis/viewtopic.php?f=19&t=1473419

/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/Hypervisor.framework/Headers/hv_vcpu_types.h:37:18: note: 'hv_vcpu_t' declared here
typedef uint64_t hv_vcpu_t;
^
In file included from ../target/i386/hvf/x86hvf.c:24:
../target/i386/hvf/vmx.h:50:43: error: unknown type name 'hv_x86_reg_t'; did you mean 'hv_sys_reg_t'?
static inline void wreg(hv_vcpuid_t vcpu, hv_x86_reg_t reg, uint64_t v)
^~~~~~~~~~~~
hv_sys_reg_t
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/Hypervisor.framework/Headers/hv_vcpu_types.h:190:1: note: 'hv_sys_reg_t' declared here
OS_ENUM(hv_sys_reg, uint16_t,
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/os/base.h:178:39: note: expanded from macro 'OS_ENUM'
typedef enum : _type { VA_ARGS } _name##_t

Is there a way to get emulation working?

@nrjdalal

This comment has been minimized.

Copy link
Owner Author

@nrjdalal nrjdalal commented Jun 7, 2021

@mateuszdrab compilation with above flags works fine, you must've been using older SDK kit, upgrade to latest version of SDK kit.

Steps to upgrade -

sudo rm -rf /Library/Developer/CommandLineTools
xcode-select --install

If you've installed Xcode using App Store, you can update it from there too.

Tested on Big Sur v11.4 and xcode-select v2384.

@mateuszdrab

This comment has been minimized.

Copy link

@mateuszdrab mateuszdrab commented Jun 7, 2021

Hi @nrjdalal, I've just reinstalled the latest xcode-select and I'm getting the same exact error still :(

xcode-select version 2384

I think the version of brew I got installed is not arm64. Would that make a difference?

@nrjdalal

This comment has been minimized.

Copy link
Owner Author

@nrjdalal nrjdalal commented Jun 7, 2021

Hi @nrjdalal, I've just reinstalled the latest xcode-select and I'm getting the same exact error still :(

It seems to be working in every case, ping me on telegram, @nrjdalal itself is the username. Let's see what's going on. :)

@mateuszdrab

This comment has been minimized.

Copy link

@mateuszdrab mateuszdrab commented Jun 7, 2021

Hey buddy, thanks for your help so far. I'm afraid I don't use telegram.
Perhaps if we stay here we can potentially help out someone else who experiences the same issue.

@corwin-of-amber

This comment has been minimized.

Copy link

@corwin-of-amber corwin-of-amber commented Jun 7, 2021

After upgrading Big Sur & reinstalling the command-line tools, make seems to generate x86_64 object files... this only happened after the update (I think). Any idea what's causing it?

Running cc, gcc, or clang without any flags generate arm64, as expected. Even running the very same command that make (ninja) is running generates arm64. But inside the build arch is somehow x64.

@corwin-of-amber

This comment has been minimized.

Copy link

@corwin-of-amber corwin-of-amber commented Jun 7, 2021

I managed to get around the problem by editing build.ninja and adding arch --arm64e at the beginning of every command (cc, c++, clang). This is so weird that it was x86 by default. Now I've built it and it runs!

I tried running the Debian 10.7.0 installer, and I had to add these flags:

  -object rng-random,id=rng0,filename=/dev/urandom -device virtio-rng-pci,rng=rng0 \
  -device qemu-xhci -device usb-kbd \

Now it hangs after Booting Linux Kernel... 🤷

@mateuszdrab

This comment has been minimized.

Copy link

@mateuszdrab mateuszdrab commented Jun 8, 2021

I end up launching QEMU via UTM and I must admit that whilst x64 OS works reasonably well, x86 is just slow as hell.

@cattyhouse

This comment has been minimized.

Copy link

@cattyhouse cattyhouse commented Jun 9, 2021

this is the only guide that works! thank you very much!

@brainard52

This comment has been minimized.

Copy link

@brainard52 brainard52 commented Jun 21, 2021

I did something slightly different for my scripts. I also made some Windows scripts:

new_ubuntu.sh:

#!/bin/bash

efi_firm="$(dirname $(which qemu-img))/../share/qemu/edk2-aarch64-code.fd"

dd if=/dev/zero conv=sync bs=1m count=64 of=ubuntu_ovmf_vars.fd

qemu-img create -f qcow2 ubuntu.qcow2 512G

echo "starting"

qemu-system-aarch64 \
  -machine virt,accel=hvf,highmem=off \
  -cpu cortex-a72 -smp 4 -m 4G \
  -device qemu-xhci,id=usb-bus \
  -device usb-tablet,bus=usb-bus.0 \
  -device usb-mouse,bus=usb-bus.0 \
  -device usb-kbd,bus=usb-bus.0 \
  -device virtio-gpu-pci \
  -display default,show-cursor=on \
  -nic user,model=virtio \
  -drive format=raw,file=$efi_firm,if=pflash,readonly=on \
  -drive format=raw,file=ubuntu_ovmf_vars.fd,if=pflash \
  -device nvme,drive=drive0,serial=drive0,bootindex=0 \
  -drive if=none,media=disk,id=drive0,format=qcow2,file=ubuntu.qcow2 \
  -boot d \
  -device usb-storage,drive=drive2,removable=true,bootindex=2 \
  -drive if=none,media=cdrom,id=drive2,file=disk_images/ubuntu-21.04-live-server-arm64.iso

start_ubuntu.sh:

#!/bin/bash

efi_firm="$(dirname $(which qemu-img))/../share/qemu/edk2-aarch64-code.fd"

echo "starting"

qemu-system-aarch64 \
  -machine virt,accel=hvf,highmem=off \
  -cpu cortex-a72 -smp 4 -m 4G \
  -device qemu-xhci,id=usb-bus \
  -device usb-tablet,bus=usb-bus.0 \
  -device usb-mouse,bus=usb-bus.0 \
  -device usb-kbd,bus=usb-bus.0 \
  -device virtio-gpu-pci \
  -display default,show-cursor=on \
  -nic user,model=virtio \
  -drive format=raw,file=$efi_firm,if=pflash,readonly=on \
  -drive format=raw,file=ubuntu_ovmf_vars.fd,if=pflash \
  -device nvme,drive=drive0,serial=drive0,bootindex=0 \
  -drive if=none,media=disk,id=drive0,format=qcow2,file=ubuntu.qcow2 \

new_windows.sh:

#!/bin/bash

windows10vhdx="disk_images/Windows10_InsiderPreview_Client_ARM64_en-us_21354.VHDX windows.qcow2"
virtio-win-iso="disk_images/virtio-win-0.1.190.iso"

efi_firm="$(dirname $(which qemu-img))/../share/qemu/edk2-aarch64-code.fd"

dd if=/dev/zero conv=sync bs=1m count=64 of=windows_ovmf_vars.fd

qemu-img convert -O qcow2 "$windows10vhdx" windows.qcow2

echo "starting"

qemu-system-aarch64 \
  -machine virt,accel=hvf,highmem=off \
  -cpu cortex-a72 -smp 4 -m 4G \
  -device ramfb \
  -device qemu-xhci,id=usb-bus \
  -device usb-tablet,bus=usb-bus.0 \
  -device usb-mouse,bus=usb-bus.0 \
  -device usb-kbd,bus=usb-bus.0 \
  -nic user,model=virtio \
  -drive format=raw,file="$efi_firm",if=pflash,readonly=on \
  -drive format=raw,file=windows_ovmf_vars.fd,if=pflash \
  -drive if=none,media=disk,id=drive0,format=qcow2,file=windows.qcow2 \
  -device nvme,drive=drive0,serial=drive0,bootindex=0 \
  -device usb-storage,drive=drive2,removable=true,bootindex=2 \
  -drive if=none,media=cdrom,id=drive2,file="$virtio-win-iso"

# To enable networking after installing:
# 1. run `bcdedit.exe -set TESTSIGNING ON`
# 2. Open device manager and find the unknown ethernet device.
# 3. Choose to update and search for the driver on the computer.
# 4. Navigate to and select D:\NetKVM\w10\ARM64

You will need to download the Windows VHDX from here and the latest virtio-win iso from here

Be sure to update windows10vhdx and virtio-win-iso to have the proper paths to the VHDX and iso respectively. I created a directory within my vm directory called "disk_images" and put them there for organization's sake.

start_windows.sh:

#!/bin/bash

efi_firm="$(dirname $(which qemu-img))/../share/qemu/edk2-aarch64-code.fd"

echo "starting"

qemu-system-aarch64 \
  -machine virt,accel=hvf,highmem=off \
  -cpu cortex-a72 -smp 4 -m 4G \
  -device ramfb \
  -device qemu-xhci,id=usb-bus \
  -device usb-tablet,bus=usb-bus.0 \
  -device usb-mouse,bus=usb-bus.0 \
  -device usb-kbd,bus=usb-bus.0 \
  -nic user,model=virtio \
  -drive format=raw,file="$efi_firm",if=pflash,readonly=on \
  -drive format=raw,file=windows_ovmf_vars.fd,if=pflash \
  -drive if=none,media=disk,id=drive0,format=qcow2,file=windows.qcow2 \
  -device nvme,drive=drive0,serial=drive0,bootindex=0 \

If you want to attach a second drive to the machine, just add it by copying the last two lines and modify the drive id and boot index in each.

For example:

-drive if=none,media=disk,id=drive1,format=qcow2,file=foo.qcow2 \
-device nvme,drive=drive1,serial=drive1,bootindex=1 \

Something you could do with this is create an extra drive that can be used for multiple different machines. If you do that, however, be sure to only run one machine at any given time because I'm sure it would cause severe corruption if both were to write to the drive at the same time.

Another thing to be aware of:
Unfortunately ARM64 Windows doesn't have virtio-gpu drivers yet (as far as I can tell, and I've searched hard for them) so we're stuck with ramfb. I'm not entirely sure, but I think this means everything is rendered in software. Additionally, my machine appears to be restricted to three different resolutions: 640x480, 800x600, and 1024x768. You can set this by hitting escape on the "TianoCore" screen before the windows loading dot circle thing.

@meshonline

This comment has been minimized.

Copy link

@meshonline meshonline commented Jul 11, 2021

So strange!

I am building qemu on Mac Mini Silicon M1 according to the above instructions, it successfully compiled, but the utility only supports the 'tcg' accelerator.

./qemu-system-aarch64 -accel help
Accelerators supported in QEMU binary:
tcg

I examined the configuration output, the compiler detected that the CPU is x86_64:

  Compilation
                         host CPU: x86_64
                  host endianness: little
  Targets and accelerators
                      KVM support: NO
                      HAX support: NO
                      HVF support: NO
                     WHPX support: NO
                     NVMM support: NO
                      Xen support: NO
                      TCG support: YES
                      TCG backend: native (x86_64)
                TCG debug enabled: NO
                      target list: aarch64-softmmu
                  default devices: YES
         out of process emulation: NO

Any hints?

@meshonline

This comment has been minimized.

Copy link

@meshonline meshonline commented Jul 11, 2021

Never mind, I figured it out.

I had Anaconda3 installed on my machine, it uses Python 3.8, which can't detect the CPU correctly.

After I removed Anaconda3 and switched to Python 3.9, everything works.

@kiranchavala

This comment has been minimized.

Copy link

@kiranchavala kiranchavala commented Aug 23, 2021

Getting invalid hcf error on m1 mac even after following the

zsh -c "$(curl -fsSL https://raw.githubusercontent.com/nrjdalal/silicon-virtualizer/master/install-qemu.sh)"

~ ❯ qemu-system-aarch64 \ 10:47:49 AM
-machine virt,accel=hvf,highmem=off
-cpu cortex-a72 -smp 4 -m 4G
-device virtio-gpu-pci
-device virtio-keyboard-pci
-drive "format=raw,file=edk2-aarch64-code.fd,if=pflash,readonly=on"
-drive "format=raw,file=ovmf_vars.fd,if=pflash"
-drive "format=qcow2,file=virtual-disk.qcow2"
-cdrom ubuntu-lts.iso
qemu-system-aarch64: invalid accelerator hvf

@jerschneider

This comment has been minimized.

Copy link

@jerschneider jerschneider commented Sep 11, 2021

Wow thank you so much for this guide! :)

@cattyhouse

This comment has been minimized.

Copy link

@cattyhouse cattyhouse commented Oct 12, 2021

Qemu 6.1 is out and patchwork v12 is here : https://patchwork.kernel.org/series/548227/mbox/

is it worth the update? how do i know which commit to apply the patch on? thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment