Linux Desktop on Apple Silicon in Practice

README.en.md

Linux Desktop on Apple Silicon in Practice

I bought M1 MacBook Air. It is the fastest computer I have, and I have been a GNOME/GNU/Linux user for long time. It is obvious conclusion that I need practical Linux desktop environment on Apple Silicon.

Fortunately, Linux already works on Apple Silicon/M1. But how practical is it?

• Two native ports exist.
  • Corellium. It is obsolete. https://corellium.com/blog/linux-m1
  • Asahi Linux. It is greatly improving and may be usable for certain workloads, but it lacks e.g. graphics acceleration for now. https://asahilinux.org
• QEMU can run code on CPU natively. But what about GPU? Unfortunately, QEMU is also not optimized so much for macOS.

As I needed Linux desktop right now, I decided to hack QEMU. The most difficult challenge is obviously accelerated graphics, but there is Virgil 3D; a bridge to expose host OpenGL to the guest. https://virgil3d.github.io

It unfortunately didn't work on macOS host. So I just made it work. That's it. Here is a video demonstrating OpenGL on Linux on Apple Silicon/M1:

https://www.youtube.com/watch?v=k0bVlVQU2JQ&list=PLesZxBYUPr3wdU3sONUv4Q7UDOg1Dn_Ue&index=4

Modifications

QEMU

ui/cocoa

• Added OpenGL support.
• Enforced pixel by pixel display.
• Added cursor composition.
• Improved key mappings (e.g. Japanese IME keys, 2021-06-17)

hw/block

• File locking on macOS is fixed. (2021-07-07, Add file.locking=on to drive to prevent drive breakage in case you concurrently launch the same virtual machine by mistake.)

coreaudio

• Fix device change (2022-02-26)

Virgil 3D renderer

Improved OpenGL ES support.

Do It Yourself

@knazarov's Homebre Formulae

It is independently maintained so may be a bit older, but you may still find it useful.

https://github.com/knazarov/homebrew-qemu-virgl

Setup

1. Open a terminal.

2. Install GLib, Meson, Pixman, pkg-config and spice-protocol with Homebrew.

brew install glib meson pixman pkg-config spice-protocol

3. Make a empty directory and change the working directory to it.

4.

curl -L https://gist.github.com/akihikodaki/87df4149e7ca87f18dc56807ec5a1bc5/raw/9589b879cce1cc90ea412da0682eab10390355c8/run.sh | bash -

5.

bin/qemu-img create var/virtio.raw 64G

It doesn't consume the physical space until it has data, so you can make the image very large. However, you will see odd behavior if you try to write data more than the physical disk allows.

6.

curl -LO https://download.fedoraproject.org/pub/alt/releases/39/respins/Silverblue/aarch64/Fedora-Silverblue-ostree-aarch64-39-1.5-respin.iso

8.

./run -cdrom Fedora-Silverblue-ostree-aarch64-39-1.5-respin.iso

Proceed the installation process, and now you can run Fedora by executing ./run.

Note: you won't get keyboard to work before Linux boots because TianoCore, the UEFI firmware does not support virtio-keyboard used in this configuration.

Updating

Just download the latest run.sh and execute it in your workspace directory.

Choosing OpenGL profile

Edit run.

• gl=off will disable Virgil 3D GPU. Most stable but laggy.
• gl=core will enable OpenGL.framework. Unstable.
• gl=es will enable ANGLE. Stable and fast.

Upstreaming

Upstreaming is in progress. Hopefully the features I implemented will work just by running brew install qemu in the future.

• Epoxy: anholt/libepoxy#239
• QEMU: https://patchew.org/search?q=project%3AQEMU+from%3Aakihiko.odaki%40gmail.com
• Virgil 3D renderer: https://gitlab.freedesktop.org/virgl/virglrenderer/-/merge_requests?scope=all&utf8=✓&state=all&author_username=akihiko.odaki

Some insights

QEMU

This QEMU modification is not secure. The graphics acceleration code lives in the same process with everything else of the virtual machine and it is huge; the graphics stack involves LLVM for shader compilation, and a simple bug in the stack can lead to complete takeover of the guest.

vhost-user-gpu provides graphics acceleration isolation, but it needs modifications to run outside Linux because:

• historically, vhost-user is a re-implementation of Linux kernel's vhost interface, and it relies on kernel headers for interface definitions and
• vhost-user uses eventfd which is only available on Linux.

It shouldn't be difficult, but I'm satisfied even without process isolation so I don't. The graphics acceleration process would be still shared and it would remain possible that one graphical process exploit leads to disclosure of the entire graphics output anyway.

Linux desktop on Apple Silicon/M1 in general

As I described here, such a virtualization software is practical and efficient approach to run Linux desktop. The performance overhead is also acceptable for daily use, and it even provides better integration of Linux and macOS. For example, you can switch macOS and Linux with three-finger gesture on trackpad. You can use VirtFS.

However, there are complexities that such a virtualization adds. It basically means sharing one hardware with two systems, so you have to allocate the resource properly or it ends up with bad user experience. The allocation problem happens everywhere (HID like keyboard, computing resource like CPU, power management, etc.). This approach is efficient but not the best.

In long term, native Linux port is the best option. Asahi Linux is promising and you may find it favorable than my modified QEMU for your use case even today.

README.ja.md

Apple Silicon で実践Linuxデスクトップ

M1 MacBook Airを買いました. これは私が持ってる一番速いコンピュータで, 私は長年の GNOME/GNU/Linux ユーザーでもあります. ここから, Apple Silicon でLinux デスクトップ環境が必要であるという明らかな結論が導かれます.

幸いにも, Linux は Apple Silicon で既に動作します. しかし 実用性 はどうでしょうか?

• 2つのネイティブの移植があります.
  • Corellium. これは古くなっています. https://corellium.com/blog/linux-m1
  • Asahi Linux. 大幅に改善されており, 場合によっては実用的と言えます. しかし, 今のところグラフィックスアクセラレーションなどが欠けています. https://asahilinux.org
• QEMU は CPU上でコードをネイティブ動作させることが可能です. しかし GPU についてはどうでしょう? また, 残念ながら, QEMU は macOS 向けにあまり最適化されていません.

私は Linux デスクトップが直ちに必要だったので, QEMU をハックすることにしました. 最大の困難は当然グラフィックスアクセラレーションですが, このために用いることができる, Virgil 3D という, ホストの OpenGL をゲストに見せるブリッジが存在します. https://virgil3d.github.io

残念ながらこれは macOS ホスト上で動きませんでした. そういうわけで 動くようにしました. 以上! 以下は Apple Silicon/M1 上での OpenGL を実演する動画です.

https://www.youtube.com/watch?v=k0bVlVQU2JQ&list=PLesZxBYUPr3wdU3sONUv4Q7UDOg1Dn_Ue&index=4

改変

QEMU

ui/cocoa

• OpenGL サポートを追加.
• Pixel by pixel 表示を実装.
• カーソルの合成を追加.
• キーマッピングを改善 (日本語 IME キーなど, 2021-06-17)

hw/block

• macOS でのファイルロッキングを修正. (2021-07-07, 間違って同じ仮想マシンを 並行に起動してしまった場合にディスクが壊れるのを防ぐためには drive に file.locking=on を追加してください.)

hvf

• 最新の Linux のために AArch64 ID レジスタを修正 (@agraf, 2022-02-07)

coreaudio

• デバイスの変更処理を修正 (2022-02-26)

Virgil 3D renderer

OpenGL ES サポートを改善

Do It Yourself

@knazarov の Homebrew formulae

独自に保守されているため少し古いかもしれませんが, 便利かもしれません.

https://github.com/knazarov/homebrew-qemu-virgl

セットアップ

1. ターミナルを開く.

2. GLib, Meson, Pixman, pkg-config, そして spice-protocol を Homebrew でインストールする.

brew install glib meson pixman pkg-config spice-protocol

3. 空のディレクトリを作って working directory をそれに変更する.

4.

curl -L https://gist.github.com/akihikodaki/87df4149e7ca87f18dc56807ec5a1bc5/raw/9589b879cce1cc90ea412da0682eab10390355c8/run.sh | bash -

5.

bin/qemu-img create var/virtio.raw 64G

データが記録されるまで物理領域を消費しないため, イメージをかなり大きくできます. ただし, 物理ディスクが許容する以上のデータを書き込むとおかしな挙動が現れます.

6.

curl -LO https://download.fedoraproject.org/pub/alt/releases/39/respins/Silverblue/aarch64/Fedora-Silverblue-ostree-aarch64-39-1.5-respin.iso

8.

./run -cdrom Fedora-Silverblue-ostree-aarch64-39-1.5-respin.iso

インストールプロセスを進めてください. ./run を実行することで Fedora を起動できるようになるはずです.

注: Linux が起動するまでキーボードは利用できません。これは、この構成で利用している virtio-keyboard に UEFI ファームウェアの TianoCore が対応していないためです。

更新

単に最新の run.sh をダウンロードしてワークスペースディレクトリで実行して ください.

OpenGL プロファイルを選択する

run を編集してください.

• gl=off は Virgil 3D GPU を無効にします. 安定していますがラグいです.
• gl=core は OpenGL.framework を有効にします. 不安定です.
• gl=es は ANGLE を有効にします. 安定していて速いです.

アップストリーミング

アップストリーミングが進行中です. 願わくば将来は私が実装した機能が brew install qemu とするだけで動作するようになるはずです.

• Epoxy: anholt/libepoxy#239
• QEMU: https://patchew.org/search?q=project%3AQEMU+from%3Aakihiko.odaki%40gmail.com
• Virgil 3D renderer: https://gitlab.freedesktop.org/virgl/virglrenderer/-/merge_requests?scope=all&utf8=✓&state=all&author_username=akihiko.odaki

考察

QEMU

この QEMU の改変は セキュアではありません . グラフィックスアクセラレーションのコードが仮想マシンの他のあらゆるもの全てと同じプロセスに存在していて, そのコードは巨大です. グラフィックススタックはシェーダコンパイルのために LLVM を含んでおり, その中の簡単な不具合1つでゲストの完全な乗っ取りが可能です.

vhost-user-gpu はグラフィックスアクセラレーションの分離を提供しますが, Linux 以外で動作するように修正が必要です.

• 歴史的には, vhost-user は Linux カーネルの vhost インターフェイスの再実装となっていて, インターフェイス定義のためにカーネルヘッダに依存しています.
• vhost-user は Linux でしか利用できない eventfd を利用しています.

これは難しくないでしょうが, プロセス分離がなくても満足してるのでやりません. やってもグラフィックスアクセラレーションプロセスは共有されたままになるでしょうし, 単独のグラフィカルプロセスの攻撃がグラフィックス出力全体の暴露につながるのは変わりないでしょう.

Apple Silicon/M1 上の Linux デスクトップ一般について

先に説明したとおり, このような仮想化ソフトウェアは Linux デスクトップを実行する 実用的で効率的な方法です. 性能上のオーバーヘッドも日常的な利用には許容できる範囲で, Linux と macOS のよりよい統合を提供しさえします. 例えば, macOS と Linux をトラックパッド上の3本指ジェスチャーで切り替えられます. VirtFS も使えます.

しかし, 仮想化による複雑さもあります. 要は1つのハードウェアを2つのシステムで 共有することになるので, 資源を適切に割り当てなければユーザー体験を悪化させます. この問題はあらゆる場面でおきます (キーボードのような HID, CPU のような計算資源, 電力管理などなど). この方法は 効率的 ですが 最善 ではありません.

長期的には ネイティブの Linux 移植が最善でしょう. Asahi Linux は期待できます. 今でも利用法によっては私が改変した QEMU よりよいかもしれません.

run.sh

set -eux
mkdir -p depot_tools build/qemu source/angle source/libepoxy source/virglrenderer source/qemu var
git -C depot_tools init
git -C depot_tools fetch https://chromium.googlesource.com/chromium/tools/depot_tools 5400d9ef5a9e1b22fc846eca81d5a27df02d838a
git -C depot_tools checkout FETCH_HEAD
git -C source/angle init
git -C source/angle fetch https://chromium.googlesource.com/angle/angle f9bad5e27d61e2ab6a7504b1793be5aa14eb1414
git -C source/angle checkout FETCH_HEAD
git -C source/libepoxy init
git -C source/libepoxy fetch https://github.com/akihikodaki/libepoxy.git macos
git -C source/libepoxy checkout FETCH_HEAD
git -C source/virglrenderer init
git -C source/virglrenderer fetch https://github.com/akihikodaki/virglrenderer.git macos
git -C source/virglrenderer checkout FETCH_HEAD
git -C source/qemu init
git -C source/qemu fetch https://github.com/akihikodaki/qemu.git macos
git -C source/qemu checkout FETCH_HEAD
export DEPOT_TOOLS_UPDATE=0
export PATH="$PWD/depot_tools:$PATH"
cd source/angle
scripts/bootstrap.py
gclient sync -D
gn gen --args=is_debug=false ../../build/angle
cd ../..
ninja -C build/angle
[ -e build/libepoxy/meson-info ] || meson setup "-Dc_args=-I$PWD/source/angle/include" -Degl=yes -Dx11=false "--prefix=$PWD" build/libepoxy source/libepoxy
meson install -C build/libepoxy
[ -e build/virglrenderer/meson-info ] || meson setup "-Dc_args=-I$PWD/source/angle/include" "--pkg-config-path=$PWD/lib/pkgconfig" "--prefix=$PWD" build/virglrenderer source/virglrenderer
meson install -C build/virglrenderer
cd build/qemu
PKG_CONFIG_PATH="$PWD/../../lib/pkgconfig" ../../source/qemu/configure "--extra-cflags=-I$PWD/../../source/angle/include" "--extra-ldflags=-L$PWD/../angle" "--prefix=$PWD/../.."
meson install
[ -e ../../var/edk2-arm-vars.fd ] || cp pc-bios/edk2-arm-vars.fd ../../var
cd ../..
cat > run <<'EOF'
#!/bin/bash
d="$(dirname "${BASH_SOURCE[0]}")"
exec sudo DYLD_FALLBACK_LIBRARY_PATH="$d/build/angle:$d/lib" "$d/bin/qemu-system-aarch64" -machine virt,accel=hvf -cpu host -smp "$(getconf _NPROCESSORS_ONLN)" -m 4G -device pcie-root-port,id=pcie -device virtio-sound-pci,addr=0x0.0x0,bus=pcie,multifunction=on,audiodev=audio,streams=1 -device virtio-gpu-gl-pci,addr=0x0.0x1,bus=pcie -device virtio-keyboard-pci,addr=0x0.0x2,bus=pcie -device virtio-net-pci,addr=0x0.0x3,bus=pcie,netdev=net -device virtio-rng-pci,addr=0x0.0x4,bus=pcie -display cocoa,gl=es -drive "if=pflash,format=raw,file=$d/share/qemu/edk2-aarch64-code.fd,readonly=on" -drive "if=pflash,format=raw,file=$d/var/edk2-arm-vars.fd" -drive "id=virtio,if=none,format=raw,file=$d/var/virtio.raw,discard=on" -device virtio-blk-pci,addr=0x0.0x5,backend_defaults=on,bus=pcie,drive=virtio -audiodev coreaudio,id=audio,out.fixed-settings=false -netdev vmnet-shared,id=net -chardev qemu-vdagent,id=spice,name=vdagent,clipboard=on -device virtio-serial-pci,addr=0x0.0x6,bus=pcie -device virtserialport,chardev=spice,name=com.redhat.spice.0 -full-screen -runas "$(id -u):$(id -g)" "$@"
chmod a+x run
EOF

Ok, I see. I've been looking over the venus branch from the fork, and it seems I can't just apply the patch, as a lot of things just won't exist on MacOS (I will be grabbing pieces from it though).

In other news, I was able to fix the slowdown in Firefox (it wasn't due to QEMU, but due to a memory leak in some other program I had running. Not sure why Chrome was able to resist it though).

Assuming that I integrate it correctly, to test it in a VM, would all I need is vulkan-virtio (Mesa's virtio Vulkan loader, or something else)?

So it turns out that part of the reason why nothing was happening was that I didn't apply the patches for the GL part of virglrenderer to talk to the VK part (res-sharing). However, that patch relies on properties that no longer exist in upstream virglrenderer (something about vkr_resource_attachment.memories). Luckily, the original author is still interested in the project, so maybe they'll get around to fixing it.

Ok, so I've tried my best to fix the remaining issues. However, when I try to run vulkaninfo, nothing appears as an output, and no errors show up in the console.

Since the patches didn't work, I restarted and trying to understand what is needed. What I don't understand is why isn't the guest detecting anything (if there's an error, that can point me to something I can fix). Like, why does virgl_cmd_resource_create_blob need to be implemented? It's only mentioned again in virtio-gpu.c which doesn't explain much.

A Google guy came up with patches to introduce Rutabaga, but it's currently hacky and uses CPU memory to pass screen from Rutabaga to QEMU:
https://patchew.org/QEMU/20230421011223.718-1-gurchetansingh@chromium.org/

I've looked at this (at least an earlier version), the main problem is that it uses rutabaga_gfx_ffi, it also uses libgbm. I ported over libgbm for virglrenderer because they didn't use that many functions. The ffi does though (also it's written in Rust, so there may be some unsupported parts.

Ok, so I fixed the hanging issue (forgot to implement CONTEXT_INIT in virgl_cmd_context_create). However, when vkCreateInstance is called on the host, QEMU crashes (this is on the guest side).

I have found that implementing Vulkan on MacOS is impossible for the time being. One necessary extension, VK_EXT_image_drm_format_modifier, is only for Linux, and so can't/won't be implemented by MoltenVK.

I've tried to run on macOS 13.2.1. Build is ok. But my VM does not receives keyboard inputs during the image boot (efi) and eventually gets to "Display output is not active". Once installer starts (fortunately both Fedura and Ubuntu starts after a certain time automatically), it starts to receive keyboard input. Can't say this is a critical problem but you probably should mention this in the docs.

Good suggestion. I added a note.

Thanks!

Would it be possible to do the opposite (running a macos guest w/ gpu acceleration on a linux host), using gfxstream? As of now it seems like the current way is to pass through a full AMD GPU, which I have not been successful at.

I'm not sure --- gfxstream was designed with Linux guests in mind (specifically, Android).

…

On Sun, Jul 16, 2023, 12:05 AM Theo Paris ***@***.***> wrote: ***@***.**** commented on this gist. ------------------------------ Would it be possible to do the opposite (running a macos guest w/ gpu acceleration on a linux host), using gfxstream? — Reply to this email directly, view it on GitHub <https://gist.github.com/akihikodaki/87df4149e7ca87f18dc56807ec5a1bc5#gistcomment-4630399> or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALXWUYAM44L6HT3PV5KA42DXQNSALBFKMF2HI4TJMJ2XIZLTSKBKK5TBNR2WLJDHNFZXJJDOMFWWLK3UNBZGKYLEL52HS4DFQKSXMYLMOVS2I5DSOVS2I3TBNVS3W5DIOJSWCZC7OBQXE5DJMNUXAYLOORPWCY3UNF3GS5DZVRZXKYTKMVRXIX3UPFYGLK2HNFZXIQ3PNVWWK3TUUZ2G64DJMNZZDAVEOR4XAZNEM5UXG5FFOZQWY5LFVEYTANZXGA3TSNJUU52HE2LHM5SXFJTDOJSWC5DF> . You are receiving this email because you commented on the thread. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub> .

No, macOS requires Metal. gfxstream is for OpenGL and Vulkan.

Right, duh. I didn't even think about that.

…

On Mon, Jul 17, 2023, 1:35 AM Akihiko Odaki ***@***.***> wrote: ***@***.**** commented on this gist. ------------------------------ No, macOS requires Metal. gfxstream is for OpenGL and Vulkan. — Reply to this email directly, view it on GitHub <https://gist.github.com/akihikodaki/87df4149e7ca87f18dc56807ec5a1bc5#gistcomment-4631129> or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALXWUYD5VPNPTXMHZZRCHODXQTFK7BFKMF2HI4TJMJ2XIZLTSKBKK5TBNR2WLJDHNFZXJJDOMFWWLK3UNBZGKYLEL52HS4DFQKSXMYLMOVS2I5DSOVS2I3TBNVS3W5DIOJSWCZC7OBQXE5DJMNUXAYLOORPWCY3UNF3GS5DZVRZXKYTKMVRXIX3UPFYGLK2HNFZXIQ3PNVWWK3TUUZ2G64DJMNZZDAVEOR4XAZNEM5UXG5FFOZQWY5LFVEYTANZXGA3TSNJUU52HE2LHM5SXFJTDOJSWC5DF> . You are receiving this email because you commented on the thread. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub> .

vhost-user-gpu provides graphics acceleration isolation, but it needs modifications to run outside Linux because:

• historically, vhost-user is a re-implementation of Linux kernel's vhost interface, and it relies on kernel headers for interface definitions and
• vhost-user uses eventfd which is only available on Linux.

Hi, I just came across this patch series that claims to enable vhost-user on non-Linux platforms, and it got merged. Can we leverage this feature?

Oh, that's nice to hear. However, vhost-user-gpu also needs some mechanism to share textures on GPU, and that can be done in a secure and reliable manner only with an undocumented API of macOS (fileport). WebKit (and other browsers) does use it. It's unfortunate Apple doesn't clearly document it.

What documentation do you need. Searching online, there are two functions exposed --- fileport_makeport and fileport_makefd.

…

On Fri, Jul 21, 2023, 2:23 AM Akihiko Odaki ***@***.***> wrote: ***@***.**** commented on this gist. ------------------------------ Oh, that's nice to hear. However, vhost-user-gpu also needs some mechanism to share textures on GPU, and that can be done in a secure and reliable manner only with an undocumented API of macOS (*fileport*). WebKit (and other browsers) does use it. It's unfortunate Apple doesn't clearly document it. — Reply to this email directly, view it on GitHub <https://gist.github.com/akihikodaki/87df4149e7ca87f18dc56807ec5a1bc5#gistcomment-4636216> or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALXWUYDFSRVUYKHSQSXHRVTXRIN5BBFKMF2HI4TJMJ2XIZLTSKBKK5TBNR2WLJDHNFZXJJDOMFWWLK3UNBZGKYLEL52HS4DFQKSXMYLMOVS2I5DSOVS2I3TBNVS3W5DIOJSWCZC7OBQXE5DJMNUXAYLOORPWCY3UNF3GS5DZVRZXKYTKMVRXIX3UPFYGLK2HNFZXIQ3PNVWWK3TUUZ2G64DJMNZZDAVEOR4XAZNEM5UXG5FFOZQWY5LFVEYTANZXGA3TSNJUU52HE2LHM5SXFJTDOJSWC5DF> . You are receiving this email because you commented on the thread. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub> .

I need documentation not to know how to use it, but to know Apple intends to maintain it as a stable interface (though I'm quite sure that they won't change it since it's used by all browsers and Electron applications).

I checked qemu/meson.build, found

have_vhost_user = get_option('vhost_user') \
  .disable_auto_if(targetos != 'linux') \
  .require(targetos != 'windows',
           error_message: 'vhost-user is not available on Windows').allowed()

vhost-user can be explicitly enabled on macOS. Not bad, until...

have_vhost_user_gpu = have_tools and targetos == 'linux' and pixman.found()

Oppps! vhost-user-gpu available only on Linux!

No, macOS requires Metal. gfxstream is for OpenGL and Vulkan.

for old macos, OpenGL is supported, I'am stick on macos 10.13.6 for best gpu and cuda

It's been quite a while, but I made an update today. This project is kind of in maintenance mode; I do no longer use this myself these days since Asahi works quite well. I only maintain patches already present and no longer add new ones. That said, the latest update has some interesting changes:

• Changed to use virtio-sound-pci, which was added last year. Having Intel HDA in an Arm VM was a bit... awkward after all. virtio-sound-pci seems to have less glitches for me, but it may be just a placebo effect.
• Reimplemented cursor composition with Core Animation, which makes it independent of rendering mechanism (Core Graphics or OpenGL), and allows upstreaming it before Virgl support.
• Changed to use PCI Express for virtio devices.
• Bunch of bug fixes that were done in course of upstreaming.

Some changes such as the full-screen mode improvement were also upstreamed and will be included in QEMU 9.0. I hope more changes to be upstreamed this year.

Awesome. I myself use UTM (I no longer have enough space to rebuild ANGLE), so I hope they upstream these changes. There was some memory leak that affects both this and UTM --- did you ever experience it; if so, have you been able to fix it?

Longer-term, I've been working on getting Vulkan to work on MacOS hosts (over TCP). I got to the point where Vulkan applications work, but they are slow --- turns out, synchronizing memory over TCP is slow. Is there any advice to speed things up, either using some QEMU-specific feature, or some faster internet interface on QEMU (right now, I get ~100MB/s between guest and host)?

@DUOLabs333 I haven't observed a memory leak. I don't actively use it after all.

ivshmem should be the fastest, but using would not be easier to port Venus:
https://www.qemu.org/docs/master/system/devices/ivshmem.html

I saw libkrun implemented Venus for macOS, but it is probably only for a headless environment.
https://sinrega.org/2024-03-06-enabling-containers-gpu-macos/

Doesn't ivshmem only exist for Linux hosts, not MacOS?

Maybe so.

I've been having some network speed issues with vmnet recently --- I've only been able to get around ~850 Mbps with TCP and ~2.0 Gbps with UDP (these were both tested with iperf3). Is this expected, or is there some parameter(s) I can tweak? I was expecting it to be much faster, as it should be as fast as copying memory, right?

I tried building this again but ran into some errors:

1. I couldn't get scripts/bootstrap.py to run with python3, only with python2.
2. When configuring libepoxy, I get

Trying to compare values of different types (MachineInfo, str) using ==.
This was deprecated and undefined behavior previously and is as of 0.60.0 a hard error.

I'm assuming you wanted host_system, not host_machine.

I was able to build your QEMU fork. I also changed my QEMU script to match run.sh. However, I found a few issues ---

1. audio no longer works (It turns out that Arch Linux does not build the virtio_snd module. However, reverting to "-device intel-hda" causes the guest to no longer boot, though the QEMU process does start
   (
   EDIT: I got around this by replacing the sound-pci with

-device ich9-intel-hda,bus=pcie,addr=0x0.0x0,multifunction=on
-device hda-micro,audiodev=audio

).

2. The mouse is no longer automatically captured/released.
   (
   EDIT: I got past this issue by removing a virtio-mouse-pci line in my script. However, while the mouse now does get successfully released/grabbed, once I mouse over the QEMU window, the corresponding cursor in the VM no longer moves.
   ).