Hybrid Graphics on t2 macbooks

_t2MacBookProVFIO.md

Where'd the bit on enabling the iGPU go????

It's here now.

Quirks for VFIO gpu passthrough on a MacBookPro16,1:

1. The Arch wiki has a very good article on this, follow it, add what is mentioned here, and that ought to get this working for you. You can ask on the t2 Linux discord if you have issues.
2. Isolate AMDGPU from boot with vfio-pci.ids=1002:7340 in kernel parameters (add to /etc/default/grub). If you have a different gpu, check lspci -nn|grep AMD and use your gpu's id. This does make the dGPU use more power than if it wasn't isolated. You don't need to bother with the audio component here.
3. Install the vendor-reset kernel module (aur: vendor-reset-dkms-git), load it as early as possible (add to /etc/modules-load.d/).
4. USB and Thunderbolt controllers are in same iommu group as the GPU (at least for MBP 16,1). Pass through the two USB controllers, But don't pass through the Thunderbolt controllers (GPU driver fails in Windows guest if you do).
5. Thunderbolt controllers must be disabled before the VM boots, i.e. sudo virsh nodedev-detach pci_0000_08_00_0 && sudo virsh nodedev-detach pci_0000_7e_00_0. This means everything in the iommu group is passed through or inactive. You can reattach them with virsh nodedev-reattach after the VM shuts down if you'd like.
6. When USB controllers are passed through, some USB devices are still accessible by the host. It seems to be everything connected to a USB-A port on a hub that is still accesable to the host.
7. GPU hostdev element in libvirt xml must have managed='no', i.e. <hostdev mode='subsystem' type='pci' managed='no'>. This stops it from handing the GPU back to Linux and putting in a state where it can't be given back to Windows. This lets you reboot the VM.
8. If you can't install normal AMDGPU drivers in your Windows guest, use the Bootcamp support software drivers. Get them from Bootcamp Assistant in macOS, transfer the folder to Windows, and run WindowsSupport/BootCamp/Drivers/AMD/AMDGraphics/Setup.exe.
9. Intel GVT-G is working.

---

No warranty, idk what I'm doing.

iommu.txt

IOMMU Group 0:
	00:00.0 Host bridge [0600]: Intel Corporation 8th Gen Core Processor Host Bridge/DRAM Registers [8086:3ec4] (rev 07)
IOMMU Group 1:
	00:01.0 PCI bridge [0604]: Intel Corporation 6th-9th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 07)
	00:01.1 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x8) [8086:1905] (rev 07)
	00:01.2 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x4) [8086:1909] (rev 07)
	01:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Upstream Port of PCI Express Switch [1002:1478] (rev 43)
	02:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Downstream Port of PCI Express Switch [1002:1479]
	03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 14 [Radeon RX 5500/5500M / Pro 5500M] [1002:7340] (rev 43) <-- Isolate with `vfio_pci.ids=1002:xxxx` in kernel arguments, and set managed to no in xml
	03:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 HDMI Audio [1002:ab38] <-- Don't need to isolate or set managed to no in xml
	06:00.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
	07:00.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
	07:01.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
	07:02.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
	07:04.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
	08:00.0 System peripheral [0880]: Intel Corporation JHL7540 Thunderbolt 3 NHI [Titan Ridge 4C 2018] [8086:15eb] (rev 06) <-- Don't pass through
	09:00.0 USB controller [0c03]: Intel Corporation JHL7540 Thunderbolt 3 USB Controller [Titan Ridge 4C 2018] [8086:15ec] (rev 06) <-- Pass through
	7c:00.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
	7d:00.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
	7d:01.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
	7d:02.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
	7d:04.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
	7e:00.0 System peripheral [0880]: Intel Corporation JHL7540 Thunderbolt 3 NHI [Titan Ridge 4C 2018] [8086:15eb] (rev 06) <-- Don't pass through
	7f:00.0 USB controller [0c03]: Intel Corporation JHL7540 Thunderbolt 3 USB Controller [Titan Ridge 4C 2018] [8086:15ec] (rev 06) <-- Pass through
IOMMU Group 10:
	04:00.0 Mass storage controller [0180]: Apple Inc. ANS2 NVMe Controller [106b:2005] (rev 01)
	04:00.1 Non-VGA unclassified device [0000]: Apple Inc. T2 Bridge Controller [106b:1801] (rev 01)
	04:00.2 Non-VGA unclassified device [0000]: Apple Inc. T2 Secure Enclave Processor [106b:1802] (rev 01)
	04:00.3 Multimedia audio controller [0401]: Apple Inc. Apple Audio Device [106b:1803] (rev 01)
IOMMU Group 11:
	05:00.0 Network controller [0280]: Broadcom Inc. and subsidiaries BCM4364 802.11ac Wireless Network Adapter [14e4:4464] (rev 04)
IOMMU Group 2:
	00:02.0 VGA compatible controller [0300]: Intel Corporation UHD Graphics 630 (Mobile) [8086:3e9b] <-- Only appears if apple-set-os has loaded correctly
IOMMU Group 3:
	00:12.0 Signal processing controller [1180]: Intel Corporation Cannon Lake PCH Thermal Controller [8086:a379] (rev 10)
IOMMU Group 4:
	00:14.0 USB controller [0c03]: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller [8086:a36d] (rev 10)
	00:14.2 RAM memory [0500]: Intel Corporation Cannon Lake PCH Shared SRAM [8086:a36f] (rev 10)
IOMMU Group 5:
	00:16.0 Communication controller [0780]: Intel Corporation Cannon Lake PCH HECI Controller [8086:a360] (rev 10)
IOMMU Group 6:
	00:1b.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #17 [8086:a340] (rev f0)
IOMMU Group 7:
	00:1c.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #1 [8086:a338] (rev f0)
IOMMU Group 8:
	00:1e.0 Communication controller [0780]: Intel Corporation Cannon Lake PCH Serial IO UART Host Controller [8086:a328] (rev 10)
IOMMU Group 9:
	00:1f.0 ISA bridge [0601]: Intel Corporation Device [8086:a313] (rev 10)
	00:1f.4 SMBus [0c05]: Intel Corporation Cannon Lake PCH SMBus Controller [8086:a323] (rev 10)
	00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH SPI Controller [8086:a324] (rev 10)

win10-gpu.xml

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>win10-gpu</name>
  <uuid>798b0f23-b57c-4741-8b41-2f78960ae0b7</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://microsoft.com/win/10"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit='KiB'>8388608</memory>
  <currentMemory unit='KiB'>8388608</currentMemory>
  <vcpu placement='static'>8</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='2'/>
    <vcpupin vcpu='1' cpuset='3'/>
    <vcpupin vcpu='2' cpuset='4'/>
    <vcpupin vcpu='3' cpuset='5'/>
    <vcpupin vcpu='4' cpuset='8'/>
    <vcpupin vcpu='5' cpuset='9'/>
    <vcpupin vcpu='6' cpuset='10'/>
    <vcpupin vcpu='7' cpuset='11'/>
    <emulatorpin cpuset='1,7'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-q35-5.2'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/edk2-ovmf/x64/OVMF_CODE.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/win10-gpu_VARS.fd</nvram>
    <bootmenu enable='no'/>
    <smbios mode='host'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vpindex state='on'/>
      <runtime state='on'/>
      <synic state='on'/>
      <stimer state='on'>
        <direct state='on'/>
      </stimer>
      <reset state='on'/>
      <vendor_id state='on' value='Microsoft Hv'/>
      <frequencies state='on'/>
      <reenlightenment state='on'/>
      <tlbflush state='on'/>
      <ipi state='on'/>
      <evmcs state='off'/>
    </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>
    <vmport state='off'/>
    <ioapic driver='kvm'/>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' cores='4' threads='2'/>
    <feature policy='disable' name='hypervisor'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
    <timer name='hypervclock' present='yes'/>
    <timer name='tsc' present='yes' mode='native'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='yes'/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <controller type='usb' index='0' model='nec-xhci' ports='15'>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='1' port='0x8'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='2' port='0x9'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='3' port='0xa'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='4' port='0xb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='5' port='0xc'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x4'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='6' port='0xd'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x5'/>
    </controller>
    <controller type='pci' index='7' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='7' port='0xe'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x6'/>
    </controller>
    <controller type='pci' index='8' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='8' port='0xf'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x7'/>
    </controller>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='spice'>
      <listen type='none'/>
      <gl enable='yes' rendernode='/dev/dri/by-path/pci-0000:00:02.0-render'/>
    </graphics>
    <video>
      <model type='none'/>
    </video>
    <hostdev mode='subsystem' type='mdev' managed='no' model='vfio-pci' display='off'>
      <source>
        <address uuid='eb32c120-a312-43a3-92d5-b01c9d20879c'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='no'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='no'>
      <source>
        <address domain='0x0000' bus='0x03' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x09' slot='0x00' function='0x0'/>
      </source>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x7f' slot='0x00' function='0x0'/>
      </source>
      <boot order='2'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
  <qemu:commandline>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.hostdev0.x-igd-opregion=on'/>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.hostdev0.romfile=/vbios_gvt_uefi.rom'/>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.hostdev0.ramfb=on'/>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.hostdev0.driver=vfio-pci-nohotplug'/>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.hostdev0.display=on'/>
    <qemu:env name='MESA_LOADER_DRIVER_OVERRIDE' value='i965'/>
  </qemu:commandline>
</domain>

ok it worked once and now it's getting error 43 for the amdgpu, rip

Ok it was the fact that i passed through the usb-c ports?

ok if i don't passthrough thunderbolt ports it's fine; time to play video game i guess

ok so vendor reset doesn't seem to work? could be the apple specific gpu having quirks. It does work as long as linux doesn't touch the amd gpu and if you don't reboot the vm.

needed managed='no'

Thanks for documenting your progress. I'm following closely (here and on t2linux Discord), will try the same soon.

Crashlog from macos after playing a game with DRI_PRIME:

panic(cpu 1 caller 0xfffffff01671bc44): x86 CPU CATERR detected
Debugger message: panic
Memory ID: 0xff
OS release type: User
OS version: 18P4347
macOS version: Not set
Kernel version: Darwin Kernel Version 20.3.0: Tue Jan  5 18:50:06 PST 2021; root:xnu-7195.80.35~4/RELEASE_ARM64_T8010
Kernel UUID: C7A021A0-C976-37C8-B2E7-A18B3A7CD973
iBoot version: iBoot-6723.80.19
secure boot?: YES
x86 EFI Boot State: 0xd
x86 System State: 0x0
x86 Power State: 0x0
x86 Shutdown Cause: 0x3
x86 Previous Power Transitions: 0x70707060400
PCIeUp link state: 0x1614
Paniclog version: 13
Kernel slide:      0x000000000e7e4000
Kernel text base:  0xfffffff0157e8000
mach_absolute_time: 0xa1e15c25b3
Epoch Time:        sec       usec
  Boot    : 0x60783ed9 0x000433a0
  Sleep   : 0x60783ee8 0x0009362c
  Wake    : 0x6078f18c 0x00021428
  Calendar: 0x607962a3 0x0003fa1c

Panicked task 0xffffffe19b926490: 3503 pages, 223 threads: pid 0: kernel_task
Panicked thread: 0xffffffe19b9445f8, backtrace: 0xffffffe802013710, tid: 411
		  lr: 0xfffffff015f14494  fp: 0xffffffe802013750
		  lr: 0xfffffff015f142f0  fp: 0xffffffe8020137c0
		  lr: 0xfffffff01603ad7c  fp: 0xffffffe802013890
		  lr: 0xfffffff0165295fc  fp: 0xffffffe8020138a0
		  lr: 0xfffffff015f14024  fp: 0xffffffe802013c20
		  lr: 0xfffffff015f14024  fp: 0xffffffe802013c80
		  lr: 0xfffffff016f0ab34  fp: 0xffffffe802013ca0
		  lr: 0xfffffff01671bc44  fp: 0xffffffe802013cd0
		  lr: 0xfffffff01670a238  fp: 0xffffffe802013d30
		  lr: 0xfffffff01670c178  fp: 0xffffffe802013dc0
		  lr: 0xfffffff0167098e0  fp: 0xffffffe802013e50
		  lr: 0xfffffff0165f9960  fp: 0xffffffe802013e80
		  lr: 0xfffffff016485bc0  fp: 0xffffffe802013ec0
		  lr: 0xfffffff016485440  fp: 0xffffffe802013f00
		  lr: 0xfffffff0165345a0  fp: 0x0000000000000000

Crash with DRI_PRIME=1 electron https://www.porterrobinson.com, which is an interactive thing:

panic(cpu 0 caller 0xfffffff01358bc44): x86 CPU CATERR detected
Debugger message: panic
Memory ID: 0xff
OS release type: User
OS version: 18P4347
macOS version: Not set
Kernel version: Darwin Kernel Version 20.3.0: Tue Jan  5 18:50:06 PST 2021; root:xnu-7195.80.35~4/RELEASE_ARM64_T8010
Kernel UUID: C7A021A0-C976-37C8-B2E7-A18B3A7CD973
iBoot version: iBoot-6723.80.19
secure boot?: YES
x86 EFI Boot State: 0xd
x86 System State: 0x0
x86 Power State: 0x0
x86 Shutdown Cause: 0x5
x86 Previous Power Transitions: 0x405060400
PCIeUp link state: 0x89271611
Paniclog version: 13
Kernel slide:      0x000000000b654000
Kernel text base:  0xfffffff012658000
mach_absolute_time: 0x1f8f0934aab
Epoch Time:        sec       usec
  Boot    : 0x6087fca6 0x00007caa
  Sleep   : 0x608bfbe4 0x00068bcc
  Wake    : 0x608cb2f6 0x00070926
  Calendar: 0x608d0b60 0x0002b6c1

Panicked task 0xffffffe19b4df6d8: 3564 pages, 223 threads: pid 0: kernel_task
Panicked thread: 0xffffffe19b611dd8, backtrace: 0xffffffe80ca63710, tid: 403
		  lr: 0xfffffff012d84494  fp: 0xffffffe80ca63750
		  lr: 0xfffffff012d842f0  fp: 0xffffffe80ca637c0
		  lr: 0xfffffff012eaad7c  fp: 0xffffffe80ca63890
		  lr: 0xfffffff0133995fc  fp: 0xffffffe80ca638a0
		  lr: 0xfffffff012d84024  fp: 0xffffffe80ca63c20
		  lr: 0xfffffff012d84024  fp: 0xffffffe80ca63c80
		  lr: 0xfffffff013d7ab34  fp: 0xffffffe80ca63ca0
		  lr: 0xfffffff01358bc44  fp: 0xffffffe80ca63cd0
		  lr: 0xfffffff01357a238  fp: 0xffffffe80ca63d30
		  lr: 0xfffffff01357c178  fp: 0xffffffe80ca63dc0
		  lr: 0xfffffff0135798e0  fp: 0xffffffe80ca63e50
		  lr: 0xfffffff013469960  fp: 0xffffffe80ca63e80
		  lr: 0xfffffff0132f5bc0  fp: 0xffffffe80ca63ec0
		  lr: 0xfffffff0132f5440  fp: 0xffffffe80ca63f00
		  lr: 0xfffffff0133a45a0  fp: 0x0000000000000000

also this in dmesg whenever i use driprime, although i think it's a wlroots thing i915 0000:00:02.0: cannot be used for peer-to-peer DMA as the client and provider (0000:03:00.0) do not share an upstream bridge or whitelisted host bridge

Compiling kernel with docker, without gpu-power-prefs set, just apple-set-os

panic(cpu 1 caller 0xfffffff0207e4190): x86 CPU CATERR detected
Debugger message: panic
Memory ID: 0xff
OS release type: User
OS version: 18P4663
macOS version: Not set
Kernel version: Darwin Kernel Version 20.5.0: Fri May  7 22:04:14 PDT 2021; root:xnu-7195.121.3~8/RELEASE_ARM64_T8010
Kernel UUID: 873F4AC5-B66C-3016-8EAC-39D538EEB5A3
iBoot version: iBoot-6723.120.36
secure boot?: YES
x86 EFI Boot State: 0xd
x86 System State: 0x0
x86 Power State: 0x0
x86 Shutdown Cause: 0x1
x86 Previous Power Transitions: 0x70707060400
PCIeUp link state: 0x89271614
Paniclog version: 13
Kernel slide:      0x0000000018888000
Kernel text base:  0xfffffff01f88c000
mach_absolute_time: 0xf90309718
Epoch Time:        sec       usec
  Boot    : 0x60b057cc 0x000a653a
  Sleep   : 0x00000000 0x00000000
  Wake    : 0x00000000 0x00000000
  Calendar: 0x60b062aa 0x000b3fab

Panicked task 0xffffffe199b30630: 3317 pages, 227 threads: pid 0: kernel_task
Panicked thread: 0xffffffe19a023600, backtrace: 0xffffffe810af3700, tid: 420
		  lr: 0xfffffff01ffd749c  fp: 0xffffffe810af3750
		  lr: 0xfffffff01ffd72f4  fp: 0xffffffe810af37c0
		  lr: 0xfffffff020102df0  fp: 0xffffffe810af3890
		  lr: 0xfffffff0205fd5fc  fp: 0xffffffe810af38a0
		  lr: 0xfffffff01ffd7028  fp: 0xffffffe810af3c20
		  lr: 0xfffffff01ffd7028  fp: 0xffffffe810af3c80
		  lr: 0xfffffff021007200  fp: 0xffffffe810af3ca0
		  lr: 0xfffffff0207e4190  fp: 0xffffffe810af3cd0
		  lr: 0xfffffff0207d2520  fp: 0xffffffe810af3d30
		  lr: 0xfffffff0207d441c  fp: 0xffffffe810af3dc0
		  lr: 0xfffffff0207d1bc4  fp: 0xffffffe810af3e50
		  lr: 0xfffffff0206d6814  fp: 0xffffffe810af3e80
		  lr: 0xfffffff020552010  fp: 0xffffffe810af3ec0
		  lr: 0xfffffff020551890  fp: 0xffffffe810af3f00
		  lr: 0xfffffff0206085a0  fp: 0x0000000000000000

compiling wine, without apple-set-os or gpu-power-prefs. Time to make microsoft do it for me

panic(cpu 0 caller 0xfffffff01384c190): x86 CPU CATERR detected
Debugger message: panic
Memory ID: 0xff
OS release type: User
OS version: 18P4663
macOS version: Not set
Kernel version: Darwin Kernel Version 20.5.0: Fri May  7 22:04:14 PDT 2021; root:xnu-7195.121.3~8/RELEASE_ARM64_T8010
Kernel UUID: 873F4AC5-B66C-3016-8EAC-39D538EEB5A3
iBoot version: iBoot-6723.120.36
secure boot?: YES
x86 EFI Boot State: 0xd
x86 System State: 0x0
x86 Power State: 0x0
x86 Shutdown Cause: 0x5
x86 Previous Power Transitions: 0x405060400
PCIeUp link state: 0x89271614
Paniclog version: 13
Kernel slide:      0x000000000b8f0000
Kernel text base:  0xfffffff0128f4000
mach_absolute_time: 0x79e5754809
Epoch Time:        sec       usec
  Boot    : 0x60c55f79 0x0008ccb2
  Sleep   : 0x60c5a4fb 0x0009d93b
  Wake    : 0x60c5acbd 0x00073e81
  Calendar: 0x60c5d8a7 0x0002bac8

Panicked task 0xffffffe199ad8630: 3326 pages, 226 threads: pid 0: kernel_task
Panicked thread: 0xffffffe199cabc00, backtrace: 0xffffffe80fc2b700, tid: 383
		  lr: 0xfffffff01303f49c  fp: 0xffffffe80fc2b750
		  lr: 0xfffffff01303f2f4  fp: 0xffffffe80fc2b7c0
		  lr: 0xfffffff01316adf0  fp: 0xffffffe80fc2b890
		  lr: 0xfffffff0136655fc  fp: 0xffffffe80fc2b8a0
		  lr: 0xfffffff01303f028  fp: 0xffffffe80fc2bc20
		  lr: 0xfffffff01303f028  fp: 0xffffffe80fc2bc80
		  lr: 0xfffffff01406f200  fp: 0xffffffe80fc2bca0
		  lr: 0xfffffff01384c190  fp: 0xffffffe80fc2bcd0
		  lr: 0xfffffff01383a520  fp: 0xffffffe80fc2bd30
		  lr: 0xfffffff01383c41c  fp: 0xffffffe80fc2bdc0
		  lr: 0xfffffff013839bc4  fp: 0xffffffe80fc2be50
		  lr: 0xfffffff01373e814  fp: 0xffffffe80fc2be80
		  lr: 0xfffffff0135ba010  fp: 0xffffffe80fc2bec0
		  lr: 0xfffffff0135b9890  fp: 0xffffffe80fc2bf00
		  lr: 0xfffffff0136705a0  fp: 0x0000000000000000

Genshin impact, I'm not sure if DRI_PRIME was being used, i was using lutris.

panic(cpu 1 caller 0xfffffff022618190): x86 CPU CATERR detected
Debugger message: panic
Memory ID: 0xff
OS release type: User
OS version: 18P4663
macOS version: Not set
Kernel version: Darwin Kernel Version 20.5.0: Fri May  7 22:04:14 PDT 2021; root:xnu-7195.121.3~8/RELEASE_ARM64_T8010
Kernel UUID: 873F4AC5-B66C-3016-8EAC-39D538EEB5A3
iBoot version: iBoot-6723.120.36
secure boot?: YES
x86 EFI Boot State: 0xd
x86 System State: 0x0
x86 Power State: 0x0
x86 Shutdown Cause: 0x5
x86 Previous Power Transitions: 0x405060400
PCIeUp link state: 0x89271614
Paniclog version: 13
Kernel slide:      0x000000001a6bc000
Kernel text base:  0xfffffff0216c0000
mach_absolute_time: 0x33e71a8170
Epoch Time:        sec       usec
  Boot    : 0x60cab721 0x0006981f
  Sleep   : 0x60cafde3 0x0009bb27
  Wake    : 0x60cb15a9 0x0001e194
  Calendar: 0x60cb20f0 0x000e78b5

Panicked task 0xffffffe19a1a8630: 3276 pages, 227 threads: pid 0: kernel_task
Panicked thread: 0xffffffe19a42fc00, backtrace: 0xffffffe81033b700, tid: 402
		  lr: 0xfffffff021e0b49c  fp: 0xffffffe81033b750
		  lr: 0xfffffff021e0b2f4  fp: 0xffffffe81033b7c0
		  lr: 0xfffffff021f36df0  fp: 0xffffffe81033b890
		  lr: 0xfffffff0224315fc  fp: 0xffffffe81033b8a0
		  lr: 0xfffffff021e0b028  fp: 0xffffffe81033bc20
		  lr: 0xfffffff021e0b028  fp: 0xffffffe81033bc80
		  lr: 0xfffffff022e3b200  fp: 0xffffffe81033bca0
		  lr: 0xfffffff022618190  fp: 0xffffffe81033bcd0
		  lr: 0xfffffff022606520  fp: 0xffffffe81033bd30
		  lr: 0xfffffff02260841c  fp: 0xffffffe81033bdc0
		  lr: 0xfffffff022605bc4  fp: 0xffffffe81033be50
		  lr: 0xfffffff02250a814  fp: 0xffffffe81033be80
		  lr: 0xfffffff022386010  fp: 0xffffffe81033bec0
		  lr: 0xfffffff022385890  fp: 0xffffffe81033bf00
		  lr: 0xfffffff02243c5a0  fp: 0x0000000000000000

genshin impact no igpu stuff. I locked the screen while game was open, came back to reboot:

CPU Machine Check Architecture Error Dump (CPU: UNKNOWN, CPUID: 0x80000009000906EA)
Core: 0 
  IA32_MC4_STATUS=0xFE00000000800400
  IA32_MC4_CTL=0x0
  IA32_MC4_ADDR=0x14947B69
  IA32_MC4_MISC=0x14947B69
Core: 0 
  IA32_MC11_STATUS=0xEE0000000040110A
  IA32_MC11_CTL=0x0
  IA32_MC11_ADDR=0xFF71EE40
  IA32_MC11_MISC=0x7880000086

same on i3 instead of sway and only in the login screen. The last one may have been from something else?

panic(cpu 1 caller 0xfffffff01bd30190): x86 CPU CATERR detected
Debugger message: panic
Memory ID: 0xff
OS release type: User
OS version: 18P4663
macOS version: Not set
Kernel version: Darwin Kernel Version 20.5.0: Fri May  7 22:04:14 PDT 2021; root:xnu-7195.121.3~8/RELEASE_ARM64_T8010
Kernel UUID: 873F4AC5-B66C-3016-8EAC-39D538EEB5A3
iBoot version: iBoot-6723.120.36
secure boot?: YES
x86 EFI Boot State: 0xd
x86 System State: 0x0
x86 Power State: 0x0
x86 Shutdown Cause: 0x1
x86 Previous Power Transitions: 0x405060400
PCIeUp link state: 0x89271614
Paniclog version: 13
Kernel slide:      0x0000000013dd4000
Kernel text base:  0xfffffff01add8000
mach_absolute_time: 0x8374e38b5
Epoch Time:        sec       usec
  Boot    : 0x60dd4cee 0x000d7e9e
  Sleep   : 0x60dd4e6d 0x00076dad
  Wake    : 0x60dd4e6f 0x00072ce1
  Calendar: 0x60dd52ab 0x0002779c

Panicked task 0xffffffe199f48630: 4897 pages, 227 threads: pid 0: kernel_task
Panicked thread: 0xffffffe199f67200, backtrace: 0xffffffe80427b700, tid: 414
		  lr: 0xfffffff01b52349c  fp: 0xffffffe80427b750
		  lr: 0xfffffff01b5232f4  fp: 0xffffffe80427b7c0
		  lr: 0xfffffff01b64edf0  fp: 0xffffffe80427b890
		  lr: 0xfffffff01bb495fc  fp: 0xffffffe80427b8a0
		  lr: 0xfffffff01b523028  fp: 0xffffffe80427bc20
		  lr: 0xfffffff01b523028  fp: 0xffffffe80427bc80
		  lr: 0xfffffff01c553200  fp: 0xffffffe80427bca0
		  lr: 0xfffffff01bd30190  fp: 0xffffffe80427bcd0
		  lr: 0xfffffff01bd1e520  fp: 0xffffffe80427bd30
		  lr: 0xfffffff01bd2041c  fp: 0xffffffe80427bdc0
		  lr: 0xfffffff01bd1dbc4  fp: 0xffffffe80427be50
		  lr: 0xfffffff01bc22814  fp: 0xffffffe80427be80
		  lr: 0xfffffff01ba9e010  fp: 0xffffffe80427bec0
		  lr: 0xfffffff01ba9d890  fp: 0xffffffe80427bf00
		  lr: 0xfffffff01bb545a0  fp: 0x0000000000000000

Seems to be consistent if i either lock the screen or use dri prime.

Genshin, locked screen. I think this is specific to my MacBook, if someone else has the same problem, please comment here or tell me in the t2linux discord, it'll give me more motivation to figure out what's happening.

My running theory is that it's caused by the dgpu doing complex things when there's nothing for it to directly output to (i.e. dri_prime has no display connected to the dgpu, when the display is off it might go into a powersaving mode that disconnects it).

panic(cpu 1 caller 0xfffffff00e9bc190): x86 CPU CATERR detected
Debugger message: panic
Memory ID: 0xff
OS release type: User
OS version: 18P4663
macOS version: Not set
Kernel version: Darwin Kernel Version 20.5.0: Fri May  7 22:04:14 PDT 2021; root:xnu-7195.121.3~8/RELEASE_ARM64_T8010
Kernel UUID: 873F4AC5-B66C-3016-8EAC-39D538EEB5A3
iBoot version: iBoot-6723.120.36
secure boot?: YES
x86 EFI Boot State: 0xd
x86 System State: 0x0
x86 Power State: 0x0
x86 Shutdown Cause: 0x5
x86 Previous Power Transitions: 0x405060400
PCIeUp link state: 0x89271611
Paniclog version: 13
Kernel slide:      0x0000000006a60000
Kernel text base:  0xfffffff00da64000
mach_absolute_time: 0x4fb194fde7
Epoch Time:        sec       usec
  Boot    : 0x60dd52c0 0x0006e5fc
  Sleep   : 0x60dd69d9 0x0007195c
  Wake    : 0x60dd6aa6 0x00081c43
  Calendar: 0x60dd93db 0x000ccddf

Panicked task 0xffffffe19a040630: 3393 pages, 226 threads: pid 0: kernel_task
Panicked thread: 0xffffffe19a189a00, backtrace: 0xffffffe810f6b700, tid: 404
		  lr: 0xfffffff00e1af49c  fp: 0xffffffe810f6b750
		  lr: 0xfffffff00e1af2f4  fp: 0xffffffe810f6b7c0
		  lr: 0xfffffff00e2dadf0  fp: 0xffffffe810f6b890
		  lr: 0xfffffff00e7d55fc  fp: 0xffffffe810f6b8a0
		  lr: 0xfffffff00e1af028  fp: 0xffffffe810f6bc20
		  lr: 0xfffffff00e1af028  fp: 0xffffffe810f6bc80
		  lr: 0xfffffff00f1df200  fp: 0xffffffe810f6bca0
		  lr: 0xfffffff00e9bc190  fp: 0xffffffe810f6bcd0
		  lr: 0xfffffff00e9aa520  fp: 0xffffffe810f6bd30
		  lr: 0xfffffff00e9ac41c  fp: 0xffffffe810f6bdc0
		  lr: 0xfffffff00e9a9bc4  fp: 0xffffffe810f6be50
		  lr: 0xfffffff00e8ae814  fp: 0xffffffe810f6be80
		  lr: 0xfffffff00e72a010  fp: 0xffffffe810f6bec0
		  lr: 0xfffffff00e729890  fp: 0xffffffe810f6bf00
		  lr: 0xfffffff00e7e05a0  fp: 0x0000000000000000

I can consistently reproduce:

1. Open game
2. swaymsg "output * dpms off"
3. wait a few minutes, it crashes

I need to figure out what games are doing in causing this, other games do it too.

panic(cpu 1 caller 0xfffffff01c2f8190): x86 CPU CATERR detected
Debugger message: panic
Memory ID: 0xff
OS release type: User
OS version: 18P4663
macOS version: Not set
Kernel version: Darwin Kernel Version 20.5.0: Fri May  7 22:04:14 PDT 2021; root:xnu-7195.121.3~8/RELEASE_ARM64_T8010
Kernel UUID: 873F4AC5-B66C-3016-8EAC-39D538EEB5A3
iBoot version: iBoot-6723.120.36
secure boot?: YES
x86 EFI Boot State: 0xd
x86 System State: 0x0
x86 Power State: 0x0
x86 Shutdown Cause: 0xf3
x86 Previous Power Transitions: 0x70707060400
PCIeUp link state: 0x1614
Paniclog version: 13
Kernel slide:      0x000000001439c000
Kernel text base:  0xfffffff01b3a0000
mach_absolute_time: 0x1a189f6bd
Epoch Time:        sec       usec
  Boot    : 0x60e046c8 0x000ae3f2
  Sleep   : 0x00000000 0x00000000
  Wake    : 0x00000000 0x00000000
  Calendar: 0x60e047e9 0x000393cd

Panicked task 0xffffffe19a1d4630: 3203 pages, 226 threads: pid 0: kernel_task
Panicked thread: 0xffffffe19a254e00, backtrace: 0xffffffe8048db700, tid: 441
		  lr: 0xfffffff01baeb49c  fp: 0xffffffe8048db750
		  lr: 0xfffffff01baeb2f4  fp: 0xffffffe8048db7c0
		  lr: 0xfffffff01bc16df0  fp: 0xffffffe8048db890
		  lr: 0xfffffff01c1115fc  fp: 0xffffffe8048db8a0
		  lr: 0xfffffff01baeb028  fp: 0xffffffe8048dbc20
		  lr: 0xfffffff01baeb028  fp: 0xffffffe8048dbc80
		  lr: 0xfffffff01cb1b200  fp: 0xffffffe8048dbca0
		  lr: 0xfffffff01c2f8190  fp: 0xffffffe8048dbcd0
		  lr: 0xfffffff01c2e6520  fp: 0xffffffe8048dbd30
		  lr: 0xfffffff01c2e841c  fp: 0xffffffe8048dbdc0
		  lr: 0xfffffff01c2e5bc4  fp: 0xffffffe8048dbe50
		  lr: 0xfffffff01c1ea814  fp: 0xffffffe8048dbe80
		  lr: 0xfffffff01c066010  fp: 0xffffffe8048dbec0
		  lr: 0xfffffff01c065890  fp: 0xffffffe8048dbf00
		  lr: 0xfffffff01c11c5a0  fp: 0x0000000000000000

Closed the game and then turned off display.

panic(cpu 1 caller 0xfffffff0147bc190): x86 CPU CATERR detected
Debugger message: panic
Memory ID: 0xff
OS release type: User
OS version: 18P4663
macOS version: Not set
Kernel version: Darwin Kernel Version 20.5.0: Fri May  7 22:04:14 PDT 2021; root:xnu-7195.121.3~8/RELEASE_ARM64_T8010
Kernel UUID: 873F4AC5-B66C-3016-8EAC-39D538EEB5A3
iBoot version: iBoot-6723.120.36
secure boot?: YES
x86 EFI Boot State: 0xd
x86 System State: 0x0
x86 Power State: 0x0
x86 Shutdown Cause: 0xf3
x86 Previous Power Transitions: 0x70707060400
PCIeUp link state: 0x1614
Paniclog version: 13
Kernel slide:      0x000000000c860000
Kernel text base:  0xfffffff013864000
mach_absolute_time: 0x32e9452d7c
Epoch Time:        sec       usec
  Boot    : 0x60e121c1 0x0004b9f9
  Sleep   : 0x00000000 0x00000000
  Wake    : 0x00000000 0x00000000
  Calendar: 0x60e14554 0x000b15b0

Panicked task 0xffffffe19a080630: 3229 pages, 226 threads: pid 0: kernel_task
Panicked thread: 0xffffffe19a159400, backtrace: 0xffffffe80433b700, tid: 373
		  lr: 0xfffffff013faf49c  fp: 0xffffffe80433b750
		  lr: 0xfffffff013faf2f4  fp: 0xffffffe80433b7c0
		  lr: 0xfffffff0140dadf0  fp: 0xffffffe80433b890
		  lr: 0xfffffff0145d55fc  fp: 0xffffffe80433b8a0
		  lr: 0xfffffff013faf028  fp: 0xffffffe80433bc20
		  lr: 0xfffffff013faf028  fp: 0xffffffe80433bc80
		  lr: 0xfffffff014fdf200  fp: 0xffffffe80433bca0
		  lr: 0xfffffff0147bc190  fp: 0xffffffe80433bcd0
		  lr: 0xfffffff0147aa520  fp: 0xffffffe80433bd30
		  lr: 0xfffffff0147ac41c  fp: 0xffffffe80433bdc0
		  lr: 0xfffffff0147a9bc4  fp: 0xffffffe80433be50
		  lr: 0xfffffff0146ae814  fp: 0xffffffe80433be80
		  lr: 0xfffffff01452a010  fp: 0xffffffe80433bec0
		  lr: 0xfffffff014529890  fp: 0xffffffe80433bf00
		  lr: 0xfffffff0145e05a0  fp: 0x0000000000000000

amdgpu.bapm=0 amdgpu.msi=0 amdgpu.dpm=0 amdgpu.aspm=0 amdgpu.runpm=0 seems to fix it, need to figure out which one it was but i'll do that later

amdgpu.dpm=0 fixes it without too much impact on battery life. Games feel a bit slower with dri_prime, i need to see if that's just dri_prime or if disabling dpm has an impact.

I had similar issues with my MacBook Pro 16,1. For instance, when I try to change the resolution, the device freezes with a black screen then automatically reboots with CATERR message, applying amdgpu.dpm=0 fixed it