Vogtinator/README.md

## README.md

      
    Raw
  

              README.md
            
          
    When following this guide on a host not capable of native arm64 KVM, replace -M virt -cpu host -accel kvm with -M virt,virtualization=on -cpu max.
The reason for virtualization=on is that the Windows bootloader does an smc #0 PSCI call, but without EL2, QEMU's TCG does not handle those because PSCI is in HVC mode and such that instruction is treated as undefined. With KVM enabled, smc #0 is handled properly.
Workaround in QEMU for using TCG without virtualization=on:
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index b871350856..de11ff51d1 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -2056,7 +2056,7 @@ static void machvirt_init(MachineState *machine)
      */
     if (vms->secure && firmware_loaded) {
         vms->psci_conduit = QEMU_PSCI_CONDUIT_DISABLED;
-    } else if (vms->virt) {
+    } else if (1 || vms->virt) {
         vms->psci_conduit = QEMU_PSCI_CONDUIT_SMC;
     } else {
         vms->psci_conduit = QEMU_PSCI_CONDUIT_HVC;

Download the VM disk image from https://www.microsoft.com/en-us/software-download/windowsinsiderpreviewarm64 and the latest virtio-win.iso from https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/latest-virtio/. Build 22589 is known to work, while build 25226 is known not to work (unhandled exception during boot).
Perform the initial boot. For some reason performance is abysmal with the GTK UI, so use VNC: qemu-system-aarch64 -M virt -cpu host -accel kvm -m 2G -smp 2 -device ramfb -bios /usr/share/qemu/qemu-uefi-aarch64.bin -device qemu-xhci -device usb-kbd -device usb-tablet -drive file=Windows11_InsiderPreview_Client_ARM64_en-us_22598.VHDX,format=vhdx,if=none,id=boot -device usb-storage,drive=boot,serial=boot -drive file=virtio-win.iso,media=cdrom,if=none,id=iso -device usb-storage,drive=iso -nic user,model=virtio-net-pci,mac=52:54:98:76:54:32 -vnc :0. On some earlier QEMU/kernel (?) versions, ramfb has some issues with invalidation (?) causing artifacts, you'll have to ignore them until virtio-gpu can be used. At least since build 25931, it's necessary to provide a network connection to finish the initial setup. The default QEMU MAC address will result in a false UUID match with some AutoPilot enrolled devices, forcing an "Intellek" domain login. To avoid that, use a different MAC address, e.g. 52:54:98:76:54:32 from the QEMU man page.
To have a higher resolution with ramfb, enter the OVMF setup by pressing F10 during boot, change the preferred resolution to 1024x768 and reset.
This might take a while. If it appears stuck at "Getting ready" for 10min (give it a bit more time without KVM) or you get a BSOD (watchdog timeout), reset. If you get an error message "Windows installation cannot proceed", press Shift-F10 to open a cmd, run regedit and set HKEY_LOCAL_MACHINE\SYSTEM\Setup\Status\ChildCompletion\Setup to 3 and click on Ok to reboot.
When it asks for the initial setup (country, keyboard layout) press Shift-F10 to open a cmd, or (if it does not work) Ctrl-Shift-Esc to open the task manager and launch cmd with admin privs.
Run bcdedit /set TESTSIGNING ON to allow loading some drivers during boot. While virtio drivers are signed, they don't work without this for some reason.
Start explorer.exe, navigate to the virtio drivers in D: and install NetKVM, viogpudo, viostor and vioscsi by right clicking on the .inf file within the w11/ARM64 folders and selecting "Install". If it complains about driver signatures, navigate to the cert folder and install the certificate into the devices Trusted Publishers store.
Complete the setup as usual. To avoid using a Microsoft account, attempt to sign in as "Administrator" with some random string as password, then click "Next" on the "Oops, something went wrong" screen.
Shut down Windows.
Now virtio devices can be used: qemu-system-aarch64 -M virt -cpu host -accel kvm -m 4G -smp 4 -device virtio-gpu-pci -bios /usr/share/qemu/qemu-uefi-aarch64.bin -device qemu-xhci -device usb-kbd -device usb-tablet -drive file=Windows11_InsiderPreview_Client_ARM64_en-us_22598.VHDX,format=vhdx,if=none,id=boot -device virtio-scsi -device scsi-hd,drive=boot -nic user,model=virtio-net-pci,mac=52:54:98:76:54:32 -vnc :0. The first boot will take a while with a disabled display, be patient. If it does not work, try to boot with the previous QEMU command but also add -device virtio-gpu-pci and some virtio-scsi HD, to make sure the driver is properly selected.


## openQA setup.md

      
    Raw
  

              openQA setup.md
            
          
    openQA expects a resolution of 1024x768. The default is 1280x1024 and for some reason there's no option to change that in the UI, so it has to be done in the registry directly. This needs to be done with a very similar command line to actual QA jobs, otherwise the device doesn't match (based on PCI address).

Ctrl-Shift-Esc to open taskmgr
"Run new task", enter regedit and run it with admin privs
Navigate to HKLM/SYSTEM/CurrentControlSet/Control/GraphicsDrivers/Configuration/RHT*/00
Set PrimSurfSize.cx to 1024, PrimSurfSize.cy to 768 and Stride to 4096 (decimal!)
Reboot

To avoid having to add new needles, switch the desktop wallpaper to the one used by the existing win11 tests and change the terminal in the developer settings to "Windows Console Host". Also set a fixed background picture for the lock and sign in screens.
Windows on Arm has a different model for serial ports, the usual methods to access them no longer work. However, FTDI ported the old serial port bus to Arm and it works with the usb-serial device! Download https://www.ftdicommunity.com/index.php?topic=753.0 and extract it somewhere. The "QEMU USB SERIAL" device needs the driver assigned manually by using the device manager to
To use it with openQA, pass -chardev ringbuf,id=usbserial,logfile=serial0,logappend=on -device usb-serial,chardev=usbserial. This uses a bit of a hack as it writes into the same file as the default serial port (pl011, not usable as COM port), but there's no other way as QEMU does not allow using a chardev from multiple devices. Native support for usb-serial in os-autoinst would be the right way. usb-serial also conflicts with -no-migratable, so QEMU_DISABLE_SNAPSHOTS=1 needs to be passed.
If this is build 25267 or newer, SMB guest authentication needs to be enabled (https://learn.microsoft.com/en-us/troubleshoot/windows-server/networking/guest-access-in-smb2-is-disabled-by-default), but it appears to be broken even with this, just differently...

Run gpedit.msc
Computer Configuration > Administrative Templates > Network > Lanman Workstation.
Enable insecure guest logons -> Enable

Not specific to Arm: To optimize the qcow2 size, start the VM with discard=unmap on the qcow2 block dev. In a powershell, run Optimize-Volume -Verbose -DriveLetter C -ReTrim and shut down. Run qemu-img convert -pc -O qcow2 for_cleanup.qcow2 Win11ArmBuild25931.qcow2 to discard the unmapped space and perform recompression.
Based on a win11 x86_64 (UEFI) test:
openqa-clone-job --within-instance https://openqa.opensuse.org --skip-chained-deps 3546891 _GROUP=38 ARCH=aarch64 HDD_1=win11_22598.qcow2 QEMUMACHINE=virt QEMUVGA= QEMU_VIDEO_DEVICE=virtio-gpu-pci UEFI_PFLASH_CODE=/usr/share/qemu/aavmf-aarch64-code.bin UEFI_PFLASH_VARS=/usr/share/qemu/aavmf-aarch64-vars.bin WORKER_CLASS=qemu_aarch64 HDDMODEL=scsi-hd QEMU_DISABLE_SNAPSHOTS=1 "QEMU_APPEND=chardev ringbuf,id=usbserial,logfile=serial0,logappend=on -device usb-serial,chardev=usbserial"

  
## WSL2.md

      
    Raw
  

              WSL2.md
            
          
    It's also possible to run WSL2 inside the Windows VM. However, that uses Hyper-V and thus requires hardware virtualization.
Nested virtualization is not supported by existing hardware yet (and might lead to issues in the guest as well), so software emulation is needed.
Just -M virt,virtualization=on -cpu max as used above in the non-KVM case will not work though: Windows no longer boots if the hypervisor is enabled.
With WinDbg attached it complains about unexpected register values which indicate that it expects the CPU to have EL3.
Using https://github.com/ARM-software/arm-trusted-firmware in EL3 (following https://trustedfirmware-a.readthedocs.io/en/latest/plat/qemu.html) does not work though,
it leads to Windows failing to initialize interrupts. Presumably some of the ATF initialization confuses the Windows kernel.
There's a way around ATF though: QEMU's -kernel option allows to have a CPU with EL3 but boot directly into EL2. To be able to use OVMF with that option (instead of -bios),
it has to be built as Linux kernel image. A prebuilt QEMU_EFI.fd is available at http://snapshots.linaro.org/components/kernel/leg-virt-tianocore-edk2-upstream/latest/QEMU-KERNEL-AARCH64/RELEASE_GCC5/QEMU_EFI.fd.
With -cpu max this still fails though because Windows tries to access HDFGRTR_EL2, which ends up trapping into EL3 which does not exist -> stuck.
Fix for QEMU (https://lists.nongnu.org/archive/html/qemu-devel/2023-09/msg03039.html):
diff --git a/hw/arm/boot.c b/hw/arm/boot.c
index 720f22531a..24fa169060 100644
--- a/hw/arm/boot.c
+++ b/hw/arm/boot.c
@@ -761,6 +761,10 @@ static void do_cpu_reset(void *opaque)
                     if (cpu_isar_feature(aa64_hcx, cpu)) {
                         env->cp15.scr_el3 |= SCR_HXEN;
                     }
+                    if (cpu_isar_feature(aa64_fgt, cpu)) {
+                        env->cp15.scr_el3 |= SCR_FGTEN;
+                    }
+
                     /* AArch64 kernels never boot in secure mode */
                     assert(!info->secure_boot);
                     /* This hook is only supported for AArch32 currently:
As a workaround, -cpu neoverse-n1 can be used, which does not expose the feature with that register.
QEMU before 8.1 is missing also SCR_HXEN and probably others, so won't work without code changes.
Full QEMU command with WSL2/Hyper-V support (tested with Build 25931):
qemu-system-aarch64 -M virt,virtualization=on,gic-version=3,secure=on -cpu neoverse-n1 -m 4G -smp 4 -device virtio-gpu-pci -kernel QEMU_EFI.fd -device qemu-xhci -device usb-kbd -device usb-tablet -drive file =Win11ArmBuild25931.qcow2,if=none,id=boot -device virtio-scsi -device scsi-hd,drive=boot,serial=hd0 -nic user,model=virtio-net-pci,mac=52:54:98:76:54:32 -vnc 0