As a quick recap, TSO or Total storing ordering is an M1 exclusive feature that enforces strong memory ordering similar to x86. The advantage of this is that emulators don't have to use fencing on weak memory model systems, notably most ARM systems. This gives a massive performance boost.
I wanted to intentionally turn off TSO inside an Ubuntu parallels VM with x86_64 emulation to test some linux binaries of my own which purposefully exploit weak order models to demonstrate some example race conditions. Ideally, it would have been as simple as writing a kernel driver to write bit 1 of ACTLR_EL1 on each core... but not really.
Before I dive into this, I'd recommend reading a bit about exception levels in ARM. The linux kernel runs at EL1 and the hypervisor operates at EL2. Now back to the topic at hand, it seemed like ACTLR_EL1 was frozen as 0 which seemed very odd due to my previous lack of understanding of how VHE works on ARM.
Marcan recently did a write up of the working of TSO on M1 https://gist.github.com/marcan/9ab73ca0614864bea0eea9e953c074d3. According to this documentation, setting ACTLR_EL1 is trapped by HCR_EL2.TACR and HACR_EL2<0>.
We will refer to these two documentation links https://developer.arm.com/documentation/ddi0601/2023-09/AArch64-Registers/HCR-EL2--Hypervisor-Configuration-Register and https://developer.arm.com/documentation/ddi0595/2021-03/AArch64-Registers/HACR-EL2--Hypervisor-Auxiliary-Control-Register. In addition to this, two bits of importance when we talk about VHE are the TGE and E2H bits of HCR_EL2. TGE is used to indicate host or guest OS, and setting E2H causes redirection of accesses from EL1 registers to EL2. HCR_EL2.TACR and HACR_EL2<0> if set on will cause a trap if ACTLR_EL1 is being accessed and TGE is currently 0 (meaning guest VM which we indeed are in)
In other words, accessing ACTLR_EL1 puts us at trying to negotiate with the hypervisor by switching to EL2 and handling this trap, of which I am pretty sure Virtualization.framework is going to straight up ignore. It was an interesting endeavour all things said. Perhaps we could disable this trapping by patching either Virtualization or Hypervisor directly?