CyberShadow/osx.org

## osx.org

      
    Raw
  

              osx.org
            
          
    Setting up OS X from scratch (on libvirt/QEMU/KVM)

Motivation


  I have a MacBook1,1, but its HDD died.
  I need an instance of OS X mainly to build OS X binaries of some of my open-source software.

OS X 10.4 (Tiger)


  Obtain installation medium
    
      I had the OS X 10.4 (Tiger) OEM installation disks that came with my MacBook1,1.
      When ripping /dev/sr0 with dd, my disks have the MD5 sums:
        
          77d30216857aad8909ebc1e53d129750  MyMacBook-disk1.iso
          a88b7fc3f3e827593b5a62e0ed273825  MyMacBook-disk2.iso
        
      
  Patch installation medium to support installation on any system
    
      The installer app will check the DSDT values.
        
          The check is to make sure that the OEM disks are only used for the hardware that they came with.
          The check seems to be done by a bit of JS code.
            
              There is a blacklist and a whitelist.
              If the DSDT doesn’t match, the installer will show the error message “This software cannot be installed on this computer”.
            
          
          DSDT is “Differentiated System Description Table” and seems to be embedded in the BIOS firmware.
          This value can be viewed by running sysctl hw.model or in the System Profiler app (from the OS X installer live CD).
          For me, the value when running in QEMU with the default BIOS was “BXPCDSDT0,1”
            
              “BXPCDSDT” is:
                
                  “BXPC” is probably “Bochs PC”
                  “DSDT” is “Differentiated System Description Table”
                
              
              “0,1” is probably the hardware version OSLT
            
          
      The proper way to patch out this check seems to be:
        
          Image the disk to a writable .dmg
          Mount it on an OS X system
          Edit /System/Installation/Packages/OSInstall.mpkg/Contents/OSInstall.dist
          Check blacklist (badMachines) and update whitelist (hwbeSupportedMachines)
          Unmount the .dmg and burn it to a new CD
          Video: https://www.youtube.com/watch?v=ngJ9q9wHcS0
        
      
      An easier way is to use a hex editor to search for hwbeSupportedMachines in the .iso file, and patch the very next return false; to return true ;
        
          On my MyMacBook-disk1.iso, there were three occurrences
            
              Patching just the first occurrence did not cause the error message to go away
                
                  (it was probably inside the installed content… thus it’s probably better to patch them all).
                
              
  Configure the VM
    
      Configuration
        
          Software
            
              Most of the examples online use raw QEMU command lines.
              I wanted to set it up in libvirt, so that it worked like my other VMs.
              Not all QEMU settings map to libvirt XML settings, so raw QEMU command lines have to be used.
            
          
          Machine
            
              All sources say that you must use the QEMU q35 machine, and not the default i440FX.
              Note that libvirt automatically expands q35 to pc-q35-2.8 (the latest version of that machine). This is harmless.
            
          
          CPU
            
              Sources say to use “Penryn”.
              Make sure the vendor is GenuineIntel
                
                  QEMU command line: -cpu Penryn,vendor=GenuineIntel
                  libvirt: <cpu mode='custom' match='exact'> <model fallback='allow'>Penryn</model> <vendor>Intel</vendor> </cpu>
                
              
          BIOS <<gsomlo-bios>>
            
              Gabriel Somlo’s page recommends using a custom UEFI firmware, however this wasn’t necessary (and in fact did not work) with OS X 10.4.
                
                  Said firmware: https://github.com/gsomlo/edk2/tree/macboot
                    
                      AUR package: https://aur.archlinux.org/packages/edk2-ovmf-macboot-git/
                    
                  
                  Attempting to build and use the firmware as directed results in:
                    
                      the VM sits at the TianoCore logo for about 1 minute
                      After which it goes to the UEFI Interactive Shell (screenshot) and hangs apparently indefinitely
                    
                  
              The default QEMU BIOS (SeaBios) seems to work fine for OS X 10.4 (aside the necessary DSDT patch mentioned above).
            
          
          Kernel
            
              A custom kernel seems to be necessary to be able to boot OS X.
                
                  Gabriel Somlo’s old page says that it’s needed “to bridge the gap between SeaBIOS and the Apple EFI BIOS expected by Mac OS X”.
                  Since it’s absent on the new page, I assume its functionality has been folded into the patched BIOS, however I haven’t been able to get to work at all.
                  Unfortunately it can only be built on an OS X machine with Xcode, so we will need to use a pre-built binary.
                  Gabriel provides a pre-built binary we can use
                    
                      Download URL: http://www.contrib.andrew.cmu.edu/~somlo/OSXKVM/chameleon_svn2783_boot
                        
                          (but check his page for a newer version).
                        
                      
                  kholia’s OSX-KVM project seems to use something slightly different called “enoch” (a fork of Chameleon?). <<enoch>>
                    
                      Enoch source code is here: http://forge.voodooprojects.org/p/chameleon/source/changes/HEAD/
                    
                  
              Andrei Ostanin’s blog post mentions that the kernel can be replaced by the Chimera bootloader.
              You can specify the kernel in the <os> tag:
                <os>
  <type arch='x86_64' machine='pc-q35-2.8'>hvm</type>
  <kernel>/path/to/chameleon_svn2783_boot</kernel>
  <boot dev='hd'/>
</os>
                
              
          Storage
            
              The OS X installer needs the disk drive to be connected via IDE
              For some reason, libvirt doesn’t allow adding IDE devices to a q35 machine, despite QEMU supporting them
                
                  Bug report: https://bugzilla.redhat.com/show_bug.cgi?id=1437253 <<libvirt-ide-bug>>
                
              
              Workaround is to specify the devices via QEMU command lines:
                <qemu:arg value='-device'/>
<qemu:arg value='ide-drive,bus=ide.0,drive=MacDVD'/>
<qemu:arg value='-drive'/>
<qemu:arg value='id=MacDVD,if=none,snapshot=on,file=/path/to/osx-install-disk1.iso'/>

<qemu:arg value='-device'/>
<qemu:arg value='ide-drive,bus=ide.2,drive=MacHDD'/>
<qemu:arg value='-drive'/>
<qemu:arg value='id=MacHDD,if=none,file=/var/lib/libvirt/images/macos.qcow2'/>
                
                
                  Don’t forget to add the qemu namespace to the root XML element, e.g.:
                    <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
                    
                  
                  After installing the OS, you can remove these entries and add the HDD as a normal SATA device using virt-manager.
                
              
          Apple SMC
            
              An additional device must be present on the ISA bus to identify the computer as an Apple device
              The device must be configured with the OSK, a 64-character string
                
                  The OSK is copyrighted, so it is not mentioned here
                  The OSK is globally constant (the same on all machines, so it doesn’t identify a machine or such)
                  The OSK can be extracted by booting a live Linux CD on a Mac and running a program available from Gabriel Somlo’s old page
                  The OSK can also be found on the Internet, such as in kholia’s OSX-KVM project.
                
              
              The device must be configured as a QEMU parameter:
                <qemu:arg value='-device'/>
<qemu:arg value='isa-applesmc,osk=insert-real-64-char-OSK-here'/>
                
              
          Network
            
              The only network adapter I could get to work with OS X 10.4 is usb-net
              QEMU wiki networking guest hints mentions that rtl8139 should work, but it did not work for me
            
          
      References
        
          <<<Gabriel Somlo’s page>>>
http://www.contrib.andrew.cmu.edu/~somlo/OSXKVM/
          <<<Gabriel Somlo’s old page>>>
http://www.contrib.andrew.cmu.edu/~somlo/OSXKVM/index_old.html
          <<<Andrei Ostanin’s blog post>>>
https://blog.ostanin.org/2014/02/11/playing-with-mac-os-x-on-kvm/
          <<<kholia’s OSX-KVM project>>>
https://github.com/kholia/OSX-KVM
          <<<QEMU wiki networking guest hints>>>
http://wiki.qemu-project.org/Documentation/Networking#Guest_Hints
        
      
  Install OS X
    
      Installation should be mostly uneventful, with the ISO patched
      When prompted for a location to install the OS, start Disk Utility from the menu above to partition the HDD
        
          OS X doesn’t seem to use dedicated swap partitions, so a single HFS+ partition spanning the entire disk is fine
        
      
      An advanced install is preferred
        
          By default, it wants to install gigabytes of printer drivers, translations, and multimedia applications
          Deselecting the above gets it to not even request the second disk
        
      
      During installation, it prompts for details like an Apple ID and personal information (name/address), but these can be skipped
    
  
  Setup
    
      Perform Software Update
        
          After completion, check Software Update again and repeat, it takes a few iterations to install all updates
        
      
OS X 10.6 (Snow Leopard)

As I discovered, 10.4 does not include the Mac App Store, and cannot be easily upgraded to 10.6 (easier than doing a clean install, anyway).
  Gabriel Somlo’s page mentions that 10.6 is the last version to come on a disk, and it also debuts the App Store (which provides a way to obtain newer OS X versions), which is why it makes sense to start from this version.
  Installation/setup is mostly the same as for 10.4. Differences:

  The version I got (MD5 a83cec287b4ab2f0ef11264c580c4e2d) did not require patching any DSDT/hw.model checks, and agreed to install without modifications.
  The version I got came in .dmg format. QEMU can use these directly.

Unfortunately this was a dead end too:

  It is not possible to upgrade directly from Snow Leopard to the latest version, Sierra
  Older versions (between Sierra and Snow Leopard) are not directly available on the App Store.
  Although it’s possible to e.g. open the App Store entry for El Capitan through a link, downloading it still fails with the error “This version of OS X 10.11 cannot be installed on this computer.”

OS X 10.10 (Yosemite)


  The .dmg and .iso of the version I had (can’t remember where I got them from) refused to boot with:
    “Can’t find boot file: ‘/mach_kernel’”
    
      Apparently this is due to the boot ISO being invalid / unbootable - it was probably the .dmg as downloaded from the App Store.
    
  
  The OSX-KVM create-install-iso.sh script doesn’t work for Yosemite:
    
      Running the script with the default arguments as root causes the entire guest OS to crash.
      Running the script in non-root mode causes the script to crash.
      Probably not surprising, since it explicitly targets El Capitan and Sierra.
    
  
  I’ve tried to follow the steps on Gabriel Somlo’s page (section “2. Creating an OS X boot DVD iso image”)
    
      Since I was doing this on OS X Snow Leopard, I needed to use Pacifist to extract the Kernels directory from Essentials.pkg.
        
          It contained a single 10M file called “kernel”.
        
      
      I seem to have followed all steps successfully and created an Yosemite.iso file. However, it does not work.
        
          The BIOS the Gabriel Somlo’s page says to use still doesn’t work (as described above).
          With the standard OVMF UEFI firmware, the effect is the same as with Gabriel’s macboot branch, but without the 1-minute delay.
          Without Chameleon and the default BIOS, SeaBIOS considers the ISO unbootable.
          With Chameleon and the default BIOS, it gets further but crashes on boot:
            
              On boot, the BIOS complains about read errors.
              If there is more than one CPU/core present, the kernel panics about the pthread kernel extension not being loaded.
              If there is only one CPU/core present, it shows a gray progress bar and stops there.
            
          
OS X 10.12 (Sierra)


  A very recent Sierra update changed the SMC validation mechanism
    
      Adding an isa-applesmc QEMU device is currently insufficient
      Can be worked around by adding a kExt module instead
      https://github.com/kholia/OSX-KVM/issues/64
      Patches for QEMU exist (see Gabriel Somlo’s page) but aren’t merged yet
    
  
  Due to the above-described problems, I couldn’t find a way to get to 10.12 from my working 10.6 installation. So, I borrowed a friend’s work MacBook.
    
      The MacBook was running 10.12.5 (latest version of the time).
      I created three installation images using three methods:
        
          By following the instructions on Gabriel Somlo’s page
            
              Finally, Gabriel Somlo’s firmware does something other than hanging at the logo.
                
                  However, it still doesn’t seem to be enough to get things working.
                  With QEMU 2.9.0, it shows the Apple logo on a black background, and a progress bar which reaches a certain point, displays a “beach ball” mouse cursor, and hangs.
                    
                      The progress bar seems to move at a constant speed (instead of displaying some actual progress), as the point at which it hangs depends on whether the disk image is cached in memory (i.e. it reaches about 45% with a cold cache, and about 15% with a warm cache).
                      Gabriel sent me a patch set that passes -v to the macOS kernel, which enables debug output. Video recording of boot process is here.
                        
                          Looking online for the last line printed, this thread looks similar, which suggests a new problem since 10.12.4.
                          I’ve been told that installing 10.12.3 then upgrading to 10.12.5 should work, implying it’s likely a problem with the installer only.
                        
                      
                  With QEMU master, it does not even show the progress bar - only the Apple logo. It looks like a regression in QEMU.
                    
                      With the -v patch, looks like this.
                    
                  
          Using the create_install_iso.sh script from kholia’s OSX-KVM project.
            
              Gabriel Somlo’s firmware does not work with this image. The project’s boot script (which uses enoch) must be used instead.
              The boot screen here has a dark gray logo and a gray background instead of a black background like above. I hypothesize this is because the machine is booting in BIOS mode instead of UEFI mode.
              Installation and booting works!
            
          
          Using the createinstallmedia script, as suggested on Red Hat’s bugtracker.
            
              This required using a separate volume for the media. I emulated a 16GB USB flash drive using my phone (using DriveDroid).