Skip to content

Instantly share code, notes, and snippets.

@mowings
Last active December 30, 2020 08:01
Show Gist options
  • Star 13 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save mowings/f7e348262d61eebf7b83754d3e028f6c to your computer and use it in GitHub Desktop.
Save mowings/f7e348262d61eebf7b83754d3e028f6c to your computer and use it in GitHub Desktop.
Set up xhyve with Ubuntu 16.04

Introduction

Setting up an OS to work with xhyve can be a bit tricky, but it's not impossible. These intructions should be generally applicable to most versions of Linux.

Create a disk image file

Xhyve will use a file as a logical disk. Be sure you have the filesize you need, because growing the file later is tricky to impossible (you could try to use qcow or similar to get around this, but qcow volumes can be tricky to mount).

dd if=/dev/zero of=hdd.img bs=1g count=32 # Create a 32 gig disk. Raise 'count' as desired

Grab required kernel files

Xhyve loads the kernel and initrd images from the local filesystem -- you'll need to get these from a mounted iso in order to boot an install cd. First download an iso. You probably want to use the mini.iso.

Once downloaded, you'll need mount the iso and copy off the kernel files. But these isos are not directly mountable in osx. To make them mountable, you'll need to make a compatible copy (which just means that the first 2k block is empty) and then mount this copy:

dd if=/dev/zero bs=2k count=1 of=tmp.iso
dd if=mini.iso bs=2k skip=1 >> tmp.iso
hdiutil attach tmp.iso

Next copy the kernel files from the mounted volume

cp /Volumes/CDROM/linux .
cp /Volumes/CDROM/initrd.gz .

You can delete tmp.iso now -- you wont need it again.

You can now boot into an installer. Run the attached install.sh script. Follow the prompts. However, once the installation is complete -- do not reboot. Select Go Back and then Execute a shell. We are going to use nc to copy off the system initrd and linux kernel that were just installed. These are the latest, non-installer versions of these files. We can't use the versions we copied earlier, because they will simply boot us back in to the installer.

At the shell prompt:

cd /target
sbin/ifconfig
tar c boot | nc -l -p 1234

Make note of the ip returned from ifconfig. On your osx host:

 nc <IP from above> 1234 | tar x

This should drop the latest boot images (kernel, initrd) into the boot/ subdirectory.

Exit the shell, and let the installation complete.

You should be able to boot up the new machine using the run.sh script below. Note that you will probably need to replace the exact filenames used for LINUX and INITRD with whatever you downloaded in boot/ in the previous step. The console is a little wonky -- in general, I install openssh when I am setting up an xhyve instance, and use that to finish setting up the new vm after booting it up.

VPNs

The VM can lose connectivity to any VPNs running on the host after those vpns have been restarted, or after sleeping. Use the attached masq.sh to restore connectivity when this happens.

#!/bin/bash
KERNEL="linux"
INITRD="initrd.gz"
CMDLINE="earlyprintk=serial console=ttyS0"
# Guest Config
MEM="-m 1G"
IMG_CD="-s 1,ahci-cd,mini.iso"
IMG_HDD="-s 2,virtio-blk,hdd.img"
NET="-s 3:0,virtio-net,en0"
PCI_DEV="-s 0:0,hostbridge -s 31,lpc"
LPC_DEV="-l com1,stdio"
ACPI="-A"
# and now run
sudo xhyve $ACPI $MEM $PCI_DEV $LPC_DEV $NET $IMG_CD $IMG_HDD -f kexec,$KERNEL,$INITRD,"$CMDLINE"
#!/bin/bash
interfaces=( $(netstat -in | egrep 'utun\d .*\d+\.\d+\.\d+\.\d+' | cut -d ' ' -f 1) )
rulefile="rules.tmp"
echo "" > $rulefile
sudo pfctl -a com.apple/tun -F nat
for i in "${interfaces[@]}"
do
RULE="nat on ${i} proto {tcp, udp, icmp} from 192.168.64.0/24 to any -> ${i}"
echo $RULE >> $rulefile
done
sudo pfctl -a com.apple/tun -f $rulefile
KERNEL="boot/vmlinuz-4.4.0-21-generic"
INITRD="boot/initrd.img-4.4.0-21-generic"
CMDLINE="earlyprintk=serial console=ttyS0 acpi=off root=/dev/vda1 ro"
UUID="-U 8e7af180-c54d-4aa2-9bef-59d94a1ac572" # A UUID will ensure we get a consistent ip address assigned
# Guest Config
MEM="-m 3G"
IMG_HDD="-s 2,virtio-blk,hdd.img"
NET="-s 3:0,virtio-net,en0"
PCI_DEV="-s 0:0,hostbridge -s 31,lpc"
LPC_DEV="-l com1,stdio"
ACPI="-A"
# and now run
sudo xhyve $UUID $ACPI $MEM $PCI_DEV $LPC_DEV $NET $IMG_CD $IMG_HDD -f kexec,$KERNEL,$INITRD,"$CMDLINE"
@dontorzz
Copy link

  • I extracted the kernel files from the mini.iso dowloaded from the link above.
  • I created the install.sh file and I ran it.
  • I got an error as follows.
    [ 0.785032] Kernel Offset: 0x1da00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
    [ 0.787160] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x00007f00
    Do you happen to know what could be messed up? Thanks.
  • My system = macOS Mojave v10.14.2

@pcvm
Copy link

pcvm commented Mar 8, 2019

The approach described here worked for me on macOS 10.14.3 - thanks.
Some notes:

  1. I had crashes and disk access fails until I used a consistent set of input files obtained via commands
    wget http://archive.ubuntu.com/ubuntu/dists/xenial-updates/main/installer-amd64/current/images/netboot/ubuntu-installer/amd64/initrd.gz
    wget http://archive.ubuntu.com/ubuntu/dists/xenial-updates/main/installer-amd64/current/images/netboot/ubuntu-installer/amd64/linux
    wget http://archive.ubuntu.com/ubuntu/dists/xenial-updates/main/installer-amd64/current/images/netboot/mini.iso
    as outlined in Makefile https://github.com/charlesportwoodii/xhyve-ubuntu/blob/master/Makefile (on 8Mar2019, resulting system is Ubuntu 16.04.6 LTS).
  2. When transferring the final system kernel and initrd files, I also needed ifconfig to discover the emulated system's IP i.e.
    export PATH=$PATH:/target/bin:/target/usr/bin:/target/sbin:/target/usr/sbin:/target/usr/local/bin ; ifconfig

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment