Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Ubuntu 20.04 with Autoinstall using VirtualBox [DRAFT]

Ubuntu 20.04 + Autoinstall + VirtualBox

Intro

This guide will explain how to set up a VirtualBox VM running Ubuntu 20.04 and, using the new autoinstall tool and cloud-init, install OpenSSH server, add a group and a user, who can then connect to the machine via SSH. This was done on a 2016 MacBook Pro running macOS 10.15 "Catalina" and VirtualBox 6.1.6 (with the VirtualBox Guest Additions installed).

You will also need a group name, a username, a password hash and an SSH key pair.

How Autoinstall Works

Ubuntu's new installer subiquity includes an "autoinstall" tool that seems to extend the popular cloud-init tool. It provides a way for us to write a YAML file that will be "ingested" (what a delightful term) by the installer, checked for validity and then used to process the installation. Autoinstall is new but pretty flexible; we can automate the whole installation or choose for configure certain parts manually. Unlike with debian-installer, we won't have to remaster a custom install disk and can put our autoinstall data on a separate disk from which the installer will pull our setup configuration and run the installation. IRL, we'd put the autoinstall data on a USB thumb drive (or burn it onto a CD or DVD), but for this guide, we'll create a local disk image that we can connect to the VM like another hard drive.

Preflight

When the VM boots from the install disk, NoCloud will scan all attached disks, looking for one that meets all of these criteria:

  1. It is connected to the machine and visible immediately upon boot.
  2. It is formatted using FAT32 (as most portable USB drives are) or ISO 9660 (as optical disks are).
  3. Its volume label is CIDATA or cidata (short for "cloud-init data", presumably).

If these three conditions are met (and if the generator doesn't have a reason to block it), the installer will mount that volume, look inside it for a YAML file called user-data (no file extension), validate it and run it. The file can contain autoinstall data, cloud-init data, or (as we'll see) a bit of both.

Let's start by making a FAT32-formatted disk image (.dmg file) on which we can store our autoinstall configuration. Open a Terminal window and enter the following to create, format and mount the disk image on your Desktop:

$ hdiutil create -ov -fs fat32 -volname CIDATA -size 100m /path/to/my-autoinstall-data.dmg
$ hdiutil attach /path/to/my-autoinstall-data.dmg

Note: The native macOS Disk Utility app can create .dmg disk images, but neither "MS-DOS (FAT)" nor "exFAT" are FAT32.

VirtualBox can't read macOS' native disk image format, but you can use a hidden command to create a "raw" disk image. This creates a .vmdk disk image file in the same folder as the .dmg file, as a "pointer" that VirtualBox can read...

$ VBoxManage internalcommands createrawvmdk -filename /path/to/my-autoinstall-data.vmdk -rawdisk /path/to/my-autoinstall-data.dmg

VirtualBox Preparations

Download the Ubuntu 20.04 server installer disk image from Ubuntu Releases.

Launch VirtualBox, click File Virtual Media Manager... and click the Hard disks tab. Click the Add button in the toolbar, find my-autoinstall-data.vmdk and select it. Switch to the Optical disks tab and add the Ubuntu server .iso file there, before returning to the main VirtualBox window.

In order to connect to our VM via SSH later, we'll need to set up a virtual "host-only" network. Click File Host Network Manager... and click Create. Make a note of the network name, IP address and network mask (in my case, it's called vboxnet0 and has the IP range 192.168.56.1/24) and make sure DHCP Server is Enabled before returning to the main VirtualBox window.

Assembling the Virtual Machine

Create a new Ubuntu (64-bit) virtual machine with the wizard. For this tutorial, our VM is called valleylodge, has 2 GB of memory and a 20 GB disk. When the wizard finishes, right-click on your VM, select Settings... and make the following changes before clicking OK:

  1. Under the System tab:
    • Un-check Floppy and check Enable EFI (special OSes only).
  2. Under the Storage tab:
    • Right-click on Controller: IDE and select Remove Controller.
    • Click on Controller: SATA, click the "add optical drive" icon and choose your Ubuntu server .iso disk image.
    • Click on Controller: SATA, click the "add hard disk" icon and choose your my-autoinstall-data.vmdk disk image.
    • The Storage tab should look something like this when you're done. The first disk is our installation target, the second is the Ubuntu server install disk and the third is the disk image where our autoinstall config will live.
  3. Under the Network tab:
    • Click Adapter 2, enable it, attach it to a Host-only Network and select the one you created earlier (vboxnet0).

The VM is pretty much ready to boot and all we need now is our user-data file, but since what we're testing start from first boot, let's take a snapshot of our VM before we go any further. With your VM selected on the left, click Snapshot Take... and give it the name pre-install. (We'll come back to this later.)

If you launch the VM with the Start button now, the VM would boot from the .iso and in a minute or so you'd be looking at the first installation screen. But we're here to automate that, so let's do that.

Autoinstall and Cloud-init Data

The autoinstall info should go in a YAML file called user-data (with no file extension) in the disk labeled "CIDATA" mounted on the Desktop. The first line of the file (as well as the first line of the user-data: section of it) should read #cloud-config. There's a lot of [documentation on autoinstall syntax][], but it's still coming together (e.g., the autoinstall-editor tool is not yet out and stable as of this writing), so I'll make this easy by giving you a whole file and then pointing out a few highlights in the comments.

#cloud-config
autoinstall:
  version: 1
  locale: en_US
  refresh-installer:
    update: yes # Check for updated installer
  network: # This is the section header for autoinstall
    network: # This is the netplan root. Both MUST be present (this took forever to solve)
      version: 2
      ethernets:
      enp0s3:
          dhcp4: true
      enp0s8:
          dhcp4: true
  storage: # partition the hard drives using curtin
    version: 1
    layout:
      name: direct
    grub:
      install_devices:
        - ssd0-boot
    swap:
      size: 0
    config:
      # goals: ESP + boot + root
      - id: ssd0 # select raw disk (block device, like sda)
        type: disk
        match:
          size: largest # select highest-capacity block device
        ptable: gpt
        wipe: superblock
      - id: ssd0-esp # create partitions on disk (like sda1)
        type: partition
        device: ssd0
        size: 512MB
        flag: boot # EFI system partition needs boot flag
      - id: ssd0-boot
        type: partition
        device: ssd0
        size: 512MB
      - id: ssd0-root
        type: partition
        device: ssd0
        size: -1 # use the rest of the disk
      - id: ssd0-esp-fs # format partitions on disk
        type: format
        volume: ssd0-esp
        fstype: fat32
        label: ESP
      - id: ssd0-boot-fs
        type: format
        volume: ssd0-boot
        fstype: ext4
        label: BOOT
      - id: ssd0-root-fs
        type: format
        volume: ssd0-root
        fstype: xfs
        label: ROOT
      - id: ssd0-esp-mount # mount partitions
        type: mount
        device: ssd0-esp-fs
        path: /boot/efi
      - id: ssd0-boot-mount
        type: mount
        device: ssd0-boot-fs
        path: "/boot"
      - id: ssd0-root-mount
        type: mount
        device: ssd0-root-fs
        path: "/"
  ssh:
    install-server: yes
    allow-pw: no
  user-data: # cloud-config data goes under this heading
    #cloud-config
    timezone: Etc/UTC
    locale: en_US.UTF-8
    hostname: valleylodge
    groups:
      - coven
    users:
      - name: torgo
        gecos: "I am Torgo; I take care of the place while the Master is away"
        primary_group: coven
        groups: sudo
        lock-passwd: false
        passwd: "$6$warren$nTXw... <snip> ...Ax4TKdx8q/" # shadow password hash
        shell: /bin/bash
        ssh_authorized_keys:
          - "ssh-rsa AAAA... <snip> ...R4hhMq0LoQ== Torgo's SSH Key"
        sudo: "ALL=(ALL) NOPASSWD:ALL"
        uid: 9000
    packages:
      - build-essential

Near the end of the above block, I've put a little "... ..." in where you would put your user's secure shadow password hash and SSH key.

Takeoff and Test Flight

Because we took a snapshot above, we can now boot the VM to test our autoinstall configuration and if it doesn't work, we can kill the VM, restore the snapshot, make changes to the autoinstall file and then boot the VM again to test it.

Once you launch your VM, it'll boot into the installation, but the only question you should be asked is Continue with autoinstall? (yes/no) If you're ready, type "yes" and press Return. When it's all done, the machine will reboot and the user-data: section of the user-data file will be processed. This might take until after the login prompt appears, but just give it a moment to finish (the last message should be "Reached target Cloud-init target".)

Safely shut down the VM (make sure not to restore your previous snapshot), go into its Settings and under Storage, click on the Ubuntu server install disk. To the right, under its Attributes, click the small CD icon next to the SATA port identifier and choose Remove Disk from Virtual Drive to "eject" the install disk. Then, right-click on my-autoinstall-data.vmdk and select Remove Attachment before clicking OK. This time when you boot without the CIDATA disk or the Ubuntu server install disk, it should boot to a prompt

STILL WORKING ON THIS PART


Copyright © 2020 by RD. License: CC BY-NC-SA 4.0.

@Miesvanderlippe

This comment has been minimized.

Copy link

@Miesvanderlippe Miesvanderlippe commented May 18, 2020

I've written a small hack to enable truly automated installs using this method (bypassing the continue with install question);
https://gist.github.com/Miesvanderlippe/d195df2d9755b5a9c48a4ae551e53ce5

Can you confirm your user is created correctly? I'm having issues getting the identity section to work, but user-data seems to at least create a user. I'm wondering if your sudo etc. works.

Edit: It works. Finally.

@mohag

This comment has been minimized.

Copy link

@mohag mohag commented Jun 14, 2020

I'm also struggling with the user - I tried an identity section instead, neither seems to work... (Logged in single-user mode to check)

I am using an extra CD-ROM drive with an ISO image fro the user-data, which doe snot exject before the first boot of the installed OS, if that is relevant though... (I'm on VMWare ESXi / vSphere) (For the iso images, cloud-localds from the cloud-init-tools works to generate them)

(I have found that the cloud-init file is used on the first boot as well - that might be a factor - I added a runcmd to the top level and that gets executed on the first boot...)

(With it automatically rebooting, removing the cidata before the first boot disk seems a bit hard...)

It seems like a user-data file is generated into /var/lib/cloud/seed/nocloud-net/user-data, but it is not used if the CD is still mounted. I got it to work by interrupting the boot and then disconnecting the CD drive before the first boot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment