Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
How to setup nested virtualization on Azure for Linux Containers on Windows

The following describes how I got Linux Containers on Windows running on an Azure VM as Stefan Scherer's Vagrant box (https://github.com/StefanScherer/insider-docker-machine#lcow---linux-container-on-windows) didn't work for me (probably because I don't know enough about Vagrant). As this was my first try with nested virtualization, I might very well have done something stupid, but it works :) Here is what the result looks like: https://www.axians-infoma.de/wp-content/uploads/2017/09/lcow_busybox.gif.gif

Setup host

  1. Create a Windows Server 2016 Datacenter - with Containers Azure VM
  2. Choose one of the Dx_v3 or Ex_v3 machine sizes, as those have nested virtualization enabled. I used a D4S_V3 machine but D2S_V3 should work as well
  3. Connect to the machine with RDP
  4. Disable Enhanced Security for IE and install Chrome - strictly optional, I just don't like IE that much to say it politely
  5. Download the Windows Server 2016 Insider Preview iso from https://www.microsoft.com/en-us/software-download/windowsinsiderpreviewserver (if you haven't already, you need to register at https://insider.windows.com) and put it into the Azure VM. I'll be using c:\nv_lcow but you can use something else if you want to
  6. Open a PowerShell as admin and run the following command to install the Hyper-V feature
Install-WindowsFeature Hyper-V -IncludeManagementTools
  1. Restart the machine manually or through PowerShell
Restart-Computer

Setup the vm with nested virtualization

  1. Open PowerShell as admin and run the following commands to create the necessary networking stuff, you'll have to confirm in between
New-VMSwitch -SwitchName "NatSwitch" -SwitchType Internal
New-NetIPAddress -IPAddress 192.168.0.1 -PrefixLength 24 -InterfaceIndex ( Get-NetAdapter | Where-Object { $_.Name.Contains("NatSwitch") } | select -ExpandProperty ifIndex)
Remove-NetNat
New-NetNat -Name NatNetwork -InternalIPInterfaceAddressPrefix 192.168.0.0/24
  1. Create a Virtual Machine with the Insider Preview, attach the DVD iso, enable nested virtualization and start it. I name it NV_LCOW
New-VM -Name NV_LCOW -MemoryStartupBytes 4GB -SwitchName NatSwitch -NewVHDPath NV_LCOW.vhdx -NewVHDSizeBytes 50000000000
Set-VMDvdDrive -VMName NV_LCOW -ControllerNumber 1 -Path C:\nv_lcow\Windows_InsiderPreview_Server_16278.iso
Set-VMProcessor -VMName NV_LCOW -ExposeVirtualizationExtensions $true
Start-VM -Name NV_LCOW
  1. Open the VM by selecting it in the Hyper-V Manager and clicking on "Connect..." on the right
  2. Start the install. Select "Windows Server Datacenter" and "Install Windows online" when you get asked
  3. Now you might want to grab a coffee as this is going to take a while
  4. When it's done set up an IP adress with a default gateway and set a DNS server
New-NetIPAddress -InterfaceIndex ( Get-NetAdapter | select -ExpandProperty ifIndex ) -IPAddress 192.168.0.2 -PrefixLength 24 -DefaultGateway 192.168.0.1
Set-DnsClientServerAddress -InterfaceIndex ( Get-NetAdapter | select -ExpandProperty ifIndex ) -ServerAddresses "8.8.8.8"
  1. Make sure this works by pinging an external address, e.g. the following
ping www.google.com

Set up Hyper-V containers in the Hyper-V VM (in the Azure VM ;) )

  1. Start powershell by just entering the following in the VM cmd
powershell
  1. In powershell, start the Hyper-V install (no management tools as we don't have a GUI) and restart the VM
Install-WindowsFeature Hyper-V
Restart-Computer
  1. Install Docker
Install-PackageProvider -Name NuGet -MinimumVersion 2.8.5.201 -Force
Install-Module -Name DockerMsftProvider -Force
Install-Package -Name docker -ProviderName DockerMsftProvider -Force
  1. Remove the Docker service as I won't use it and restart the machine
(Get-WmiObject -Class Win32_Service -Filter "Name='Docker'").delete()
Restart-Computer

Setup the latest Docker binaries and start the Linux Container

  1. Download docker, dockerd and linuxkit and change acl
Invoke-WebRequest -UseBasicParsing -OutFile "C:\program files\docker\dockerd.exe" https://master.dockerproject.org/windows/x86_64/dockerd.exe
Invoke-WebRequest -UseBasicParsing -OutFile "C:\program files\docker\docker.exe" https://master.dockerproject.org/windows/x86_64/docker.exe
mkdir "$Env:ProgramFiles\Linux Containers”
Invoke-WebRequest -UseBasicParsing -OutFile linuxkit.zip https://github.com/friism/linuxkit/releases/download/preview-1/linuxkit.zip
Expand-Archive linuxkit.zip -DestinationPath "$Env:ProgramFiles\Linux Containers\."
rm linuxkit.zip
$acl = Get-Acl -Path "$Env:ProgramFiles\Linux Containers"
$vmGroupRule = new-object System.Security.AccessControl.FileSystemAccessRule("NT VIRTUAL MACHINE\Virtual Machines", "FullControl","ContainerInherit,ObjectInherit", "None", "Allow")
$acl.SetAccessRule($vmGroupRule)
Set-Acl -AclObject $acl -Path "$Env:ProgramFiles\Linux Containers”
$acl = Get-Acl -Path "c:\lcow"
$vmGroupRule = new-object System.Security.AccessControl.FileSystemAccessRule("NT VIRTUAL MACHINE\Virtual Machines", "FullControl","ContainerInherit,ObjectInherit", "None", "Allow")
$acl.SetAccessRule($vmGroupRule)
Set-Acl -AclObject $acl -Path "c:\lcow"
  1. Run the daemon in the foreground to make sure this works
$Env:LCOW_SUPPORTED=1
& "C:\Program Files\docker\dockerd.exe" --experimental --data-root c:\lcow
  1. This should end with something like
API listen on //./pipe/docker_engine

If not, you might want to run dockerd with param -D to get debug info 4. Press Ctrl-C to stop it now run it in the background

Start-Job { & "C:\Program Files\docker\dockerd.exe" --experimental --data-root c:\lcow }
  1. Do something like docker version or docker info to make sure it actually is running
  2. Run your Linux container!
docker run --rm -ti busybox

This should result in something like this (I got some weird errors about docker not being able to register layers, so I did Set-ItemProperty -Path 'HKLM:SOFTWARE\Microsoft\Windows NT\CurrentVersion\Virtualization\Containers' -Name VSmbDisableOplocks -Type DWord -Value 1 -Force as suggested by https://github.com/docker/kitematic/wiki/Common-Issues-and-Fixes#windows-processbaselayer-error and then rebooted):

PS C:\Users\Administrator> docker run -ti --rm busybox
/ # uname -a
Linux 8a018606a2a8 4.11.1-linuxkit #1 SMP Thu Sep 7 16:48:33 UTC 2017 x86_64 GNU/Linux

What's next

I then tried to run a SQL Server on Linux but that complained about not enough memory. I restarted with -m 4g but still cat /proc/meminfo | grep MemTotal returns only 1GB, so I guess this is not yet working with LCOW

Useful links where I got most of the information

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.