Create a Virtual Machine Scaleset for Azure Pipelines

create-vmss.md

Creating an Azure DevOps Scale Set Agent Pool

Find SKUs

To create a VM, you need an image. This can be found by using az vm image list. Some examples below:

`az vm image list -p MicrosoftWindowsServer -f windowsserver -s datacenter-core-2004 --all

`az vm image list -p MicrosoftWindowsServer -f windowsserver -s datacenter-core-20H2 --all

`az vm image list -p MicrosoftWindowsServer -f windowsserver -s 2019-datacenter-core --all

For Linux:

az vm image list -p Canonical

Sample output from above:

[
    {
    "offer": "WindowsServer",
    "publisher": "MicrosoftWindowsServer",
    "sku": "datacenter-core-2004-with-containers-smalldisk-g2",
    "urn": "MicrosoftWindowsServer:WindowsServer:datacenter-core-2004-with-containers-smalldisk-g2:19041.450.2008080726",
    "version": "19041.450.2008080726"
  },
  {
    "offer": "WindowsServer",
    "publisher": "MicrosoftWindowsServer",
    "sku": "datacenter-core-2004-with-containers-smalldisk-g2",
    "urn": "MicrosoftWindowsServer:WindowsServer:datacenter-core-2004-with-containers-smalldisk-g2:19041.508.2009070256",
    "version": "19041.508.2009070256"
  },
  {
    "offer": "WindowsServer",
    "publisher": "MicrosoftWindowsServer",
    "sku": "datacenter-core-2004-with-containers-smalldisk-g2",
    "urn": "MicrosoftWindowsServer:WindowsServer:datacenter-core-2004-with-containers-smalldisk-g2:19041.572.2010091946",
    "version": "19041.572.2010091946"
  }
]

From the above output, you need to retrieve the urn for the desired image. The last portion of the urn (version) can be replaced with :latest

Setting up some variables

• urn: fully qualified name of the image to use for VM creation

  $urn = "MicrosoftWindowsServer:WindowsServer:datacenter-core-2004-with-containers-smalldisk-g2:latest"

• imageResourceGroup: this is where we store all VM images

  $imageResourceGroup = "DevOps-Images"

• agentResourceGroup: this is where the vmss (Azure DevOps agent scale sets) are created

  $agentResourceGroup = "DevOps-Agents"

• vmName: should be set to a short name to identy, for example, the version. This will be used to create other resource names

  $vmName = "ltsc2019"

Create VM (change SKU based on needs)

This document uses Windows Server 2019 v2004 as an example.

Windows:

az vm create -g $imageResourceGroup -n "$vmName-vm" --image $urn --os-disk-size-gb 500 --admin-username sitecore --admin-password Sup3rS3cret! --size Standard_D16s_v3

Linux:

az vm create -g $imageResourceGroup -n "$vmName-vm" --image $urn --os-disk-size-gb 500 --generate-ssh-keys --size Standard_D8s_v3

Customize the created VM

1. Prepare the VM - Execute remote script on VM. The vm-script.ps1 and vm-script.sh need to be executed on the remote machine. The script will (slightly differs between Linux and Windows):

   • Expand the disk to maximum partition size
   • Create a new user (AzDevOps) and add to Administrator's group
   • Install docker-compose
   • Add "C:\Program Files\dotnet" to PATH
   • Install az cli
   • Restart the Computer

   To run the script:

   • Windows:

     az vm run-command invoke  --command-id RunPowerShellScript --name "$vmName-vm" -g $imageResourceGroup --scripts @vm-script.ps1

   • Linux

     az vm run-command invoke  --command-id RunShellScript --name "$vmName-vm" -g $imageResourceGroup --scripts @vm-script.sh

2. Get the IP address of the newly created VM

$ipAddress = $(az vm show -d -g $imageResourceGroup -n "$vmName-vm"  --query publicIps -o tsv)

1. Sysprep the VM - Remote Desktop into the VM

   • Windows Remote Desktop to the machine and execute:

   C:\Windows\System32\sysprep\sysprep.exe /generalize /oobe /shutdown

   • Linux

     sudo apt-get update -y
     sudo apt-get upgrade -y
     sudo waagent -deprovision+user -force --verbose

   Wait for the machine to be in "Stopped" state in the portal before proceeding to the next step

Create the Image

1. Deallocate and Generalize VM

   az vm deallocate --resource-group $imageResourceGroup --name "$vmName-vm"
   az vm generalize --resource-group $imageResourceGroup --name "$vmName-vm"
   

2. Get the ID for the VM

   $vmId = ((az vm show -g $imageResourceGroup -n "$vmName-vm") |ConvertFrom-Json).id
   

3. Create image from VM

   az image create  --resource-group $imageResourceGroup --name "$vmName-image" --source $vmId --hyper-v-gen v2
   
   #Get the Id
   
   $imgId = ((az image show --resource-group $imageResourceGroup --name "$vmName-image") |ConvertFrom-Json).id
   

Create a new VMSS pool (or update existing)

Create a new VMSS

• Set a variable with scale set name for easy use

$scaleSetName = "vmss-${vmName}"

• Create VMSS

az vmss create --resource-group $agentResourceGroup --name $scaleSetName --image $imgId --admin-username sitecore --admin-password ChooseY0urP@ssword! --instance-count 1 --disable-overprovision --upgrade-policy-mode manual --load-balancer '""' --vm-sku Standard_D16s_v3

MODIFY an existing VMSS

If you've created a new VM image and want to replace the one the VMSS is using, you can modify it, instead of deleting and recreating. Note that this will update the model of the VMSS but not change any of the running instances.

az vmss update --resource-group $agentResourceGroup --name $scaleSetName --set virtualMachineProfile.storageProfile.imageReference.id=$imgId

If you've updated an existing VMSS with a new image, you can force an update of existing instances. You can also delete all running instances from the portal and new ones will be created.

az vmss update-instances --instance-ids * --name $scaleSetName --resource-group $agentResourceGroup

DONE

At this point you can browse to the Azure DevOps portal and create a new agent pool connected to the VMSS!

vm-script.ps1

#   - Expand the disk to maximum partition size
#   - Create a new user (`AzDevOps`) and add to Administrator's group
#   - Install docker-compose
#   - Add "C:\Program Files\dotnet" to PATH. The installation of the dotnet core framework is handled in the pipeline
#   - Install az cli
 #  - Restart the Computer

# Increase drive space to max allocated
$MaxSize = (Get-PartitionSupportedSize -DriveLetter c).sizeMax
Resize-Partition -DriveLetter c -Size $MaxSize

# Create new 'AzDevOps' user and add to Local Administrators group
$password = ConvertTo-SecureString "ChooseY0urP@ssword!" -AsPlainText -Force
New-LocalUser "AzDevOps" -Password $password -FullName "Azure DevOps Agent User" -Description "Azure DevOps Agent User"
Add-LocalGroupMember -Group "Administrators" -Member "AzDevOps"

# Install Docker-Compose
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
Invoke-WebRequest "https://github.com/docker/compose/releases/download/1.27.4/docker-compose-Windows-x86_64.exe" -UseBasicParsing -OutFile $Env:ProgramFiles\Docker\docker-compose.exe

# Add dotnet core folder to path (dotnet core gets installed via pipelines)
$newPath = $env:PATH +=  "C:\Program Files\dotnet"
Set-ItemProperty -Path 'Registry::HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment' -Name PATH -Value $newPath

# Install az cli
$ProgressPreference = "SilentlyContinue"
Invoke-WebRequest -Uri https://aka.ms/installazurecliwindows -OutFile .\AzureCLI.msi; Start-Process msiexec.exe -Wait -ArgumentList '/I AzureCLI.msi /quiet'; rm .\AzureCLI.msi
$ProgressPreference = "Continue"

# Restart computer once completed to ensure all changes take effect
Restart-Computer -Force

vm-script.sh

# Install pre-requisite packages.
sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common -y

# Install az cli
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash

# # Create new 'AzDevOps' user and add to Local Administrators group
password='ChooseY0urP@ssword!'
username='AzDevOps'
pass=$(perl -e 'print crypt($ARGV[0], "password")' $password)
sudo useradd -m -p "$pass" "$username"

# Install powershell
# Update the list of packages
sudo apt-get update -y

# Download the Microsoft repository GPG keys
wget -q https://packages.microsoft.com/config/ubuntu/18.04/packages-microsoft-prod.deb
# Register the Microsoft repository GPG keys
sudo dpkg -i packages-microsoft-prod.deb
# Update the list of products
sudo apt-get update -y
# Enable the "universe" repositories
sudo add-apt-repository universe
# Install PowerShell
sudo apt-get install -y powershell

# Install Docker
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo apt-key fingerprint 0EBFCD88
sudo add-apt-repository \
  "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) \
  stable"

sudo apt-get update -y
sudo apt-get install docker-ce docker-ce-cli containerd.io -y

# Install Docker-compose
sudo curl -L "https://github.com/docker/compose/releases/download/1.27.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

sudo usermod -a -G docker $username

sudo fallocate -l 6G /swap.img
sudo chmod 600 /swap.img
sudo mkswap /swap.img
sudo swapon /swap.img

sudo reboot