Skip infrastructure deployments if there are no changes to deploy (Azure Pipelines)

_Readme.md

Skip infrastructure deployments if there are no changes to deploy (Azure Pipelines)

⚠️ Update 2024: For anyone looking at this gist in 2024, we have moved on from using the Pipelines API to instead using tags. Each time we successfully deploy to an environment, we tag the commit with the environment name. When testing for changes, we then compare the current commit being deployed to the tagged commit for the same environment. This has proven to be a bit more robust than using the changes API. The principles are the same though. Let me know in a comment if it's not obvious how to adapt the script.

One devops strategy is to keep infrastructure state and code in a single repository, and deploy them together. It ensures your infrastructure is always in the correct state to match the version of your code.

The downside is that infrastructure deployments are slow, and generally don't change as frequently as code.

This gist shows how to skip an infrastructure deployment task if there are no changes in a particular sub-path of the repository.

It takes advantage of the pipelines API, which can provide a list of all commits since the last successful pipeline execution. This ensures that it works even when multiple commits are pushed at once, or when the build with the infrastructure change failed. The pipelines API does the hard work of working out which commits need checking.

azure-pipelines.yaml

resources:
  repositories:
    - repository: self
      type: git
      ref: main

stages:
  - stage: Build
    pool:
      vmImage: ubuntu-latest
    jobs:
      - job: BuildJob
        displayName: Build Job
        steps:
          - checkout: self
            clean: true
          - task: PowerShell@2
            inputs:
              filePath: "azure-pipelines/Test-ChangesMadeInPath.ps1"
              arguments: >-
                -authorisation "Bearer $(system.accesstoken)" 
                -pathFilter "azure-pipelines/deployment" 
                -buildId $(Build.BuildId)
                -collectionUri $(System.CollectionUri)
                -project $(System.TeamProject)
            name: DetermineChangesMadeInDeploymentFolder
            env:
              SYSTEM_ACCESSTOKEN: $(system.accesstoken)
          - task: DotNetCoreCLI@2
            displayName: Build project
            inputs:
              projects: "**/*.csproj"
              arguments: --output publish_output --configuration Release
          - task: PublishBuildArtifacts@1
            displayName: "Publish Artifact: drop"

  - stage: Deploy
    pool:
      vmImage: ubuntu-latest
    condition: and(succeeded(), eq(variables['build.sourceBranch'], 'refs/heads/main'))
    jobs:
      - deployment: DeployInfrastructure
        condition: eq(stageDependencies.Build.BuildJob.outputs['DetermineChangesMadeInDeploymentFolder.filesUpdated'], 'True')
        displayName: Deploy infrastructure
        environment: "prod"
        strategy:
          runOnce:
            deploy:
              steps:
                - template: deployment/YAML/deploy-infrastructure.yml
                  parameters:
                    environment: $(Environment.Name)
      - deployment: DeployCode
        displayName: Deploy code
        dependsOn: DeployInfrastructure
        condition: in(dependencies.DeployInfrastructure.result, 'Succeeded', 'Skipped')
        environment: "prod"
        strategy:
          runOnce:
            deploy:
              steps:
                - template: deployment/YAML/deploy-code.yml
                  parameters:
                    environment: $(Environment.Name)

Test-ChangesMadeInPath.ps1

[CmdletBinding()]
param (
  $authorisation,
  $pathFilter,
  $collectionUri,
  $project,
  $buildId
)

$changesUrl = "$collectionUri/$project/_apis/build/builds/$buildId/changes?api-version=6.0"
$changesResponse = Invoke-RestMethod -Uri $changesUrl -Headers @{Authorization = $authorisation } -Method Get
$commits = @($changesResponse.value | ForEach-Object { $_.id })

Write-Host "##vso[task.setvariable variable=filesUpdated;isOutput=true]False"
Write-Host "Checking $($commits.Length) commits for changes matching path $pathFilter"
for ($j = 0; $j -lt $commits.Length; $j++) {
  Write-Host "Checking commit: $($commits[$j]) with its parent"
  $files = $(git diff "$($commits[$j])~" $commits[$j] --name-only)
  Write-Host $files
  if ($files -like "*$pathFilter/*") {
    Write-Host "Found file matching path filter in commit $($commits[$j])"
    Write-Host "##vso[task.setvariable variable=filesUpdated;isOutput=true]True"
    break
  }
}

https://gist.github.com/Myrddraal/f5a84cf3242e3b3804fa727005ed2786#file-azure-pipelines-yaml-L23:
There is an extra ' in the end of this line.

https://gist.github.com/Myrddraal/f5a84cf3242e3b3804fa727005ed2786#file-azure-pipelines-yaml-L23: There is an extra ' in the end of this line.

Thanks, fixed!

Thanks for the gist. It's working great.
In case if anyone's facing fatal: ambiguous argument '<commit_hash>~': unknown revision or path not in the working tree., add below in the checkout step of the pipeline (reference - https://stackoverflow.com/questions/75055130/resolving-git-diff-error-in-azure-devops-pipeline)

- checkout: self
  clean: true
  fetchDepth: 0

Hi Myrddraal, this sounds like a very interesting solution! As I'm quite new to Azure pipelines unfortunately for me it isn't too obvious how to adapt the script in order to replace the API interaction with tags. What would be the Environment.Name in this scenario?
Is it always the same for a certain repo? Then wouldn't I have multiple commits with the same tag?
Thanks in advance!

Hi Elisabeth,
(EDIT replying with my work account)
I'm afraid I don't have a huge amount of time to write this up properly, but hopefully the following will help:

Our Test-ChangesMadeInPath powershell script now looks like this:

[CmdletBinding()]
param (
  [String] $pathFilter,
  [String] $tag,
  [String] $stageToBeDeployed,
  [String] $defaultValue
)

Write-Host "##vso[task.setvariable variable=Deploy${stageToBeDeployed};isOutput=true]${defaultValue}";
Write-Host "Checking diff to tag ${tag} for changes. Setting variable name: Deploy${stageToBeDeployed} with default value: ${defaultValue}.";

if (git tag -l $tag) {
  $files = $(git diff $tag --name-only)
  $pathFilters = $pathFilter.Split(",")
  Write-Host ($files -replace ' ', "`r`n")
  for ($j = 0; $j -lt $pathFilters.Length; $j++) {
    if ($files -like "*$($pathFilters[$j])*") {
      Write-Host "Found file matching path filter $($pathFilters[$j]) for tag ${tag}";
      Write-Host "##vso[task.setvariable variable=Deploy${stageToBeDeployed};isOutput=true]true";
      Write-Host "##[warning]${stageToBeDeployed} will be deployed to environment ${tag}";
      return
    }
  }

  Write-Host "No files found matching path filters for tag ${tag}";
} else {
  Write-Host "Tag ${tag} does not exist, setting to deploy";
  Write-Host "##vso[task.setvariable variable=Deploy${stageToBeDeployed};isOutput=true]true";
  Write-Host "##[warning]${stageToBeDeployed} will be deployed to environment ${tag}";
  return
}

At the end of a successful deployment, we then do this:

  - script: |
      if [ $(git tag -l ${{parameters.environment}}) ]; then
        git tag -d ${{parameters.environment}}                      # delete the tag for most recent deployment
        git push origin :refs/tags/${{parameters.environment}}      # delete remotely
      fi

      git tag ${{parameters.environment}} $(Build.SourceVersion)  # make a new tag for most recent deployment locally
      git push origin ${{parameters.environment}}                 # push the new local tag to the remote
    workingDirectory: $(Build.SourcesDirectory)

The idea is that you can compare the current git commit to the tagged version to see what changes have happened since the last successful deployment. This has several advantages, not least that you can rollback by deploying a previous deployment, and it will correctly identify if files have changed between what is live and the version you are redeploying.

I'm thinking it probably makes sense to use branches rather than tags but tags are working for us right now and it hasn't been a priority to change.

Let me know if that gives you enough info to work it out!

Hi David, thanks so much for your reply!
I'll try it out, will definitely help :)