evaluate deprecated aks versions with open policy agent

aks-versions-opa-policy.md

Hello, world. v1.15.11 is indeed deprecated by azure kubernetes service, let's not try to build that. This policy can stop you building or for that matter destroying a cluster that's fell off the bottom of azure's N-2 version's sla unintentionally. There's enough in here that you could probably do some pretty cool things with external dependency calls, regardless of aks or azure. We use opa quite a bit, with gatekeeper as a mutating admission controller within our cluster and external in our build and deploy pipelines. You can use this or the concept in general to stop anything crazy from going down when no one is watching. I can't imagine living without it.

Here's a mock cluster tf file

// 1.15.11.tf
resource "azurerm_kubernetes_cluster" "cluster" {
  name                = "cluster"
  resource_group_name = "cluster"
  location            = "uksouth"
  dns_prefix          = "demo"

  kubernetes_version  = "1.15.11"
  tags                = local.common-tags

  lifecycle {
    ignore_changes = [linux_profile[0].ssh_key[0].key_data]
  }

  role_based_access_control {
    enabled = true
    azure_active_directory {
      client_app_id     = data.azuread_application.aad-aks-client.application_id
      server_app_id     = data.azuread_application.aad-aks-server.application_id
      server_app_secret = data.azurerm_key_vault_secret.aad-aks-server.value
    }
  }

  linux_profile {
    admin_username = "cluster_admin"

    ssh_key {
      key_data = file("~/.ssh/id_rsa.pub")
    }
  }

  default_node_pool {
    name            = "aks"
    node_count      = 1
    vm_size         = "Standard_D4s_v3"
    max_pods        = "10"
    os_disk_size_gb = 10
    vnet_subnet_id  = data.azurerm_subnet.demo-cluster.id
  }

  identity {
    type = "SystemAssigned"
  }

  network_profile {
    network_plugin     = "azure"
    dns_service_ip     = "1.1.1.1"
    docker_bridge_cidr = "2.2.2.2/16"
    service_cidr       = "3.3.3.3/24"
    load_balancer_sku  = "Basic"
  }
}

The only thing we care about here is kubernetes_version = "1.15.11". I'd like to make sure no one builds a cluster that's not in support, or in preview OR the inverse I want to make sure the pipeline only builds a certain cluster, take your pick.

I have mocks 1.15.11.tf and 1.19.11.tf and as of this writing 1.19.11 is in support. So we'll write a rego policy to make sure that whatever kubernetes_version is, that it's currently in the supported list but not in preview.

Running this policy, everything checks out against the mocks.

➜  aks-version-rego git:(main) ✗ conftest test --data token --namespace aks.version --policy aks.rego 1.15.11.tf
FAIL - 1.15.11.tf - aks - 1.15.11 is a deprecated aks version

1 test, 0 passed, 0 warnings, 1 failure, 0 exceptions
➜  aks-version-rego git:(main) ✗ conftest test --data token --namespace aks.version --policy aks.rego 1.19.11.tf

1 test, 1 passed, 0 warnings, 0 failures, 0 exceptions

Before I get into unit testing here's the aks.rego

# aks.rego
package aks.version

# repopulate token with az account get-access-token > token/data.json

import input as hcl
import data.subscription as subscription_id
import data.tokenType as token_type
import data.accessToken as access_token

authorisation = sprintf("%v %v", [token_type, access_token])
location = "uksouth"
microsoft_api_endpoint = sprintf("https://management.azure.com/subscriptions/%v/providers/Microsoft.ContainerService/locations/%v/orchestrators?api-version=2019-04-01&resource-type=managedClusters", [subscription_id, location])

headers = {
    "Content-Type": "application/json",
    "Authorization": authorisation,
    "Accept": "application/json"
}

available_versions = http.send(
  {
  "method": "get",
  "url": microsoft_api_endpoint,
  "headers": headers
  }
)

versions[supported] {
     response := available_versions.body[_]
     some version
     not response.orchestrators[version].isPreview
     supported := response.orchestrators[version].orchestratorVersion
}

deny[msg] {
    tf_version := hcl.resource.azurerm_kubernetes_cluster.cluster.kubernetes_version
    not versions[tf_version]
    msg := sprintf("%v is a deprecated aks version", [tf_version])
}

This builds an api call based off a token value that you can choose to pull in dynamically. It then builds a list of aks version currently in service and excludes preview versions. We can run the unit tests policy using json stubs of the hcl above. For the uninitiated conftest verify will look for test- files in the specified dir.

conftest verify -p . --data token
3 tests, 3 passed, 0 warnings, 0 failures, 0 exceptions, 0 skipped

Here's a look at test-aks.rego

# test-ask.rego
package aks.version

test_aks_v1_15_11_is_deprecated {
    deny["1.15.11 is a deprecated aks version"] with input as { "resource": { "azurerm_kubernetes_cluster": { "cluster": { "kubernetes_version": "1.15.11" } } } }  
} 

test_aks_v1_17_11_is_deprecated {
    deny["1.17.11 is a deprecated aks version"] with input as { "resource": { "azurerm_kubernetes_cluster": { "cluster": { "kubernetes_version": "1.17.11" } } } }
}

# This eval to true, so we don't want to deny, or in this case "not deny" 
test_aks_v1_19_11_is_serviceable {
    not deny["1.19.11 is a deprecated aks version"] with input as { "resource": { "azurerm_kubernetes_cluster": { "cluster": { "kubernetes_version": "1.19.11" } } } }
}