cleanup_tags.md

Problem

Lets say you have Gitlab CI defined that generates docker images on merge request. This is great because it lets people review your work in a consistent environment. The problem is all those image tags from branches hang around in the registry and clutter things up. Wouldn't it be great if you could remove those tags as apart of the CI when you merge to master?

Solution

When merging a branch to gitlab it automatically provides a merge title "Merge branch 'branch-name' into 'master'." During the master CI pipeline we can parse out the branch name from the commit and use that to remove any tagged images in the form gitlab.kitware.com:4567/group/project/container:branch-name.

There are a few sticking points we have to negotiate. In order to use a reasonable tag name when generating the merge request image tag we use the $CI_COMMIT_REF_SLUG which is a url friendly conversion of the branch name (e.g. "dev/feature/branch-name" becomes "def-feature-branch-name"). The script parses out 'dev/feature/branch-name' from the commit message and transforms it in something that approximates the Gitlab ruby function. Once we have the correct tag name, we use the Gitlab registry API to identify all images that are tagged with that name.

Unfortunately we can't just remove those tags because of a long standing Gitlab/Docker issue where all tags related to a particular image are removed. This means if 'dev-feature-branch-name' and 'latest' point to the same image id, then 'latest' will be removed along with 'dev-feature-branch-name' (gross). To avoid this problem we push a "dud" image (i.e. alpine:latest) to the tag we're looking to remove, then remove the tag using the gitlab API.

Minimum working example .gitlab-ci

Note that you'll have to change some of the tags here to be specific to your project (e.g. not chris.kotfila/gitlabci-experiment/container)

branch_run:
  script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - docker build -t $CI_REGISTRY/chris.kotfila/gitlabci-experiment/container:$CI_COMMIT_REF_SLUG .
    - docker push $CI_REGISTRY/chris.kotfila/gitlabci-experiment/container:$CI_COMMIT_REF_SLUG 
  only:
    - merge_request

master_run:
  script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - docker build -t $CI_REGISTRY/chris.kotfila/gitlabci-experiment/container:latest .
    - docker push $CI_REGISTRY/chris.kotfila/gitlabci-experiment/container:latest 
    - ./cleanup_tags.sh
  only:
    - master

The cleanup_tags.sh script

#!/bin/bash
set -e

# Note: PRIVATE_API_TOKEN must be set in the CI variables and available in the
# environment for this script to run. It should be an Access Token with the
# 'api' scope of a user who has read/write access to the project registry.

PAT="Merge branch '(.*)' into 'master'"
BRANCH_NAME="NONE"
# Note: uncomment to enable debug messages
# DEBUG=1

function debug {
    if [[ ! -z "$DEBUG" ]]; then
        echo -e "[[ DEBUG ]] - $1"
    fi
}

# Note: Here we 'slugify' the branch name so it is consistent with
# $CI_COMMIT_REF_SLUG. From GitLab Predefined Variables documentation:
#
# $CI_COMMIT_REF_NAME lowercased, shortened to 63 bytes, and with everything
# except 0-9 and a-z replaced with -. No leading / trailing -. Use in URLs, host
# names and domain names.
if [[ $CI_COMMIT_TITLE =~ $PAT ]]; then
    # Lowercase branch name
    BRANCH_NAME="$( echo "${BASH_REMATCH[1]}" | tr "[:upper:]" "[:lower:]")"

    # shorten to 63 bytes
    if [[ ${#BRANCH_NAME} > 63 ]]; then
        BRANCH_NAME="${BRANCH_NAME::63}"
    fi
    # Replace non alnum with "-" and remove leading/trailing "-"
    BRANCH_NAME="$(echo $BRANCH_NAME | \
                                              sed 's/[^[:alnum:]]/-/g' | \
                          sed 's/^-//' | sed 's/-$//')"
fi

# Ignore commits that don't match $PAT
# (e.g. "Merge branch 'branch-name' into 'master'")
if [[ "$BRANCH_NAME" == "NONE" ]]; then
    exit 0;
else
    echo "Trying to clean up containers tagged: $BRANCH_NAME"
fi

debug "Project ID: $CI_PROJECT_ID"

REGISTRY_IDS=$(curl -s -XGET \
     --header "PRIVATE-TOKEN: $PRIVATE_API_TOKEN" \
     "https://gitlab.kitware.com/api/v4/projects/$CI_PROJECT_ID/registry/repositories" | jq ".[].id")

debug "Registry Ids:\n$REGISTRY_IDS"

docker pull alpine:latest > /dev/null 2>&1

while read -r REGISTRY_ID; do
    REMOTE_REPO_URL="https://gitlab.kitware.com/api/v4/projects/$CI_PROJECT_ID/registry/repositories/$REGISTRY_ID/tags"

    debug "Looking for remote repo tags at: $REMOTE_REPO_URL"

    REMOTE_CURL=$(curl -s -XGET --header "PRIVATE-TOKEN: $PRIVATE_API_TOKEN" "$REMOTE_REPO_URL")
    debug "Remote curl response: $REMOTE_CURL"

    REMOTE_TAG=$(echo "$REMOTE_CURL" | jq -r ".[] |  select(.name == \"$BRANCH_NAME\").location")
    debug "Remote Tag: $REMOTE_TAG"

    if [[ -z "$REMOTE_TAG" ]]; then
        echo "Could not find '$BRANCH_NAME' tag at $REMOTE_REPO_URL"
    else
        # Note: Docker/Gitlab will remove all tags related to a particular
        # image, if the $BRANCH_NAME tag is the same as the :latest tag then
        # :latest will be removed also, recommended workaround is to push a
        # "dud" image to the tag and then remove it. It's ugly, but the problem
        # is upstream (womp womp) - @kotfic
        # See:
        #   https://gitlab.com/gitlab-org/gitlab-foss/issues/20737
        #   https://gitlab.com/gitlab-org/gitlab-foss/issues/21405
        debug "tagging alpine:latest as $REMOTE_TAG"
        docker tag alpine:latest "$REMOTE_TAG" > /dev/null 2>&1

        debug "Pushing \"dud\" $REMOTE_TAG for deletion"
        docker push $REMOTE_TAG > /dev/null 2>&1

        DELETE_URL="https://gitlab.kitware.com/api/v4/projects/$CI_PROJECT_ID/registry/repositories/$REGISTRY_ID/tags/$BRANCH_NAME"

        echo "Removing: $REMOTE_TAG"
        debug "Deleting - $DELETE_URL"
        curl -s -XDELETE --header "PRIVATE-TOKEN: $PRIVATE_API_TOKEN" $DELETE_URL > /dev/null 2>&1
    fi
done <<< "$REGISTRY_IDS"