GitLab EE/CE V4 API: Erase all jobs for project

erase_all_jobs.sh

#!/usr/bin/env bash
# Author: Thomas Goodwin
# Company: Geon Technologies
# Purpose: Erase all jobs for a given GitLab project.
# Arguments:
#    1) URL of gitlab project (https://some.server.com/group/[group/]project)
#    2) private token (personal access token, impersonation token, etc.)
#
# Requirements: bash > 4.0, jq

# Parse the first argument into the server and group-project paths
GITLAB_PROJECT=${1#*//*/}
GITLAB_SERVER=${1%${GITLAB_PROJECT}}
GITLAB_SERVER=${GITLAB_SERVER%%/}
TOKEN=${2}

if [ -z "${GITLAB_SERVER}" ] || [ -z "${GITLAB_PROJECT}" ]; then
    echo "Unable to resolve the server vs. project address from: $1"
    exit 1
fi

if [[ -z "${TOKEN}" ]]; then
    echo "You must provide the private token for this operation"
    exit 1
fi

cat <<EOF
Configuration
    Server:  ${GITLAB_SERVER}
    Project: ${GITLAB_PROJECT}
    Token:   $(echo ${TOKEN} | sed 's/./X/g')
EOF

# The group/[group/]project has its slashes replaced with %2f as an
# alternate for project ID in the GitLab API.
GITLAB_PROJECT=${GITLAB_PROJECT//'/'/'%2F'}
GITLAB_API="${GITLAB_SERVER}/api/v4"

# Procedure:
# 1) Retrieve a list of all jobs, filter for id, store as bash array
# 2) Loop over each job ID and delete it
ALL_JOBS=$(curl -X GET \
    --silent --fail --show-error \
    --header "PRIVATE-TOKEN: ${TOKEN}" \
    "${GITLAB_API}/projects/${GITLAB_PROJECT}/jobs" \
    )

function call_api() {
    echo "Calling '${2}' for job ${1}"
    RESPONSE=$(curl -X POST \
        --silent --fail --show-error \
        --header "PRIVATE-TOKEN: ${TOKEN}" \
        "${GITLAB_API}/projects/${GITLAB_PROJECT}/jobs/${1}/${2}" \
        )
}

ALL_JOBS=$(echo "${ALL_JOBS}" | jq -r '.[] | "\(.id) \(.status)"')
while read job status
do
    case "$status" in
        created)
            ;&
        pending)
            ;&
        running)
            call_api ${job} cancel
            ;;
        failed)
            ;&
        success)
            ;&
        manual)
            call_api ${job} erase
            ;;
        *)
            echo "Skipping job ${job} with status '${status}'"
            ;;
    esac
done <<< ${ALL_JOBS}

For anyone using this, it was an attempt at maybe addressing part of this issue. The thought I had was that if you're erasing a job's artifacts and trace, what would be the point of keeping the job record around (and especially its failed status)? I could clean the pipeline statistics to get around the issue of having no DELETE endpoint in their API to touch (and not wanting to muck around in the database).

Apparently, there's some value to retaining the job reference even if all we did was gut it (don't ask me...). And unfortunately, the returned job structure does not get updated to indicate it has been erased so we don't have a way to skip that previously-erased job on subsequent runs (so far as I can tell). The net impact is that if you know your private token has permission to erase, and you still get 403 errors, it's probably because the job was previously erased.

Not works (

./purge.sh https://gitlab.XXX.net/gcu/caching-system XXXXXXXXXXXXXXXXXXXX
Configuration
    Server:  https://gitlab.XXX.net
    Project: gcu/caching-system
    Token:   XXXXXXXXXXXXXXXXXXXX
Skipping job 6 with status 'skipped'
Skipping job 5 with status 'skipped'
Calling 'erase' for job 4
curl: (22) The requested URL returned error: 403 Forbidden
Skipping job 3 with status 'skipped'
Skipping job 2 with status 'skipped'
Calling 'erase' for job 1
curl: (22) The requested URL returned error: 403 Forbidden

root@GitLab:~#