Skip to content

Instantly share code, notes, and snippets.

@btgoodwin
Last active March 20, 2019 13:07
Show Gist options
  • Save btgoodwin/2cf70c577fd0338aabff0f127e3c92f1 to your computer and use it in GitHub Desktop.
Save btgoodwin/2cf70c577fd0338aabff0f127e3c92f1 to your computer and use it in GitHub Desktop.
GitLab EE/CE V4 API: Erase all jobs for project
#!/usr/bin/env bash
# Author: Thomas Goodwin
# Company: Geon Technologies
# Purpose: Erase all jobs for a given GitLab project.
# Arguments:
# 1) URL of gitlab project (https://some.server.com/group/[group/]project)
# 2) private token (personal access token, impersonation token, etc.)
#
# Requirements: bash > 4.0, jq
# Parse the first argument into the server and group-project paths
GITLAB_PROJECT=${1#*//*/}
GITLAB_SERVER=${1%${GITLAB_PROJECT}}
GITLAB_SERVER=${GITLAB_SERVER%%/}
TOKEN=${2}
if [ -z "${GITLAB_SERVER}" ] || [ -z "${GITLAB_PROJECT}" ]; then
echo "Unable to resolve the server vs. project address from: $1"
exit 1
fi
if [[ -z "${TOKEN}" ]]; then
echo "You must provide the private token for this operation"
exit 1
fi
cat <<EOF
Configuration
Server: ${GITLAB_SERVER}
Project: ${GITLAB_PROJECT}
Token: $(echo ${TOKEN} | sed 's/./X/g')
EOF
# The group/[group/]project has its slashes replaced with %2f as an
# alternate for project ID in the GitLab API.
GITLAB_PROJECT=${GITLAB_PROJECT//'/'/'%2F'}
GITLAB_API="${GITLAB_SERVER}/api/v4"
# Procedure:
# 1) Retrieve a list of all jobs, filter for id, store as bash array
# 2) Loop over each job ID and delete it
ALL_JOBS=$(curl -X GET \
--silent --fail --show-error \
--header "PRIVATE-TOKEN: ${TOKEN}" \
"${GITLAB_API}/projects/${GITLAB_PROJECT}/jobs" \
)
function call_api() {
echo "Calling '${2}' for job ${1}"
RESPONSE=$(curl -X POST \
--silent --fail --show-error \
--header "PRIVATE-TOKEN: ${TOKEN}" \
"${GITLAB_API}/projects/${GITLAB_PROJECT}/jobs/${1}/${2}" \
)
}
ALL_JOBS=$(echo "${ALL_JOBS}" | jq -r '.[] | "\(.id) \(.status)"')
while read job status
do
case "$status" in
created)
;&
pending)
;&
running)
call_api ${job} cancel
;;
failed)
;&
success)
;&
manual)
call_api ${job} erase
;;
*)
echo "Skipping job ${job} with status '${status}'"
;;
esac
done <<< ${ALL_JOBS}
@btgoodwin
Copy link
Author

For anyone using this, it was an attempt at maybe addressing part of this issue. The thought I had was that if you're erasing a job's artifacts and trace, what would be the point of keeping the job record around (and especially its failed status)? I could clean the pipeline statistics to get around the issue of having no DELETE endpoint in their API to touch (and not wanting to muck around in the database).

Apparently, there's some value to retaining the job reference even if all we did was gut it (don't ask me...). And unfortunately, the returned job structure does not get updated to indicate it has been erased so we don't have a way to skip that previously-erased job on subsequent runs (so far as I can tell). The net impact is that if you know your private token has permission to erase, and you still get 403 errors, it's probably because the job was previously erased.

@minzak
Copy link

minzak commented Mar 20, 2019

Not works (

./purge.sh https://gitlab.XXX.net/gcu/caching-system XXXXXXXXXXXXXXXXXXXX
Configuration
    Server:  https://gitlab.XXX.net
    Project: gcu/caching-system
    Token:   XXXXXXXXXXXXXXXXXXXX
Skipping job 6 with status 'skipped'
Skipping job 5 with status 'skipped'
Calling 'erase' for job 4
curl: (22) The requested URL returned error: 403 Forbidden
Skipping job 3 with status 'skipped'
Skipping job 2 with status 'skipped'
Calling 'erase' for job 1
curl: (22) The requested URL returned error: 403 Forbidden

root@GitLab:~#

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment