Skip to content

Instantly share code, notes, and snippets.

@JamesCullum
Last active March 18, 2024 15:37
Show Gist options
  • Star 6 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save JamesCullum/982415c460b3a94ae92c31d4a9698361 to your computer and use it in GitHub Desktop.
Save JamesCullum/982415c460b3a94ae92c31d4a9698361 to your computer and use it in GitHub Desktop.
Gitlab doesn't have a functionality to automatically untag old images (https://gitlab.com/gitlab-org/gitlab-ce/issues/25322). You can use this script to do that for you using the API from any host with HTTP connection (doesn't require access to the server via SSH). Please be aware that you will need a separate cronjob to purge all untagged images.
#!/usr/bin/env python3
'''
DEVELOPMENT SPONSORED BY
PANASONIC INFORMATION SYSTEMS COMPANY EUROPE
Interested in a job? Apply below:
https://application.job.panasonic.eu/data/ruP0pHQvHrGZJKvL/rc.php?nav=jobsearch&custval12=ite&lang=EN&custval11=PBSEU_GER
'''
import json, sys, os
from urllib.parse import urlencode
from urllib.error import HTTPError
from urllib.request import urlopen, Request
import logging
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
stdoutHandler = logging.StreamHandler(sys.stdout)
stdoutHandler.setLevel(logging.DEBUG)
logger.addHandler(stdoutHandler)
# Config
GITLAB_DOMAIN = "https://gitlab.example.com"
AUTH_TOKEN = "ppuYp5B4v2zXDm8ZZrnp" # to use environment variable, replace with os.environ["AUTH_TOKEN"]
PURGE_CRITERIA = "name_regex=.*&keep_n=3&older_than=14d" # untag all images older than 14 days, but always leave the latest 3 images
USE_PROXY = "example.com:80" # None or hostname:port
# Start of code --------------------
HEADERS = {"PRIVATE-TOKEN": AUTH_TOKEN}
API_URL = GITLAB_DOMAIN + "/api/v4"
# List repopsitories
projreq = Request(
API_URL + "/projects",
None,
HEADERS
)
if USE_PROXY != None:
projreq.set_proxy(USE_PROXY, 'http')
with urlopen(projreq) as projectlist:
json_projectlist = json.load(projectlist)
logger.info("Found " + str(len(json_projectlist)) + " repositories")
for repository in json_projectlist:
# List registries within repository
regreq = Request(
API_URL + "/projects/" + str(repository["id"]) + "/registry/repositories",
None,
HEADERS
)
if USE_PROXY != None:
regreq.set_proxy(USE_PROXY, 'http')
with urlopen(regreq) as reglist:
json_reglist = json.load(reglist)
logger.info(repository["name"] + ": Found " + str(len(json_reglist)) + " registries")
if len(json_reglist) == 0:
continue
# Run bulk delete
for registry in json_reglist:
deletereq = Request(
API_URL + "/projects/" + str(repository["id"]) + "/registry/repositories/" + str(registry["id"]) + "/tags",
PURGE_CRITERIA.encode('ascii'),
HEADERS,
method='DELETE'
)
if USE_PROXY != None:
deletereq.set_proxy(USE_PROXY, 'http')
try:
with urlopen(deletereq) as deleteresponse:
logger.debug(repository["name"] + "/" + str(registry["id"]) + ": Bulk delete successful")
except HTTPError as error:
logger.warning(repository["name"] + "/" + str(registry["id"]) + ": Bulk delete failed => " + str(error))
@timrizzi
Copy link

timrizzi commented Jun 5, 2019

Just making sure you didn't display your auth token in line 24.

@JamesCullum
Copy link
Author

Thanks for the hint, but no worry - just generated a random string of the same length for illustration purposes :)

@iddo
Copy link

iddo commented Sep 19, 2019

Here's another variation (in bash) that traverses all the projects of a specific group

#!/usr/bin/env bash
set -e # Stop on error
# Cleanup all gitlab registry images not located in the HEAD of any of the GIT branches
# https://gitlab.com/gitlab-org/gitlab-foss/issues/20247

GROUP="mygroup"
PRIVATE_TOKEN="vewy_vewy_pwivate"

echo "--- Getting projects"

curl --silent --header "PRIVATE-TOKEN: ${PRIVATE_TOKEN}" "https://gitlab.example.com/api/v4/groups/${GROUP}/projects" | jq -r '.[]._links.self' | while read -r PROJECT_URL; do

	echo "--- Getting registry repositories for '${PROJECT_URL}'"
	curl --silent --header "PRIVATE-TOKEN: ${PRIVATE_TOKEN}" "${PROJECT_URL}/registry/repositories" | jq -r '.[].id' | while read -r REPOSITORY; do
		echo "--- Cleaning old registry images for project '${PROJECT_URL}' (repo id ${REPOSITORY})"
		curl --silent --request DELETE --data 'keep_n=5' --data 'name_regex=.*' --data 'older_than=3month' --header "PRIVATE-TOKEN: ${PRIVATE_TOKEN}" "${PROJECT_URL}/registry/repositories/${REPOSITORY}/tags"
		echo "--- Done"
	done
	
done

echo "--- Finished requesting docker image cleanup for all project of group ${GROUP}. Don't forget to run 'gitlab-ctl registry-garbage-collect -m' when the asynchronous cleanup finises to free up disk space"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment