Skip to content

Instantly share code, notes, and snippets.

@pbabics
Created November 11, 2016 13:13
Show Gist options
  • Star 21 You must be signed in to star a gist
  • Fork 13 You must be signed in to fork a gist
  • Save pbabics/32840a15ed3ce4dea13a53713aaf94a2 to your computer and use it in GitHub Desktop.
Save pbabics/32840a15ed3ce4dea13a53713aaf94a2 to your computer and use it in GitHub Desktop.
Manual garbage collector for gitlab registry, it removes old revisions that are not referenced by any tag
#!/bin/bash
BASE_PATH=/var/opt/gitlab/gitlab-rails/shared/registry/docker/registry/v2/repositories
DRY_RUN=0
KEEP_LAST_IMAGES=10
RUN_GARBAGE_COLLECTOR=0
GITLAB_CTL_COMMAND=`which gitlab-ctl`
if [ ! -x "${GITLAB_CTL_COMMAND}" ]; then
echo "Missing gitlab-ctl command"
exit 1
fi
while (( "$#" )); do
case "$1" in
"-b" | "--base-path")
BASE_PATH=$2
shift
;;
"-r" | "--run-gc")
RUN_GARBAGE_COLLECTOR=1
;;
"-d"|"--dry-run")
DRY_RUN=1
;;
"-k"|"--keep")
if ! ( echo $2 | grep -q '^[0-9]\+$') || [ $2 -eq 0 ]; then
echo "Invalid value for keep last images '$2'"
exit 1
fi
KEEP_LAST_IMAGES=$2
shift
;;
"-h"|"--help")
echo "Usage: ${0} [options]"
echo "Options:"
echo -e "\t-k NUM, --keep NUM"
echo -e "\t\tKeeps last NUM revisions, except current tags"
echo
echo -e "\t-d, --dry-run"
echo -e "\t\tEnables dry run, no changes will be made"
echo
echo -e "\t-b, --base-path"
echo -e "\t\tSets base path of Gitlab Registry repository storage"
echo
echo -e "\t-r, --run-gc"
echo -e "\t\tStarts garbage collector after revision removal"
exit 0
;;
*)
echo "Unknown argument: $1"
exit 1
;;
esac
shift
done
IFS=$'\n'
used_hashes=`mktemp`
marked_hashes=`mktemp`
for repository in `find ${BASE_PATH} -mindepth 2 -maxdepth 2 -type d | sed "s#${BASE_PATH}/##"`; do
for tag_hash in ${BASE_PATH}/${repository}/_manifests/tags/*/current/link; do
cat "${tag_hash}" | cut -d':' -f2;
done > "${used_hashes}"
echo "Removing revisions of $repository:"
ls -t ${BASE_PATH}/${repository}/_manifests/revisions/sha256 | fgrep -vf "${used_hashes}" | tail -n+${KEEP_LAST_IMAGES} | tee ${marked_hashes}
if [ ${DRY_RUN} -ne 1 ]; then
cat ${marked_hashes} | sed "s#^#${BASE_PATH}/${repository}/_manifests/revisions/sha256/#" | xargs rm -rf
fi
done
rm ${used_hashes}
rm ${marked_hashes}
if [ ${DRY_RUN} -eq 0 -a ${RUN_GARBAGE_COLLECTOR} -eq 1 ]; then
"${GITLAB_CTL_COMMAND}" registry-garbage-collect
fi
@ronaldevers
Copy link

Works like a charm in GitLab 9.1 EE. Thanks!

@eedugon
Copy link

eedugon commented Jul 13, 2017

Thanks very very much for sharing!
I have created my own version removing the fixed maxdepth/mindepth of 2 and also including some extra control for directories that do not have valid "tags" or "revisions/sha256" directories, to avoid processing them.
Feel free to merge it to your release if you want, as the payload is exactly the same.
Regards!

@eedugon
Copy link

eedugon commented Jul 13, 2017

This is my revision if you want to take a look:
https://gist.github.com/eedugon/69da0397c142e56f4186ee691502e7ee

Small features added:

  • Searching in all BASE_PATH, not fixing the search to a depth of 2
  • Directories without valid tags or revisions directories won't be processed (to avoid unexpected issues)
  • Logging in case there's nothing to delete
  • running registry-garbage-collect only if something was deleted
  • Added crontab example

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment