Skip to content

Instantly share code, notes, and snippets.

@Smasherr
Last active October 10, 2023 07:33
Show Gist options
  • Save Smasherr/972272b56eebae3b860d62ef05eea6eb to your computer and use it in GitHub Desktop.
Save Smasherr/972272b56eebae3b860d62ef05eea6eb to your computer and use it in GitHub Desktop.
This script can be used to copy all docker images from one GitLab-project to another. It was created because in GitLab it's not possible to move projects that contain docker images.
#!/usr/bin/env bash
# This script can be used to copy all docker images from one GitLab-project to another.
# It was created because in GitLab it's not possible to move projects that contain docker images.
#
# Related issue: https://gitlab.com/gitlab-org/gitlab/-/issues/18383
#
# Author: Daniel Estermann <estermad@apache.org>
# Thanks to komar <komar@core.org.ua> for his brilliant support
ME=$(basename "$0")
DEPENDS="docker curl jq tac"
die() {
if [[ "$*" && -t 2 ]]; then
printf "\e[31;1m%s\e[0m\n" "$*" >&2
else
printf "%s\n" "$*" >&2
fi
exit 1
}
usage() {
cat <<- EOM
Usage: $ME <gitlab-server> <path-to-project> <new-path-to-project>
Example: $ME gitlab.com johndoe/foo johndoe/bar
EOM
exit 1
}
[[ ${GITLAB_TOKEN} ]] || die "Please provide an access token via the environment variable GITLAB_TOKEN"
for CMD in $DEPENDS; do
command -v $CMD >/dev/null 2>&1 || die "This script requires $CMD but it's not installed. Aborting."
done
docker ps >/dev/null 2>&1 || die "This script requires docker but its engine is unreachable. Aborting."
[[ ${1} && ${2} && ${3} ]] || usage
PROJECT_NS=${2//\//%2F}
REPOSITORY_IDS=$(curl -s -H "PRIVATE-TOKEN:${GITLAB_TOKEN}" \
"https://$1/api/v4/projects/${PROJECT_NS}/registry/repositories/" |
jq -r '.[].id' 2>/dev/null)
[[ ${REPOSITORY_IDS} ]] || die "$1/$2 has no docker images"
TMPFILE=$(mktemp -u /tmp/$ME.XXXXXX)
for REPOSITORY_ID in $REPOSITORY_IDS; do
TAGS_URL="https://$1/api/v4/projects/${PROJECT_NS}/registry/repositories/${REPOSITORY_ID}/tags?per_page=100"
TOTAL_PAGES=$(curl -sI -X HEAD -H "PRIVATE-TOKEN:${GITLAB_TOKEN}" ${TAGS_URL} |
sed -nE 's/X-Total-Pages:[[:space:]]*([^'$'\r'']*)'$'\r''?/\1/ip')
[[ $TOTAL_PAGES =~ ^[0-9]+$ ]] || die "Error getting total page number using HTTP HEAD on $TAGS_URL (got $TOTAL_PAGES)"
for PAGE in $(seq 1 $TOTAL_PAGES); do
curl -s -H "PRIVATE-TOKEN:${GITLAB_TOKEN}" "${TAGS_URL}&page=${PAGE}" |
jq -r '.[].location'
done |
sed -E "s#(.*)#docker pull \1 \&\& docker tag \1 \1#g
s#(.*)($2)(.*:.*)#\1$3\3#i" |
tee -a $TMPFILE
done
cat << EOM >> $TMPFILE
while true; do
read -p "Do you want to push the images now? [Y/n] " yn
case \${yn:-Y} in
[Yy]* ) break;;
* ) echo "Abort."; exit;;
esac
done
EOM
tac $TMPFILE | sed '1,7d' | tac |
sed -E "s#.* (.*)#docker push \1#" |
tee -a $TMPFILE
[[ -f $TMPFILE ]] &&
chmod +x $TMPFILE &&
echo "You can now review the generated script in stdout or in $TMPFILE and if it looks fine for you just execute it." ||
die "Something went wrong, sorry :("
@JanWendler
Copy link

JanWendler commented Oct 10, 2023

@Smasherr This is the output:

$ ./copy_images.sh <our_server> <original_repo> jan.wendler/temp
++ basename ./copy_images.sh
+ ME=copy_images.sh
+ DEPENDS='docker curl jq tac'
+ [[ -n <Token> ]]
+ for CMD in $DEPENDS
+ command -v docker
+ for CMD in $DEPENDS
+ command -v curl
+ for CMD in $DEPENDS
+ command -v jq
+ for CMD in $DEPENDS
+ command -v tac
+ docker ps
+ [[ -n <our_server> ]]
+ [[ -n <original_repo> ]]
+ [[ -n jan.wendler/temp ]]
+ PROJECT_NS=<original_repo_%2F>
++ curl -s -H PRIVATE-TOKEN:<Token> https://<our_server>/api/v4/projects/<original_repo_%2F>/registry/repositories/
++ jq -r '.[].id'
+ REPOSITORY_IDS=1702
+ [[ -n 1702 ]]
++ mktemp -u /tmp/copy_images.sh.XXXXXX
+ TMPFILE=/tmp/copy_images.sh.AIoZz4
+ for REPOSITORY_ID in $REPOSITORY_IDS
+ TAGS_URL='https://<our_server>/api/v4/projects/<original_repo_%2F>/registry/repositories/1702/tags?per_page=100'
++ curl -sI -X HEAD -H PRIVATE-TOKEN:<Token> 'https://<our_server>/api/v4/projects/<original_repo_%2F>/registry/repositories/1702/tags?per_page=100'
++ sed -nE 's/X-Total-Pages:[[:space:]]*([^]*)?/\1/ip'
sed: -e expression #1, char 41: unterminated `s' command
+ TOTAL_PAGES=
+ [[ '' =~ ^[0-9]+$ ]]
+ die 'Error getting total page number using HTTP HEAD on https://<our_server>/api/v4/projects/<original_repo_%2F>/registry/repositories/1702/tags?per_page=100 (got )'
+ [[ -n Error getting total page number using HTTP HEAD on https://<our_server>/api/v4/projects/<original_repo_%2F>/registry/repositories/1702/tags?per_page=100 (got ) ]]
+ [[ -t 2 ]]
+ printf '\e[31;1m%s\e[0m\n' 'Error getting total page number using HTTP HEAD on https://<our_server>/api/v4/projects/<original_repo_%2F>/registry/repositories/1702/tags?per_page=100 (got )'
Error getting total page number using HTTP HEAD on https://<our_server>/api/v4/projects/<original_repo_%2F>/registry/repositories/1702/tags?per_page=100 (got )
+ exit 1

I'm working on Windows 10 with an elevated gitbash console. I hope this helps : )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment