Skip to content

Instantly share code, notes, and snippets.

@ahmetb
Last active February 26, 2024 09:14
Show Gist options
  • Save ahmetb/7ce6d741bd5baa194a3fac6b1fec8bb7 to your computer and use it in GitHub Desktop.
Save ahmetb/7ce6d741bd5baa194a3fac6b1fec8bb7 to your computer and use it in GitHub Desktop.
Script to clean up Google Container Registry images pushed before a particular date
#!/bin/bash
# Copyright © 2017 Google Inc.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
IFS=$'\n\t'
set -eou pipefail
if [[ "$#" -ne 2 || "${1}" == '-h' || "${1}" == '--help' ]]; then
cat >&2 <<"EOF"
gcrgc.sh cleans up tagged or untagged images pushed before specified date
for a given repository (an image name without a tag/digest).
USAGE:
gcrgc.sh REPOSITORY DATE
EXAMPLE
gcrgc.sh gcr.io/ahmet/my-app 2017-04-01
would clean up everything under the gcr.io/ahmet/my-app repository
pushed before 2017-04-01.
EOF
exit 1
elif [[ "${#2}" -ne 10 ]]; then
echo "wrong DATE format; use YYYY-MM-DD." >&2
exit 1
fi
main(){
local C=0
IMAGE="${1}"
DATE="${2}"
for digest in $(gcloud container images list-tags ${IMAGE} --limit=999999 --sort-by=TIMESTAMP \
--filter="timestamp.datetime < '${DATE}'" --format='get(digest)'); do
(
set -x
gcloud container images delete -q --force-delete-tags "${IMAGE}@${digest}"
)
let C=C+1
done
echo "Deleted ${C} images in ${IMAGE}." >&2
}
main "${1}" "${2}"
@jabaganz
Copy link

jabaganz commented Jun 8, 2018

@wibobm i ran into the same requirement. Its easy to accomplish by changing the filter in line 44 of the orinial script to:

--filter "NOT tags:* AND timestamp.datetime < '${DATE}'"

@or-shachar
Copy link

I think there's a way to delete multiple images in the same command, which can be a time saver

@ahmetb
Copy link
Author

ahmetb commented Dec 4, 2019

UPDATE:

Consider using https://github.com/sethvargo/gcr-cleaner which is a Cloud Run app that you deploy and can be triggered periodically with Cloud Scheduler to garbage collect old images.

@jeunii
Copy link

jeunii commented Jan 6, 2020

Quick question. I have something similar that Im going to implement. But I want this script to run inside a Pod in my k8s cluster.

So after dockerizing this script and invoking this as a cronjob in my cluster, how can I make sure that im authorized run the gcloud list and delete image commands ?

@perbu
Copy link

perbu commented Feb 11, 2021

@matti
Copy link

matti commented Sep 2, 2021

This one automatically scans all images from the project, no need to setup per repo as with this or gcr-cleaner: https://github.com/matti/gcr-pruner/blob/main/README.md

@wizzle
Copy link

wizzle commented Jan 17, 2022

One more inline command

gcloud container images list-tags gcr.io/<project-name>/<image_name> --filter="timestamp.date('%Y-%m-%d', Z)<'2021-05-01'" --format="get(digest)" --limit=999999 |  awk '{print "gcr.io/<project-name>/<image_name>@" $1}' | xargs gcloud container images delete --force-delete-tags --quiet

@oprudkyi
Copy link

oprudkyi commented Apr 20, 2022

delete images older then 7 days

GCLOUD_PROJECT_ID=<project> \
    CONTAINER_IMAGE_NAME=<image name> \
    gcloud container images list-tags  \
    --project="${GCLOUD_PROJECT_ID}" \
    "gcr.io/${GCLOUD_PROJECT_ID}/${CONTAINER_IMAGE_NAME}"  \
    --filter="timestamp.date('%Y-%m-%d', Z)<$(date --date='-7 days' +'%Y-%m-%d')" \
    --format="get(digest)" --limit=999999 |  awk '{print "'"gcr.io/${GCLOUD_PROJECT_ID}/${CONTAINER_IMAGE_NAME}@"'" $1}' \
    | xargs -n 1 gcloud container images delete --project="${GCLOUD_PROJECT_ID}" --force-delete-tags --quiet

@burakuluu
Copy link

Is there a way to check if any pod is using the image before deleting it rather than just day check?

@burakuluu
Copy link

@matti perfect! But how can I add it into this sh file before deleting the container?

@matti
Copy link

matti commented Nov 25, 2022

by programming, sometimes you just have to learn it.

@bbakkebo
Copy link

bbakkebo commented Dec 7, 2022

Thanks oprudkyi I have updated your script to what works for me today! :)
Seems --project is not allowed for me, through cloud edit shell, so I removed this argument.

gcloud container images list-tags "gcr.io/{Project_ID}/{Container_Image_Path}" --filter="timestamp.date('%Y-%m-%d', Z)<$(date --date='-7 days' +'%Y-%m-%d')" --format="get(digest)" --limit=999999 | awk '{print "'"gcr.io/{Project_ID}/{Container_Image_Path}@"'" $1}' | xargs -n 1 gcloud container images delete --force-delete-tags --quiet

This gcr.io/{Project_ID}/{Container_Image_Path} should be replaced with the full path located under the container registry :
image

But this is horribly slow to run..

@oprudkyi
Copy link

oprudkyi commented Dec 8, 2022

@bbakkebo
xargs -n1 -P4 might be faster , it will run 4 processes in parallel but output will be unreadable
also we run it inside cicd pipeline after each deploy so there less images to delete (only the first run is slow)

@santygcp
Copy link

GCR.IO is depreciated

@santygcp
Copy link

GCR.IO is depreciated

@transducer
Copy link

transducer commented Oct 31, 2023

Script to only keep 10 latest images for every registry in a project:

#!/bin/bash
PROJECT=<PROJECT>

# Get a list of all registries in the project (assuming eu.gcr.io)
REGISTRIES=$(gcloud container images list --repository=eu.gcr.io/$PROJECT)

# Loop over each registry
while IFS= read -r REGISTRY; do
  # Skip the header line
  if [[ $REGISTRY == "NAME" ]]; then
    continue
  fi

  while true; do
    # Get a list of all image digests, sorted by date (newest first)
    ALL_DIGESTS=$(gcloud container images list-tags $REGISTRY --format="get(digest)" --sort-by="~timestamp")

    # Reset the array and index counter
    unset DIGEST_ARRAY
    i=0

    # Process the multi-line output into an array
    while IFS= read -r line; do
      DIGEST_ARRAY[i++]="$line"
    done <<< "$ALL_DIGESTS"

    # Get the count of all digests
    DIGEST_COUNT=${#DIGEST_ARRAY[@]}

    # If there are more than 10 digests, delete the oldest ones
    if [[ $DIGEST_COUNT -gt 10 ]]; then
      for i in $(seq 10 $((DIGEST_COUNT - 1))); do
        DIGEST=${DIGEST_ARRAY[$i]}
        gcloud container images delete $REGISTRY@$DIGEST --force-delete-tags --quiet
      done
    else
      echo "There are $DIGEST_COUNT images in $REGISTRY, no images to delete."
      break
    fi

    # Sleep for a while before the next iteration (optional)
    sleep 300
  done
done <<< "$REGISTRIES"

This works for Container Registry. In Artifact Repository a cleanup policy can be configured to do the same: https://cloud.google.com/artifact-registry/docs/repositories/cleanup-policy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment