shinsenter/Reduce docker image size.md

## Reduce docker image size.md

      
        

            Raw
  

        
        
          
            
          
          
            
              Reduce docker image size.md
            
          
        
      
      
    Reduce docker image size by minimizing the number of layers
Even though the Docker build process is easy, many organizations make the mistake of building bloated Docker images without optimizing the container images.
It is best to have small-sized images to reduce the image transfer and deploy time.
Docker official website has a great article Best practices for writing Dockerfiles which covers recommended best practices and methods for building efficient images.
Docker images work in the following way – each RUN, COPY, ADD, FROM instruction adds a new layer. All changes made to the running container, such as writing new files, modifying existing files, and deleting files, are written to layers. The layers are stacked and each one is a delta of the changes from the previous layer. All of them make docker images become bigger.
But using all optimization techniques is tough. So, I made a small script to combine all docker image layers into one, and also delete junk files in the image.
This script is a perfect alternative to the still experimental --squash flag in docker build.
Download
Download the tidy-docker.sh file below, and give it the execution ability by running: chmod +xr tidy-docker.sh.
Usage
./tidy-docker.sh <soure_image> [output_image] [docker build options]

soure_image: The original image (ID or name:tag)
output_image: Target image name (optional)

./tidy-docker.sh <output> <soure_image> [output_image] [docker build options]

output: Create new Dockerfile (must be an existing file path)
soure_image: The original image (ID or name:tag)
output_image: Target image name (optional)

Example
docker pull ubuntu:xenial
./tidy-docker.sh ubuntu:xenial ubuntu:xenial-tidy
docker image ls
The result of docker image ls:
REPOSITORY   TAG           IMAGE ID       CREATED         SIZE
ubuntu       xenial-tidy   e0a6604ab241   4 seconds ago   78.1MB
ubuntu       xenial        fe3b34cb9255   6 months ago    119MB

Using with docker build options:
docker pull ubuntu:xenial --platform linux/amd64
./tidy-docker.sh ubuntu:xenial ubuntu:xenial-tidy --platform linux/amd64
Creating new Dockerfile:
docker pull ubuntu:xenial
./tidy-docker.sh /my/working/dir/Dockerfile ubuntu:xenial

  

  

## tidy-docker.sh
#!/bin/bash
# This file belongs to the project https://code.shin.company/php
# Author:  Shin <shin@shin.company>
# License: https://code.shin.company/php/blob/main/LICENSE
################################################################################

[ ! -x "$(command -v docker)" ] && echo "Missing Docker." && exit 1

################################################################################

PARSE_CACHE=

# extracts docker repository name and tag
d_name_id () {
    docker images --format "{{.Repository}}:{{.Tag}}\t{{.ID}}" \
    | grep -F $1 \
    | head -n1
}

# parses instruction and replace all '/bin/sh -c #(nop)'
d_parse () {
    if [ -z "$PARSE_CACHE" ]; then
        PARSE_CACHE="$(
            docker history --no-trunc --format "{{.CreatedBy}}" $1 \
                | awk '{arr[i++]=$0}END{while(i>0)print arr[--i]}' \
                | sed 's/^\/[^ ]* *-c */RUN /' \
                | sed 's/^RUN \#[^ ]* *//' \
                | sed 's#EXPOSE map\[\(.*\):.*\]#EXPOSE \1#g' \
                | awk '
                {
                    if($1=="SHELL"){
                        gsub("\\\[","[\"",$0); gsub("\\\]","\"]",$0);
                        i=1; printf("%s ",$i);
                        while(i++<NF){if(i>2){printf("\", \"%s",$i)}else{printf("%s",$i)}};
                        printf "\n";
                    } else print $0
                }' 2>/dev/null \
                | awk '
                {
                    if(($1=="WORKDIR")||($1=="ENTRYPOINT")||($1=="CMD")||($1=="STOPSIGNAL"))
                        {cmd[$1]=$0;if($1=="ENTRYPOINT")delete cmd["CMD"]}
                    else print $0
                } END {
                    if(cmd["WORKDIR"])    print cmd["WORKDIR"];
                    if(cmd["ENTRYPOINT"]) print cmd["ENTRYPOINT"];
                    if(cmd["CMD"])        print cmd["CMD"];
                    if(cmd["STOPSIGNAL"]) print cmd["STOPSIGNAL"];
                }'
        )"
    fi
    echo "$PARSE_CACHE"
}

# extracts given unique key-value instruction list
d_attr () {
    d_parse $1 \
        | grep "^${2:-ENV}" \
        | awk     '{if(gsub("=","=")<2){print $0}else{i=1;while(i++<=NF){if($i!="")printf("%s %s\n",$1,$i)}}}' \
        | awk -F= '{a[$1]=$2}END{for(v in a)printf("%s=\"%s\"\n",v,a[v])}' \
        | sort
}

# extracts other instructions
ins_cmd () {
    d_parse $1 | grep -v '^\(ADD\|ARG\|COPY\|ENV\|HEALTHCHECK\|LABEL\|ONBUILD\|RUN\)'
}

# aliases
ins_name () { d_name_id $1 | awk '{printf $1}'; }
ins_id   () { d_name_id $1 | awk '{printf $2}'; }

# command aliases
ins_env    () { d_attr  $1 ENV; }
ins_labels () { d_attr  $1 LABEL; }
ins_shell  () { ins_cmd $1 | grep    '^SHELL'; }
ins_others () { ins_cmd $1 | grep -v '^SHELL'; }

# builds minified image
# @param $output Output new Dockerfile (optional, must be a valid file path)
# @param $base   The original image (ID or name:tag)
# @param $save   Target image name
minify() {
    PARSE_CACHE=
    local output="$1" ; [ ! -z "$output" ] && [ -f "$output" ] && shift || output=""
    local base="$1"   ; [ ! -z "$base"   ] && shift || return 1
    local save="$1"   ; [ ! -z "$save"   ] && shift || save="${base}-tidy"
    local repo="$(ins_name $base)" ; [ -z "$output" ] && [ -z "$repo" ] && echo "Invalid image ID $base" && return 1
    local temp="$([ ! -z "$output" ]    && echo "$base" || echo "tidy-docker:build-$(ins_id $base)")"
    local command="$([ ! -z "$output" ] && echo "tee $output" || echo "docker build $@ --rm -t $save -")"

    # make temporary tag name for building image
    [ -z "$output" ] && docker tag $base $temp 2>/dev/null

    # build the tidy
    echo "🗜 Start minifying image '$repo'"
    echo "  Build arguments: $@"
    DOCKER_BUILDKIT=${DOCKER_BUILDKIT:-1} $command <<Dockerfile
# Input Image: "${base}"
#        (aka: "${repo}")
# Final Image: "${save}"
################################################################################
# CLEANING UP THE SOURCE IMAGE. ################################################
FROM shinsenter/scratch as scratch
FROM ${temp} as source
USER root
RUN [ -x "\$(command -v apt-get)" ]  && apt-get -yq autoremove --purge || true
RUN [ -x "\$(command -v yum)" ]      && yum clean all -y               || true
RUN [ -x "\$(command -v composer)" ] && composer clearcache -q --ansi  || true
RUN [ -x "\$(command -v npm)" ]      && npm cache clean --force        || true
RUN [ -x "\$(command -v docker)" ]   && docker system prune -af        || true
RUN [ -x "\$(command -v find)" ]     && find / \( \\
    -name "._*" -or -name "*~" -or -name "*.swp" \\
    -or -name ".git" -or -name ".svn" \\
    -or -name ".DS_Store" -or -name "Thumbs.db" -or -name "thumbs.db" \\
    -or -name "*.pyc" -or -name "*.pyo" -or -name "*pip*" -or -name "*__pycache__*" \\
    -or -name "*easy_install*" -or -name "*dist-info*" \\
    \) ! -path "/sys/*" ! -path "/proc/*" \\
    -exec rm -rf {} + || true
RUN [ -x "\$(command -v rm)" ]       && rm -rf \\
    ~/.wp-cli/ ~/.git/ ~/.composer/ ~/.npm/ ~/.cache/ ~/.log/ ~/.tmp/ \\
    /var/cache/apk /var/cache/yum /var/lib/apt/lists/* \\
    /usr/share/doc/* /tmp/* /var/tmp/* \\
    || true
################################################################################
# BUILDING OPTIMIZED IMAGE FROM SCRATCH. #####################################
FROM  scratch
$(ins_shell $base)
COPY  --from=source / /
$(ins_env $base)
$(ins_others $base)
LABEL org.opencontainers.image.title="$save"
LABEL org.opencontainers.image.description="Minified from $repo"
$(ins_labels $base)
# FINISH. ######################################################################
################################################################################
Dockerfile

    # cleanup
    if [ $? -eq 0 ] && [ -z "$output" ]; then docker rmi "$temp" >/dev/null; fi
}

################################################################################

echo ; date
minify $@ && echo "Done." || echo "Failed."
	#!/bin/bash
	# This file belongs to the project https://code.shin.company/php
	# Author: Shin <shin@shin.company>
	# License: https://code.shin.company/php/blob/main/LICENSE
	################################################################################

	[ ! -x "$(command -v docker)" ] && echo "Missing Docker." && exit 1

	################################################################################

	PARSE_CACHE=

	# extracts docker repository name and tag
	d_name_id () {
	docker images --format "{{.Repository}}:{{.Tag}}\t{{.ID}}" \
	\| grep -F $1 \
	\| head -n1
	}

	# parses instruction and replace all '/bin/sh -c #(nop)'
	d_parse () {
	if [ -z "$PARSE_CACHE" ]; then
	PARSE_CACHE="$(
	docker history --no-trunc --format "{{.CreatedBy}}" $1 \
	\| awk '{arr[i++]=$0}END{while(i>0)print arr[--i]}' \
	\| sed 's/^\/[^ ]* -c /RUN /' \
	\| sed 's/^RUN \#[^ ]* *//' \
	\| sed 's#EXPOSE map\[\(.\):.\]#EXPOSE \1#g' \
	\| awk '
	{
	if($1=="SHELL"){
	gsub("\\\[","[\"",$0); gsub("\\\]","\"]",$0);
	i=1; printf("%s ",$i);
	while(i++<NF){if(i>2){printf("\", \"%s",$i)}else{printf("%s",$i)}};
	printf "\n";
	} else print $0
	}' 2>/dev/null \
	\| awk '
	{
	if(($1=="WORKDIR")\|\|($1=="ENTRYPOINT")\|\|($1=="CMD")\|\|($1=="STOPSIGNAL"))
	{cmd[$1]=$0;if($1=="ENTRYPOINT")delete cmd["CMD"]}
	else print $0
	} END {
	if(cmd["WORKDIR"]) print cmd["WORKDIR"];
	if(cmd["ENTRYPOINT"]) print cmd["ENTRYPOINT"];
	if(cmd["CMD"]) print cmd["CMD"];
	if(cmd["STOPSIGNAL"]) print cmd["STOPSIGNAL"];
	}'
	)"
	fi
	echo "$PARSE_CACHE"
	}

	# extracts given unique key-value instruction list
	d_attr () {
	d_parse $1 \
	\| grep "^${2:-ENV}" \
	\| awk '{if(gsub("=","=")<2){print $0}else{i=1;while(i++<=NF){if($i!="")printf("%s %s\n",$1,$i)}}}' \
	\| awk -F= '{a[$1]=$2}END{for(v in a)printf("%s=\"%s\"\n",v,a[v])}' \
	\| sort
	}

	# extracts other instructions
	ins_cmd () {
	d_parse $1 \| grep -v '^\(ADD\\|ARG\\|COPY\\|ENV\\|HEALTHCHECK\\|LABEL\\|ONBUILD\\|RUN\)'
	}

	# aliases
	ins_name () { d_name_id $1 \| awk '{printf $1}'; }
	ins_id () { d_name_id $1 \| awk '{printf $2}'; }

	# command aliases
	ins_env () { d_attr $1 ENV; }
	ins_labels () { d_attr $1 LABEL; }
	ins_shell () { ins_cmd $1 \| grep '^SHELL'; }
	ins_others () { ins_cmd $1 \| grep -v '^SHELL'; }

	# builds minified image
	# @param $output Output new Dockerfile (optional, must be a valid file path)
	# @param $base The original image (ID or name:tag)
	# @param $save Target image name
	minify() {
	PARSE_CACHE=
	local output="$1" ; [ ! -z "$output" ] && [ -f "$output" ] && shift \|\| output=""
	local base="$1" ; [ ! -z "$base" ] && shift \|\| return 1
	local save="$1" ; [ ! -z "$save" ] && shift \|\| save="${base}-tidy"
	local repo="$(ins_name $base)" ; [ -z "$output" ] && [ -z "$repo" ] && echo "Invalid image ID $base" && return 1
	local temp="$([ ! -z "$output" ] && echo "$base" \|\| echo "tidy-docker:build-$(ins_id $base)")"
	local command="$([ ! -z "$output" ] && echo "tee $output" \|\| echo "docker build $@ --rm -t $save -")"

	# make temporary tag name for building image
	[ -z "$output" ] && docker tag $base $temp 2>/dev/null

	# build the tidy
	echo "🗜 Start minifying image '$repo'"
	echo " Build arguments: $@"
	DOCKER_BUILDKIT=${DOCKER_BUILDKIT:-1} $command <<Dockerfile
	# Input Image: "${base}"
	# (aka: "${repo}")
	# Final Image: "${save}"
	################################################################################
	# CLEANING UP THE SOURCE IMAGE. ################################################
	FROM shinsenter/scratch as scratch
	FROM ${temp} as source
	USER root
	RUN [ -x "\$(command -v apt-get)" ] && apt-get -yq autoremove --purge \|\| true
	RUN [ -x "\$(command -v yum)" ] && yum clean all -y \|\| true
	RUN [ -x "\$(command -v composer)" ] && composer clearcache -q --ansi \|\| true
	RUN [ -x "\$(command -v npm)" ] && npm cache clean --force \|\| true
	RUN [ -x "\$(command -v docker)" ] && docker system prune -af \|\| true
	RUN [ -x "\$(command -v find)" ] && find / \( \\
	-name "._" -or -name "~" -or -name "*.swp" \\
	-or -name ".git" -or -name ".svn" \\
	-or -name ".DS_Store" -or -name "Thumbs.db" -or -name "thumbs.db" \\
	-or -name ".pyc" -or -name ".pyo" -or -name "pip" -or -name "__pycache__" \\
	-or -name "easy_install" -or -name "dist-info" \\
	\) ! -path "/sys/" ! -path "/proc/" \\
	-exec rm -rf {} + \|\| true
	RUN [ -x "\$(command -v rm)" ] && rm -rf \\
	~/.wp-cli/ ~/.git/ ~/.composer/ ~/.npm/ ~/.cache/ ~/.log/ ~/.tmp/ \\
	/var/cache/apk /var/cache/yum /var/lib/apt/lists/* \\
	/usr/share/doc/* /tmp/* /var/tmp/* \\
	\|\| true
	################################################################################
	# BUILDING OPTIMIZED IMAGE FROM SCRATCH. #####################################
	FROM scratch
	$(ins_shell $base)
	COPY --from=source / /
	$(ins_env $base)
	$(ins_others $base)
	LABEL org.opencontainers.image.title="$save"
	LABEL org.opencontainers.image.description="Minified from $repo"
	$(ins_labels $base)
	# FINISH. ######################################################################
	################################################################################
	Dockerfile

	# cleanup
	if [ $? -eq 0 ] && [ -z "$output" ]; then docker rmi "$temp" >/dev/null; fi
	}

	################################################################################

	echo ; date
	minify $@ && echo "Done." \|\| echo "Failed."