Caching Docker builds in GitHub Actions is an excellent article by @dtinth which analyses various strategies for speeding up builds in GitHub Actions. The upshot of the article is a fairly decisive conclusion that the best two ways to improve build times are:
-
Build images via a standard
docker build
command, while using GitHub Packages' Docker registry as a cache = Longer initial build but fastest re-build times. -
Build your images via docker integrated BuildKit (
DOCKER_BUILDKIT=1 docker build
), while using a local registry and actions/cache to persist build caches = Fastest initial build but slightly longer re-build times.
Unfortunately, when trying to implement these solutions in my own project, I discovered that both approaches fall short when it comes to multi-stage builds.
The problem is that inlined build cache only includes layers directly involved in the creation of the image. It excludes caches from various stages in my Dockerfile
which are copied into the final image using
COPY --from=...
Using the above strategies did improve my image re-building time. It has gone down from ~ 3m 15s
to about 1m 50s
. However, this improvement was not as much as I would have liked...
Luckily, the guys behind docker integrated BuildKit have a standalone tool for building docker images. The tool comes in two parts: buildkitd
- a build daemon, and buildctl
- a controller for the build daemon.
The features that I was most interested in, is the ability of BuildKit controller (buildkitctl
) to export/import a full set of build caches, including caches for all multi-stage layers. Combining this ability with actions/cache gave me a nearly perfect solution for speeding up multi-stage docker builds in Github Actions.
Final time to re-build the image has now been reduced from ~ 3m 15s
to ~ 38s
!
PS: Interestingly, BuildKit daemon (buildkitd
) can run either locally or remotely. This opens up the possibility of hosting a standalone build process on your own infrastructure, which might further improve building time.
@epicserve, I think the first question to ask is: why do you want to have separate caches for different Git branch? As @LarsFronius pointed out, the cache key for this workflow is defined on line 24. However, if your
Dockerfile
doesn't change between branches then you can just reuse the cache from the old branch.The manual page for actions/cache explains it best: The cache action first searches for cache hits for
key
andrestore-keys
in the branch containing the workflow run. If there are no hits in the current branch, the cache action searches forkey
andrestore-keys
in the parent branch and upstream branches.That means that all you have to do is make this action run on all of your branches by removing the
branch
filter on line 5. I haven't used this code in a while but from what I remember removing line 5 all-together will make this action run on allpush
andpull_request
to any branch.