relates to moby/moby#32507, moby/buildkit#442
Doing some silly experimenting with RUN --mount
:
# syntax=docker/dockerfile:1
FROM alpine AS stage1
RUN mkdir -p /stage1-files
RUN echo "testing stage 1" > /stage1-files/s1-file
FROM alpine AS stage2
RUN mkdir -p /stage2-files
RUN echo "testing stage 2" > /stage2-files/s2-file
# The "utility" stage mounts "stage1" (readonly), and "stage2" (readwrite),
# processes files from "stage1", and writes the result to "stage2".
#
# The idea here is to have "utility" build-stages that have tools installed
# to manipulate other stages, without those tools ending up in the stage (layer)
# itself
FROM alpine AS utility
RUN --mount=from=stage1,dst=/stage1 --mount=from=stage2,dst=/stage2,readwrite cp -r /stage1/stage1-files /stage2/
# this "utility" stage processes files from the "stage1" stage, mounting it
# read-write to make modifications
FROM alpine AS utility2
RUN --mount=from=stage1,dst=/stage1,readwrite touch /stage1/stage1-files/s1-file2
# this of course works
FROM alpine AS utility3
RUN mkdir /utility3-files
RUN --mount=from=stage1,dst=/stage1 cp -r /stage1/stage1-files /utility3-files/
# this doesn't work: this still gives the original, unmodified layer from stage1
FROM stage1 AS attempt1
RUN apk add --no-cache tree
CMD tree /stage1-files
# this doesn't work for stage1 and 2: --mount still gives the original,
# unmodified layers from those stages
FROM alpine AS attempt2
RUN apk add --no-cache tree
RUN mkdir -p /results/stage1-result /results/stage2-result /results/utility3-result
RUN --mount=from=stage1,dst=/stage1 cp -r /stage1/stage1-files /results/stage1-result
RUN --mount=from=stage2,dst=/stage2 cp -r /stage2/stage2-files /results/stage2-result
RUN --mount=from=utility3,dst=/utility3 cp -r /utility3/utility3-files /results/utility3-result
CMD tree /results
Building works (no errors):
docker build --no-cache -t bla .
[+] Building 4.7s (18/18) FINISHED
=> local://dockerfile (Dockerfile) 0.0s
=> => transferring dockerfile: 1.88kB 0.0s
=> local://context (.dockerignore) 0.0s
=> => transferring context: 02B 0.0s
=> docker-image://docker.io/tonistiigi/dockerfile:runmount20180618@sha256:576332cea88216b4bf20c56046fabb150c675be4a504440da11970bea501281b 0.0s
=> => resolve docker.io/tonistiigi/dockerfile:runmount20180618@sha256:576332cea88216b4bf20c56046fabb150c675be4a504440da11970bea501281b 0.0s
=> => sha256:576332cea88216b4bf20c56046fabb150c675be4a504440da11970bea501281b 528B / 528B 0.0s
=> => sha256:d0fbaded5db6066249af00e1c83c06c976dc9ba74bfca3d5efee1c7856253aa3 1.58kB / 1.58kB 0.0s
=> local://dockerfile (Dockerfile) 0.0s
=> local://context (.dockerignore) 0.0s
=> CACHED docker-image://docker.io/library/alpine:latest 0.0s
=> /bin/sh -c mkdir -p /stage2-files 0.4s
=> /bin/sh -c mkdir -p /stage1-files 0.5s
=> /bin/sh -c mkdir /utility3-files 0.6s
=> /bin/sh -c apk add --no-cache tree 1.0s
=> /bin/sh -c echo "testing stage 2" > /stage2-files/s2-file 0.5s
=> /bin/sh -c echo "testing stage 1" > /stage1-files/s1-file 0.4s
=> /bin/sh -c mkdir -p /results/stage1-result /results/stage2-result /results/utility3-result 0.5s
=> /bin/sh -c cp -r /stage1/stage1-files /utility3-files/ 0.5s
=> /bin/sh -c cp -r /stage1/stage1-files /results/stage1-result 0.4s
=> /bin/sh -c cp -r /stage2/stage2-files /results/stage2-result 0.4s
=> /bin/sh -c cp -r /utility3/utility3-files /results/utility3-result 0.5s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:3ef6a42407019d37b5d13c9be1685e3ce2fabdf4c4f3d3fa294fc64e7836ded7 0.0s
=> => naming to docker.io/library/bla 0.0s
Running the resulting image produces:
docker run --rm bla
/results
├── stage1-result
│ └── stage1-files
│ └── s1-file
├── stage2-result
│ └── stage2-files
│ └── s2-file
└── utility3-result
└── utility3-files
└── stage1-files
└── s1-file
7 directories, 3 files
It's obvious from the above that I don't have a clue what/how the readwrite
option is used (in combination with from=<stage|image>
)
@rickywu more layers isn't always bad; adding an extra layer doesn't make the image bigger, unless that layer is removing or replacing files that were in previous layers.
In fact, in many case, it's good to keep more layers, especially if those layers are from a common base-image; that way those layers can be shared between images, reducing the overall size on disk (e.g. 5 images all based on "centos:7" will only store (and pull) the "centos:7" layers once, whereas "squashing" those layers will store (and pull) the layers 5 times