Skip to content

Instantly share code, notes, and snippets.

@armand1m
Last active March 10, 2024 14:54
Show Gist options
  • Save armand1m/b8061bcc9e8e9a5c1303854290c7d61e to your computer and use it in GitHub Desktop.
Save armand1m/b8061bcc9e8e9a5c1303854290c7d61e to your computer and use it in GitHub Desktop.
Yarn cache compatible Dockerfile

Yarn Dockerfile

An yarn cache compatible Dockerfile, for building node.js images faster.

Usage

Clone this gist into your project root, and add it to your source control. Change the image service-name:latest tag to your project name in the Dockerfile and build.sh files. Then, always build your image using the build.sh script.

$ chmod +x ./build.sh
$ ./build.sh
#!/bin/bash
if [ ! -f .yarn-cache.tgz ]; then
echo "+ build: Init empty .yarn-cache.tgz"
tar cvzf .yarn-cache.tgz --files-from /dev/null
fi
docker build -t service-name:latest .
docker run \
--rm \
--entrypoint cat \
service-name:latest \
/tmp/yarn.lock > /tmp/yarn.lock
if ! diff -q yarn.lock /tmp/yarn.lock > /dev/null 2>&1; then
echo "+ build: Saving Yarn cache"
docker run \
--rm \
--entrypoint tar \
service-name:latest \
czf - /root/.yarn-cache/ > .yarn-cache.tgz
echo "+ build: Saving yarn.lock"
cp /tmp/yarn.lock yarn.lock
fi
FROM alpine
RUN apk add --update --no-cache nodejs
RUN npm i -g yarn
ADD package.json yarn.lock /tmp/
ADD .yarn-cache.tgz /
RUN cd /tmp && yarn
RUN mkdir -p /service && cd /service && ln -s /tmp/node_modules
COPY . /service
WORKDIR /service
ENV FORCE_COLOR=1
ENTRYPOINT ["npm"]
CMD ["start"]
@etgrieco
Copy link

etgrieco commented Apr 5, 2019

Here is a setup I configured without the need for a local build.sh script to build your image. This will not build node dependencies in to the image, but instead preserve them on a mounted volume:

docker-compose.yml

version: '3.7'

services:
  web:
    build: .
    volumes:
    - yarn_cache:/yarn
    - node_modules_cache:/app/node_modules

volumes:
  yarn_cache:
  node_modules_cache:

Dockerfile

FROM node:8.15.0-alpine

ENV YARN_CACHE_FOLDER /yarn
WORKDIR /app/

# Add entrypoint script to handle yarn cache
RUN mkdir -p $YARN_CACHE_FOLDER
COPY ./modules/docker-entrypoint.sh /
RUN chmod +x /docker-entrypoint.sh
ENTRYPOINT ["/docker-entrypoint.sh"]

ADD . .

CMD ["yarn", "start"]

docker-entrypoint.sh

#!/bin/sh

yarn config set cache-folder $YARN_CACHE_FOLDER
cd $CUSTOMIZATIONS_PATH
(yarn check --integrity && yarn check --verify-tree) || yarn install --frozen-lockfile
cd $DEVENV_PATH

# Call command issued to the docker service
exec "$@"

@weisk
Copy link

weisk commented Oct 2, 2019

Great ! Thanks

@Joseph7451797
Copy link

@etgrieco, thanks for sharing. It works for me. One limit in the solution is that IDE can't use node modules(for type hints or other function) because the modules do not exist in the project folder.

My workaround is replacing this line: - node_modules_cache:/app/node_modules with - .:/app:cached, and node modules show in the project folder. Doing this works fine with IDE, but another problem comes - every time I run yarn add <package>, it seems to reinstall all packages without using Yarn cache.

Do you have any idea about this problem? Maybe I missed something or use this in the wrong way.

FYI, I don't use docker compose. I use docker CLI instead:

$ docker volume create yarn_cache
$ docker build -f Dockerfile.dev -t create-react-app-project:dev .
$ docker run --rm -v ${PWD}:/app:cached -v yarn_cache:/yarn_cache create-react-app-project:dev /bin/sh -c "yarn install"
$ docker run --rm -itd -v ${PWD}:/app:ro,cached -v yarn_cache:/yarn_cache -p 3000:3000 -p 35729:35729 create-react-app-project:dev
$ docker run --rm -v ${PWD}:/app:cached -v yarn_cache:/yarn_cache create-react-app-project:dev /bin/sh -c "yarn add|remove <package name>"

@Rendez
Copy link

Rendez commented Apr 27, 2021

In case someone arrives here wondering why yarn install doesn't even cache locally, check out your COPY commands beforehand, and split them in two:

WORKDIR /app
COPY package.json yarn.lock .yarnrc ./
RUN yarn --pure-lockfile --non-interactive
COPY . ./

If you just do COPY . ./, it's very very likely something has changed in your project, and caching will never work.

@sambacha
Copy link

In case someone arrives here wondering why yarn install doesn't even cache locally, check out your COPY commands beforehand, and split them in two:

WORKDIR /app
COPY package.json yarn.lock .yarnrc ./
RUN yarn --pure-lockfile --non-interactive
COPY . ./

If you just do COPY . ./, it's very very likely something has changed in your project, and caching will never work.

If you are also wondering why your container doesn't work after building on OSX, its because yarn has issues symlinking on OSX/BSD

@ManolisPap
Copy link

ManolisPap commented Mar 7, 2024

@etgrieco The provided code works. On the initial container start up the dependencies need to be downloaded and then are stored into the volume. When the container it is stoped and started again, dependencies does not need to re-installed but yarn take some time to "Link the dependencies", and this happens on every container restart. Any idea about that ? It take a good amount of time in my end (~20 seconds to link the dependencies).

@etgrieco
Copy link

@etgrieco The provided code works. On the initial container start up the dependencies need to be downloaded and then are stored into the volume. When the container it is stoped and started again, dependencies does not need to re-installed but yarn take some time to "Link the dependencies", and this happens on every container restart. Any idea about that ? It take a good amount of time in my end (~20 seconds to link the dependencies).

I haven't used this docker-based workflow in a while, but hopefully others who ran into your issue can chime in!

@ManolisPap
Copy link

@etgrieco Ah, okay. I am developing a Next.js app, and its dependency installation is quite annoying. Without such a "trick", when I change something in package.json, it takes over 150 + seconds just for the yarn install step when a rebuild the Docker Image. So, any trick/optimization technique would be greatly appreciated!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment