Composite github action to improve CI time with yarn 3+ / node-modules linker.

ci-yarn-install.md

Why

Although @setup/node as a built-in cache option, it lacks an opportunity regarding cache persistence. Depending on usage, the action below might give you faster installs and potentially reduce carbon emissions (♻️🌳❤️).

Requirements

Yarn 3+ with nodeLinker: node-modules. (Not using yarn ? see the corresponding pnpm 7 action gist)

Bench

With cache: at least twice faster than without (see PS at the bottom for results)

Structure

.
└── .github
    ├── actions
    │   └── yarn-nm-install/action.yml (composite action)    
    └── workflows
        └── ci.yml (uses: ./.github/actions/yarn-nm-install)    

Composite action

Create a file in .github/actions/yarn-nm-install/action.yml and paste

########################################################################################
# "yarn install" composite action for yarn 2/3/4+ and "nodeLinker: node-modules"       #
#--------------------------------------------------------------------------------------#
# Cache:                                                                               #
#   - Downloaded zip archive (multi-arch, preserved across yarn.lock changes)          #
#   - Yarn install state (discarded on yarn.lock changes)                              #
# References:                                                                          #
#   - bench: https://gist.github.com/belgattitude/0ecd26155b47e7be1be6163ecfbb0f0b     #
#   - vs @setup/node: https://github.com/actions/setup-node/issues/325                 #
########################################################################################

name: 'Yarn install'
description: 'Run yarn install with node_modules linker and cache enabled'

runs:
  using: 'composite'
  steps:
    - name: Expose yarn config as "$GITHUB_OUTPUT"
      id: yarn-config
      shell: bash
      run: |
        echo "CACHE_FOLDER=$(yarn config get cacheFolder)" >> $GITHUB_OUTPUT

    # Yarn rotates the downloaded cache archives, @see https://github.com/actions/setup-node/issues/325
    # Yarn cache is also reusable between arch and os.
    - name: Restore yarn cache
      uses: actions/cache@v3
      id: yarn-download-cache
      with:
        path: ${{ steps.yarn-config.outputs.CACHE_FOLDER }}
        key: yarn-download-cache-${{ hashFiles('yarn.lock') }}
        restore-keys: |
          yarn-download-cache-

    # Invalidated on yarn.lock changes
    - name: Restore yarn install state
      id: yarn-install-state-cache
      uses: actions/cache@v3
      with:
        path: .yarn/ci-cache/
        key: ${{ runner.os }}-yarn-install-state-cache-${{ hashFiles('yarn.lock', '.yarnrc.yml') }}

    - name: Install dependencies
      shell: bash
      run: |
        yarn install --immutable --inline-builds
      env:
        # CI optimizations. Overrides yarnrc.yml options (or their defaults) in the CI action.
        YARN_ENABLE_GLOBAL_CACHE: 'false' # Use local cache folder to keep downloaded archives
        YARN_NM_MODE: 'hardlinks-local' # Hardlinks-(local|global) reduces io / node_modules size
        YARN_INSTALL_STATE_PATH: .yarn/ci-cache/install-state.gz # Very small speedup when lock does not change
        # Other environment variables
        HUSKY: '0' # By default do not run HUSKY install

Workflow action

To use it in the workflows

    steps:
      - uses: actions/checkout@v3

      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v3
        with:
          node-version: ${{ matrix.node-version }}

      - name: 📥 Monorepo install
        uses: ./.github/actions/yarn-nm-install

yarnrc.yml

# .yarnrc.yml
nodeLinker: node-modules
# This line can be omiited with corepack enabled 
# in this case set the packageManager field in package.json
yarnPath: .yarn/releases/yarn-4.0.0-rc.36.cjs # or v3...

Results

On install, when only few deps changed

Cost of action/cache compression

Cleanup caches

When a PR is closed or merged the best is to remove install cache rather than letting github reach the max (10GB) and prune.

Link: https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#force-deleting-cache-entries

Here's an example (feel free to adapt if you need to preserse some things, ie gh actions-cache list -R $REPO -B $BRANCH | cut -f 1 | grep yarn will only clear yarn related caches)

# .github/workflows/clean-up-pr-caches.yml
name: Cleanup caches for closed branches

on:
  pull_request:
    types:
      - closed
  workflow_dispatch:

jobs:
  cleanup:
    runs-on: ubuntu-latest
    steps:
      - name: Check out code
        uses: actions/checkout@v3

      - name: Cleanup yarn caches
        run: |
          gh extension install actions/gh-actions-cache

          REPO=${{ github.repository }}
          BRANCH=${{ github.ref }}

          echo "Fetching list of cache key"
          cacheKeysForPR=$(gh actions-cache list -R $REPO -B $BRANCH | cut -f 1 )

          ## Setting this to not fail the workflow while deleting cache keys. 
          set +e
          echo "Deleting caches..."
          for cacheKey in $cacheKeysForPR
          do
              gh actions-cache delete $cacheKey -R $REPO -B $BRANCH --confirm
          done
          echo "Done"
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Bench details

Based on the nextjs-monorepo-example and the results of https://github.com/belgattitude/compare-package-managers it's twice fast. Depending on repo (renovatebot...), the slight complexity increase in ci setup might worth it. Example based on yarn 4.0.0-rc.36 with most post install scripts disabled and supportedArchitecture: current.

CACHE	YARN_COMPRESSION_LEVEL=mixed (default)	YARN_COMPRESSION_LEVEL=0
COLD	1m38s	44s
FULL WARM	54s	18s

PS: YARN_COMPRESSION_LEVEL=0 disable zip compression, on the ci it creates 2-3 more extra seconds (github will (un-)zstd it), in your local install you'll have to deal with a bigger cache (it's a choice you need to do for all as it changes the yarn.lock md5 checksums)