Skip to content

Instantly share code, notes, and snippets.

@ThatGuySam
Forked from belgattitude/ci-pnpm-install.md
Created June 10, 2023 20:23
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ThatGuySam/663e2d7b798bbedc41a0ff150858ba2c to your computer and use it in GitHub Desktop.
Save ThatGuySam/663e2d7b798bbedc41a0ff150858ba2c to your computer and use it in GitHub Desktop.
Composite github action to improve CI time with pnpm

Why

Although @setup/node as a built-in cache option, it lacks an opportunity regarding cache persistence. Depending on usage, the action below might give you faster installs and potentially reduce carbon emissions (β™»οΈπŸŒ³β€οΈ).

Requirements

pnpm v7 or v8 (not using pnpm ? see the corresponding yarn action gist)

Bench

Based on the nextjs-monorepo-example with pnpm.

A cold cache install on the ci is around Β±1m20s.

With warmed cache: Β±40s + (add Β±10s for compression). Crafted from benchmarks results in https://gist.github.com/belgattitude/0ecd26155b47e7be1be6163ecfbb0f0b. Depending on repo (renovatebot...), the slight complexity increase in ci setup might worth it.

Structure

.
└── .github
    β”œβ”€β”€ actions
    β”‚   └── pnpm-install/action.yml (composite action)    
    └── workflows
        └── ci.yml (uses: ./.github/actions/pnpm-install)    

Composite action

Create a file in .github/actions/pnpm-install/action.yml and paste

########################################################################################
# "pnpm install" composite action for pnpm 7/8+                                        #
#--------------------------------------------------------------------------------------#
# Requirement: @setup/node should be run before                                        #
#                                                                                      #
# Usage in workflows steps:                                                            #
#                                                                                      #
#      - name: πŸ“₯ Monorepo install                                                     #
#        uses: ./.github/actions/pnm-install                                           #
#        with:                                                                         #
#          enable-corepack: false # (default)                                          #
#                                                                                      #
# Reference:                                                                           #
#   - latest: https://gist.github.com/belgattitude/838b2eba30c324f1f0033a797bab2e31    #
########################################################################################

name: 'Monorepo install (pnpm)'
description: 'Run pnpm install with cache enabled'
inputs:
  enable-corepack:
    description: 'Enable corepack'
    required: false
    default: 'false'

runs:
  using: 'composite'

  steps:
    - name: βš™οΈ Enable Corepack
      if: ${{ inputs.enable-corepack }} == 'true'
      shell: bash
      working-directory: ${{ inputs.cwd }}
      run: corepack enable  
  
    - uses: pnpm/action-setup@v2.2.4
      if: ${{ inputs.enable-corepack }} == 'false' 
      # If you're not setting the packageManager field in package.json, add the version here
      # with:
      #   version: 8.5.1

    - name: Expose pnpm config(s) through "$GITHUB_OUTPUT"
      id: pnpm-config
      shell: bash
      run: |
        echo "STORE_PATH=$(pnpm store path)" >> $GITHUB_OUTPUT

    - name: Cache rotation keys
      id: cache-rotation
      shell: bash
      run: |
        echo "YEAR_MONTH=$(/bin/date -u "+%Y%m")" >> $GITHUB_OUTPUT

    - uses: actions/cache@v3
      name: Setup pnpm cache
      with:
        path: ${{ steps.pnpm-config.outputs.STORE_PATH }}
        key: ${{ runner.os }}-pnpm-store-cache-${{ steps.cache-rotation.outputs.YEAR_MONTH }}-${{ hashFiles('**/pnpm-lock.yaml') }}
        restore-keys: |
          ${{ runner.os }}-pnpm-store-cache-${{ steps.cache-rotation.outputs.YEAR_MONTH }}-

    # Prevent store to grow over time (not needed with yarn)
    # Note: not perfect as it prune too much in monorepos so the idea
    #       is to use cache-rotation as above. In the future this might work better.
    #- name: Prune pnpm store
    #  shell: bash
    #  run: pnpm prune store

    - name: Install dependencies
      shell: bash
      run: pnpm install --frozen-lockfile --prefer-offline
      env:
        # Other environment variables
        HUSKY: '0' # By default do not run HUSKY install        

Workflow action

To use it in the workflows

    steps:
      - uses: actions/checkout@v3

      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v3
        with:
          node-version: ${{ matrix.node-version }}

      - name: πŸ“₯ Monorepo install
        uses: ./.github/actions/pnpm-install

Recommended .npmrc

Ensure you're running a recent pnpm version (or adapt)

## PNPM related ###############
## https://pnpm.io/npmrc      #
###############################

# Not always possible to be strict, but if it works for you, keep it to true.
# https://pnpm.io/next/npmrc#strict-peer-dependencies
strict-peer-dependencies=false

# Auto install peers should be false to avoid downloading
# extraneous deps. If the install fails, try first to explicitly add
# the missing deps in your package. Set it to true at last resort
# (when the problem comes from upstream dependencies). The best is false.
# https://pnpm.io/npmrc#auto-install-peers
auto-install-peers=false

# Helps locating duplicates, default in v8
# https://pnpm.io/next/npmrc#use-lockfile-v6
use-lockfile-v6=true

# Will fix duplicates due to peer-dependencies (>=7.29.0), default in v8
# https://github.com/pnpm/pnpm/releases/tag/v7.29.0
dedupe-peer-dependents=true

# Helps with peer-deps (>=7.23.0), default in v8
# https://pnpm.io/npmrc#resolve-peers-from-workspace-root
resolve-peers-from-workspace-root=true

# default to 'lowest' in v8.5.0
# set to highest for reasons specified here: https://github.com/pnpm/pnpm/issues/6463
# https://pnpm.io/npmrc#resolution-mode
resolution-mode=highest

# Default in 8.1.0 to fix issues with root/workspaces hoisting
# https://pnpm.io/npmrc#dedupe-direct-deps
dedupe-direct-deps=false

# Pinlock to exact version (default is '^')
# https://pnpm.io/npmrc#save-prefix
# see also how save-workspace-protocol affect this https://pnpm.io/npmrc#save-workspace-protocol
save-prefix=''

# Most of the time, you want to use the rolling protocol for monorepos
# https://pnpm.io/npmrc#save-workspace-protocol
save-workspace-protocol=rolling

Notes

Install

image

Post-install

image

Cleanup caches

When a PR is closed or merged the best is to remove install cache rather than letting github reach the max (10GB) and prune.

image

Link: https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#force-deleting-cache-entries

Here's an example (feel free to adapt if you need to preserse some things, ie gh actions-cache list -R $REPO -B $BRANCH | cut -f 1 | grep pnpm will only clear pnpm related caches)

# https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#force-deleting-cache-entries
name: Cleanup caches for closed branches

on:
  pull_request:
    types:
      - closed
  workflow_dispatch:

jobs:
  cleanup:
    runs-on: ubuntu-latest
    steps:
      - name: Check out code
        uses: actions/checkout@v3

      - name: Cleanup
        run: |
          gh extension install actions/gh-actions-cache

          REPO=${{ github.repository }}
          BRANCH="refs/pull/${{ github.event.pull_request.number }}/merge"

          echo "Fetching list of cache key"
          cacheKeysForPR=$(gh actions-cache list -R $REPO -B $BRANCH | cut -f 1 )

          ## Setting this to not fail the workflow while deleting cache keys. 
          set +e
          echo "Deleting caches..."
          for cacheKey in $cacheKeysForPR
          do
              gh actions-cache delete $cacheKey -R $REPO -B $BRANCH --confirm
          done
          echo "Done"
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment