Skip to content

Instantly share code, notes, and snippets.

@imhoffd
Last active April 19, 2024 19:46
Show Gist options
  • Save imhoffd/b4ca0a94c2496d81303ebb00063a863d to your computer and use it in GitHub Desktop.
Save imhoffd/b4ca0a94c2496d81303ebb00063a863d to your computer and use it in GitHub Desktop.
Parallelizing Jest in GitHub Actions
name: CI
on: [push]
jobs:
setup:
runs-on: ubuntu-latest
outputs:
test-chunks: ${{ steps['set-test-chunks'].outputs['test-chunks'] }}
test-chunk-ids: ${{ steps['set-test-chunk-ids'].outputs['test-chunk-ids'] }}
steps:
- uses: actions/checkout@v2
- run: npm install
- id: set-test-chunks
name: Set Chunks
run: echo "::set-output name=test-chunks::$(npx jest --listTests --json | jq -cM '[_nwise(length / 2 | ceil)]')"
- id: set-test-chunk-ids
name: Set Chunk IDs
run: echo "::set-output name=test-chunk-ids::$(echo $CHUNKS | jq -cM 'to_entries | map(.key)')"
env:
CHUNKS: ${{ steps['set-test-chunks'].outputs['test-chunks'] }}
test:
runs-on: ubuntu-latest
name: test (chunk ${{ matrix.chunk }})
needs:
- setup
strategy:
matrix:
chunk: ${{ fromJson(needs.setup.outputs['test-chunk-ids']) }}
steps:
- uses: actions/checkout@v2
- run: npm install
- name: jest
run: echo $CHUNKS | jq '.[${{ matrix.chunk }}] | .[] | @text' | xargs npx jest
env:
CHUNKS: ${{ needs.setup.outputs['test-chunks'] }}
@havenchyk
Copy link

oh and one more: you need to run npm install at least 3 times, do you think it's faster than jest?

@imhoffd
Copy link
Author

imhoffd commented Feb 22, 2022

@havenchyk

  1. My understanding is the check has to actually run first, then it will appear in Repo Settings -> Branches -> Edit -> Require status checks to pass before merging automatically.
  2. I think you may be conflating concurrency with parallelism. Jest will automatically use up the available cores on the VM. This script will run jest tests on separate VMs entirely.
  3. You can cache your node_modules directory in Github Actions if you find it's becoming a problem. There will never be a silver bullet for "fastest performance". You have to do your own testing for your own test suite.

@havenchyk
Copy link

Thanks, @imhoffd! I managed to solve those problems with caching, as for required checks, I just made chunks static, so I know how many of those I'm going to use

@imhoffd
Copy link
Author

imhoffd commented Mar 4, 2022

FYI, Jest is potentially getting a --shard option which makes this way easier: jestjs/jest#12546

@havenchyk
Copy link

Thanks! seems it will be available in jest@28

@finalight
Copy link

i got this error when using the above github actions

Error reading JToken from JsonReader. Path '', line 0, position 0.,.github/workflows/chunks.yml (Line: 75, Col: 16): Unexpected type of value '', expected type: Sequence.

any idea how to resolve this?

@CharlieGreenman
Copy link

Do you have to specify the length manually, or is there a way to set up that automatically? For instance, for every 100 tests, create a new test chunk

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment