Skip to content

Instantly share code, notes, and snippets.

@ArturT
Last active December 5, 2023 12:01
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ArturT/8c901ebfa773fc934a7d30cf1de26558 to your computer and use it in GitHub Desktop.
Save ArturT/8c901ebfa773fc934a7d30cf1de26558 to your computer and use it in GitHub Desktop.
CircleCI rerun failed tests in RSpec with Knapsack Pro Queue Mode. Thanks to that you can rerun only failed tests instead of rerunning the whole CI build. https://docs.knapsackpro.com/ruby/circleci/ & https://knapsackpro.com
- run:
name: Run tests
command: |
export CIRCLE_TEST_REPORTS=/tmp/test-results
mkdir -p $CIRCLE_TEST_REPORTS
# split slow spec files by test examples
# https://docs.knapsackpro.com/ruby/split-by-test-examples/
export KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES=true
# Use circleci CLI to find out if we need to run all tests or rerun failed tests
# We are telling circleci to split the tests across 1 node to get the full list of all tests for consideration. We leave the splitting to Knapsack Pro.
export KNAPSACK_PRO_TEST_FILE_LIST_SOURCE_FILE=/tmp/tests_to_run.txt
circleci tests glob "spec/**/*_spec.rb" | circleci tests run --index 0 --total 1 --command ">$KNAPSACK_PRO_TEST_FILE_LIST_SOURCE_FILE xargs -n1 echo" --verbose
# tell Knapsack Pro to run only tests from the file (and still use Knapsack Pro Queue Mode magic)
bundle exec rake "knapsack_pro:queue:rspec[--format documentation --format RspecJunitFormatter --out tmp/rspec.xml]"
@ArturT
Copy link
Author

ArturT commented Dec 1, 2023

@sethherr
Copy link

sethherr commented Dec 4, 2023

You should probably use the CIRCLE_NODE_TOTAL env variable rather than hard coding 1

RUNNER_TOTAL=$(($CIRCLE_NODE_TOTAL - 1))
circleci tests glob "spec/**/*_spec.rb" | circleci tests run --index 0 --total $RUNNER_TOTAL --command ">/tmp/tests_to_run.txt xargs echo" --verbose > /tmp/tests_to_run.txt

@ArturT
Copy link
Author

ArturT commented Dec 4, 2023

You should probably use the CIRCLE_NODE_TOTAL env variable rather than hard coding 1

RUNNER_TOTAL=$(($CIRCLE_NODE_TOTAL - 1))
circleci tests glob "spec/**/*_spec.rb" | circleci tests run --index 0 --total $RUNNER_TOTAL --command ">/tmp/tests_to_run.txt xargs echo" --verbose > /tmp/tests_to_run.txt

@sethherr That's incorrect.

You need to specify --total 1 on purpose to feed Knapsack Pro Queue API with list of all test files. Only then tests could be properly distributed across CI nodes by Knapsack Pro.

KNAPSACK_PRO_TEST_FILE_LIST_SOURCE_FILE expects list of all test files. Each CI node must provide exactly the same list of tests. That's why we use --total 1.

@sethherr
Copy link

sethherr commented Dec 4, 2023

Ahhhh. Ok, thanks for explaining.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment