Skip to content

Instantly share code, notes, and snippets.

@rishitells
Last active March 30, 2024 09:34
Show Gist options
  • Save rishitells/3c4536131819cff4eba2c8ab5bbb4570 to your computer and use it in GitHub Desktop.
Save rishitells/3c4536131819cff4eba2c8ab5bbb4570 to your computer and use it in GitHub Desktop.
Setting up Jest tests and coverage in GitLab CI

Configuring Jest Tests in GitLab CI

1. Add GitLab CI configuration file in the root

In the root of your project, add .gitlab-ci.yml with the configuration below.

image: node:latest

stages:
  - build
  - test

build:
  stage: build
  script:
    - yarn
  cache:
    paths:
      - node_modules/
  artifacts:
    expire_in: 1 days
    when: on_success
    paths:
      - node_modules/

test:
  stage: test
  coverage: /All files[^|]*\|[^|]*\s+([\d\.]+)/
  dependencies:
    - build
  script:
    - yarn test:ci
  artifacts:
    when: always
    reports:
      junit:
        - junit.xml
      cobertura: coverage/cobertura-coverage.xml

Note that We have cached the node_modules/ in build stage to make them available for subsequent jobs without having to download them again.

Pushing this to GitLab will automatically trigger the CI build. But before that, we'll add the required packages/configuration so that the build passes.

2. Install jest-junit package for reporting of tests

Next, we'll configure jest-junit, which will generate JUnit report format XML file (junit.xml) in the project root. GitLab will parse this XML format and then these reports can be viewed inside the pipelines details page, and also in the reports panel in Merge Requests. See the GitLab Unit test reports docs for more details.

npm install --save-dev jest-junit

OR

yarn add --dev jest-junit

3. Add coverageReporters and other required config to jest.config.js

From the GitLab Docs -

Collecting the coverage information is done via GitLab CI/CD’s artifacts reports feature. You can specify one or more coverage reports to collect, including wildcard paths. GitLab then takes the coverage information in all the files and combines it together.

For the coverage analysis to work, you have to provide a properly formatted Cobertura XML report to artifacts:reports:cobertura.

So we need to add Cobertura coverage reporter in jest.config.js for test coverage in GitLab Merge Requests

module.exports = {
  "collectCoverageFrom": ["src/**/*.js", "!**/node_modules/**"],
  "coverageReporters": ["html", "text", "text-summary", "cobertura"],
  "testMatch": ["**/*.test.js"]
}

Adding cobertura to coverageReporters will generate cobertura-coverage.xml inside /coverage/ folder created by Jest, and will be parsed by GitLab.

4. Add CI test run script to package.json with jest-junit reporter

{
  "scripts": {
    "test:ci": "jest --config ./jest.config.js --collectCoverage --coverageDirectory=\"./coverage\" --ci --reporters=default --reporters=jest-junit --watchAll=false"
  }
}

This script is used in the test stage in the .gitlab-ci.yaml file we created in step 1.

5. Modify GitLab Project CI/CD settings for test coverage parsing

Go to Project > Settings > CI/CD > General pipelines > Test coverage parsing Add the following RegEx -

Lines\s*:\s*(\d+.?\d*)%

This regular expression is used to find test coverage output in the job log. This coverage % can be viewed on Project > CI/CD > Jobs. We can also configure Badges on Project Overview page to show coverage % (see next step).

Finally, push all the changes to GitLab and you should see your pipeline up and running. Also in the subsequent Merge Requests, you should see the number of tests, failing tests (if any) and failure reason, and test coverage information infiles.

6. Show Pipeline and Coverage Badge on the Project Overview page

We can add Badges to the overview page of GitLab projects to display useful information such as pipeline status, current release version, test coverage percentage etc. Below is how we can configure and add Badges -

From GitLab Docs:

To add a new badge to a project:

  1. Navigate to your project’s Settings > General > Badges.
  2. Under “Link”, enter the URL that the badges should point to and under “Badge image URL” the URL of the image that should be displayed.
  3. Submit the badge by clicking the Add badge button.

Example project badge: Pipeline Status

A common project badge presents the GitLab CI pipeline status.

To add this badge to a project:

  1. Navigate to your project’s Settings > General > Badges.
  2. Under Name, enter Pipeline Status.
  3. Under Link, enter the following URL: https://gitlab.com/%{project_path}/-/commits/%{default_branch}
  4. Under Badge image URL, enter the following URL: https://gitlab.com/%{project_path}/badges/%{default_branch}/pipeline.svg
  5. Submit the badge by clicking the Add badge button.

In the similar way, we can add a coverage badge to project. Just replace pipeline.svg with coverage.svg in step 4 above.

7. Publish Jest Coverage Report to GitLab pages

We can publish our Jest coverage report (.html) to GitLab pages to view detailed Jest coverage report on a GitLab Pages URL.

To publish - modify .gitlab-ci.yml to add deploy stage for publishing the coverage report HTML to GitLab pages

image: node:latest

stages:
  - build
  - test
+ - deploy

build:
  stage: build
  script:
    - yarn
  cache:
    paths:
      - node_modules/
  artifacts:
    expire_in: 1 days
    when: on_success
    paths:
      - node_modules/

test:
  stage: test
  coverage: /All files[^|]*\|[^|]*\s+([\d\.]+)/
  dependencies:
    - build
  script:
    - yarn test:ci
+ cache:
+   paths:
+     - coverage/
  artifacts:
+   paths:
+     - coverage/
    when: always
    reports:
      junit:
        - junit.xml
      cobertura: coverage/cobertura-coverage.xml

+ pages:
+  stage: deploy
+  dependencies:
+    - test
+  script:
+    - mkdir .public
+    - cp -r coverage/* .public
+    - mv .public public
+  artifacts:
+    paths:
+      - public
+  only:
+    - master

After pushing the changes, when the deploy step is successful in pipeline, We can access the Jest coverage report page using the URL mentioned in Project > Settings > Pages.

Each time the deploy job runs, a new coverage report will be published to the GitLab pages URL. Note that we have published coverage report to Pages only for master branch, because we don't want all branch commits to publish coverage report.

References

  1. GitLab CI/CD process overview - GitLab Docs
  2. Unit test reports - GitLab Docs
  3. Test Coverage Visualization - GitLab Docs
  4. How to display code coverage of a Vue project in Gitlab
@makirby
Copy link

makirby commented Aug 3, 2021

This was incredibly helpful to get up and running quickly. Thankyou for posting!

@rishitells
Copy link
Author

This was incredibly helpful to get up and running quickly. Thankyou for posting!

I am glad it was helpful Merrick :)

@refaelsh
Copy link

refaelsh commented Oct 3, 2021

This helped me a lot. Thank you! :-)

@vishalp-integrella
Copy link

this script will fail to publish report if any test fails

@rishitells
Copy link
Author

rishitells commented Oct 11, 2021

this script will fail to publish report if any test fails

It's not desirable to publish the test coverage reports in a production setup if the tests are failing, because -

  1. Running coverage report publish step on all commits irrespective of whether the tests are failing/passing would make it run many times unnecessarily. The first thing you would want to do if your tests are failing is to fix them on priority and get pipelines green again, as the failing pipeline would be blocking the rest of the team from deploying their code. If your tests are failing - the reports won't give correct coverage information anyways, so it wouldn't be useful to look at them while the test pipelines are red.
  2. Collecting and publishing reports is slow, so it's better to run them only if previous jobs are green. Running them on every commit would make the pipelines slow as well.

@tjerkw
Copy link

tjerkw commented Nov 1, 2021

Super helpfull! Thanks muchos

@SihemBouhenniche
Copy link

Thanks a lot, it's so helpful

@rordi
Copy link

rordi commented Feb 28, 2022

Great and helpful, thanks a lot!

@RezaRahmati
Copy link

Super helpful,
I don't see "Project > Settings > Pages", even after successful step, not sure why

@jose-83
Copy link

jose-83 commented Apr 20, 2022

Super helpful, I don't see "Project > Settings > Pages", even after successful step, not sure why

Check your access right! As a developer, you probably won't be able to see that. Check with your Gitlab Administrator and ask for a maintainer access right.

@patricktunez123
Copy link

I'm using pure React instead of next, will this work the same for both create-react-app and next projects?

Because with pure React I'm getting the 'Unkown coverage' and I don't know what I might be doing wrong.

cov

@ismoil793
Copy link

Hello, awesome guide, can you suggest any tips for the following task:
I need to show the coverage test difference between the source branch and the target branch.
How can I accomplish this?

@chaodonghu
Copy link

chaodonghu commented Apr 30, 2022

Hello, awesome guide, can you suggest any tips for the following task:
I need to show the coverage test difference between the source branch and the target branch.
How can I accomplish this?

@ismoil793 I had trouble figuring this out as well but finally accomplished it a couple of days ago. If you're referring the the (+/- %) change that you want to show on the MR like this.

Screen Shot 2022-04-30 at 11 15 32 AM

You would want to run your test job on the main (source branch) and all merge requests (https://docs.gitlab.com/ee/ci/jobs/job_control.html#specify-when-jobs-run-with-only-and-except)

Something like

  only:
    - production
    - main
    - merge_requests

@weckx
Copy link

weckx commented May 24, 2022

Very nice work, thank you very much. Just an update, for Gitlab 15.0+ the Project > Settings > CI/CD > General pipelines > Test coverage parsing setting does not exist anymore, so just the coverage property on the test job in the .gitlab-ci.yaml is enough.

@smith558
Copy link

smith558 commented Jun 29, 2022

The line cobertura: coverage/cobertura-coverage.xml should be replaced with

coverage_report:
    coverage_format: cobertura
    path: coverage/cobertura-coverage.xml

@nitinmagdumvista
Copy link

Has anyone implemented this
https://gitlab.com/gitlab-org/gitlab/-/issues/6284

I want to set minimum coverage to 80% and after every new commit and build if the new coverage is below 80% i want to fail the job so changes will not be able to merge and developer increase code coverage

@ismoil793
Copy link

ismoil793 commented Jul 21, 2022

Has anyone implemented this https://gitlab.com/gitlab-org/gitlab/-/issues/6284

I want to set minimum coverage to 80% and after every new commit and build if the new coverage is below 80% i want to fail the job so changes will not be able to merge and developer increase code coverage

Hi, for this case you need to add some configuration to your jest.config
First of all you should run test coverage and see what you are getting from it (branches, functions, lines).
Then, add configuration accordingly. I guess you are asking about lines coverage to 80%
Would be smth like this:

coverageThreshold: { global: { branches: WhateverPercentYouWant, functions: WhateverPercentYouWant, lines: 80, } }

@nitinmagdumvista
Copy link

@caasi
Copy link

caasi commented Aug 4, 2022

@smith558 thank you!

@nford88
Copy link

nford88 commented Aug 31, 2022

Brilliant guide! Trying to see if there is a way of deploying coverage pages per branch. Using the guide above the public folder is constantly overwritten every time the job is triggered so even if you set up coverage in a subdirectory it will not persist.

Was thinking of having a job which pushed coverage/BRANCH_NAME directory to a "coverage" branch when and configuring the pages job to only run and deploy on branch "coverage" but it seems hassle setting up access with git ssh keys etc. GitHub Actions works around but for me we are using GitLab so do not have these helper utils.

Anybody found a similar solution?

@erperejildo
Copy link

erperejildo commented Mar 22, 2023

The last part was deprecated from Gitlab. You should now use the coverage setting in your Gitlab CI/CD configuration file:

unit-test:
  stage: test
  coverage: '/coverage: \d+.\d+% of statements/'
  script:
    - go test -cover

But you already added this coverage with Lines\s*:\s*(\d+.?\d*)%. Does it mean we don't need that part of the configuration?

More info: https://stackoverflow.com/a/72735773/4858133

cobertura was deprecated as well. No should be:

coverage_report:
    coverage_format: cobertura
    path: coverage/cobertura-coverage.xml

More info: https://stackoverflow.com/a/72173213/4858133

@AndreyTheWeb
Copy link

@chaodonghu Hi, сan you please tell us in more detail how you managed to achieve the result as in the screenshot?

@liweijian
Copy link

liweijian commented Jun 6, 2023

@chaodonghu Hi, сan you please tell us in more detail how you managed to achieve the result as in the screenshot?

@AndreyTheWeb It sounds a lot like we could get some hints from this tutorial https://dev.to/muhamadhhassan/adding-phpunit-test-log-and-coverage-to-gitlab-cicd-33b5

@beesaferoot
Copy link

I successfully used this to setup coverage report for our project on Gitlab. Thank so much for the guide!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment