Skip to content

Instantly share code, notes, and snippets.

@cenfun
Last active January 28, 2024 10:34
Show Gist options
  • Save cenfun/0754c23f03d1aa541b4467920ab4d09f to your computer and use it in GitHub Desktop.
Save cenfun/0754c23f03d1aa541b4467920ab4d09f to your computer and use it in GitHub Desktop.
Embracing Native V8 Coverage Reports in 2024

Embracing Native V8 Coverage Reports in 2024

Background and Current Situation

As early as 2017, V8 started offering native support for coverage data, as seen here However, we have essentially only been able to see coverage reports from Istanbul, and this is because most testing tools convert native V8 coverage data into the format of Istanbul, such as Jest, Vitest. This has led to several issues:

  • Native Byte coverage statistics are not available.
  • Coverage support for runtime code, especially minified code, is not available.
  • CSS coverage reports are not available.
  • The commonly used conversion tool is v8-to-istanbul, but it has a few problems here

Introducing Native V8 Coverage Reports

Nowadays, native V8 coverage reports are possible, and you can directly use V8 coverage data to generate V8 coverage reports. This can be achieved using the tool monocart-coverage-reports. More details can be found at the GitHub repository

v8

A Brand-New V8 Coverage Report User Interface

  • It provides a superior user interaction experience.
  • It offers file tree grouping and supports search and filter functions.
  • Supports an interface that lets you view and quickly locate uncovered code blocks.
  • It provides the positioning of the code with the highest execution number, making it easier for you to identify the most frequently executed code (which may be crucial areas affecting performance).
  • It provides markers for blocks of code run multiple times, as well as additional statistics for blank lines and comment lines.

Support for Native Byte Statistics

Byte coverage is natively supported by V8, displaying how many bytes of a file are covered. This is an excellent measure of the completeness of coverage.

image

Coverage for Any Runtime Code

Since the runtime code is most likely to be compressed and minified, 'monocart-coverage-reports' provides a formatter function, allowing easy access to coverage of formatted code. An asynchronous worker provides support for large volume files, which also includes syntax highlighting.

image

CSS Coverage Support

In today's front-end projects, CSS code is crucial, making checking CSS coverage quite important. This makes it easier to find and remove uesless CSS code.

image

Attempting to Solve the Conversion Problems from v8-to-istanbul

More details can be found here

How to Collect Native V8 Coverage Data

  • On the browser side (Chromium only) Use the Chromium Coverage API to collect coverage data. For example, Playwright Coverage Class
await Promise.all([
    page.coverage.startJSCoverage({
        resetOnNavigation: false
    }),
    page.coverage.startCSSCoverage({
        resetOnNavigation: false
    })
]);
await page.goto(url);
const [jsCoverage, cssCoverage] = await Promise.all([
    page.coverage.stopJSCoverage(),
    page.coverage.stopCSSCoverage()
]);

const coverageData = [... jsCoverage, ... cssCoverage];

For more detailed introductions please visit this link

How to Generate Native V8 Coverage Reports

First, create a new coverage report coverageReport. Next, add the collected V8 coverage data such as coverageData1 and coverageData2 to coverageReport. Finally, execute generate() to generate the report. The coverage data of the same file will automatically combine together.

const MCR = require('monocart-coverage-reports');
const options = {
    outputDir: './coverage-reports',
    reports: "v8"
}
const coverageReport = MCR(options);
await coverageReport.add(coverageData1);
await coverageReport.add(coverageData2);
const coverageResults = await coverageReport.generate();
console.log(coverageResults.summary);

Other

  • In fact, monocart-coverage-reports also fully supports all Istanbul coverage reports and integrates it with V8 reports. The report formats include: v8, v8-json,console-summary, clover,cobertura,html,html-spa,json,json-summary,lcov,lcovonly,none,teamcity,text,text-lcov,text-summary. see Available reports
  • An onEnd hook is also provided to enable users for custom programming, like checking whether the coverage has reached the required thresholds, see example
  • If you use @playwright/test as your automation test tool, you can directly use the monocart-reporter custom report as it has already integrated native V8 coverage report features. More details can be found here.
@AriPerkkio
Copy link

I'm mostly thinking about a complete rewrite of v8-to-istanbul with your approach. Apply all the converting logic you've come up with, and finally output everything in Istanbul format.

Maybe the format could also be extended to include the bytes metric. Then in future new reporters could show that if it's present in the output.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment