Skip to content

Instantly share code, notes, and snippets.

@chetanyakan
Last active February 20, 2021 16:41
Show Gist options
  • Save chetanyakan/793a86faac8a58bb7539bfa1af9456d9 to your computer and use it in GitHub Desktop.
Save chetanyakan/793a86faac8a58bb7539bfa1af9456d9 to your computer and use it in GitHub Desktop.
Attributes of a High Quality Unit Test

Attributes of a High Quality Unit Test

Attributes of a high quality unit test

Key attributes of a high quality unit test:

  • Narrow Scope - each unit test should follow single responsibility, testing a single activity of a method.
  • Is Deterministic - each unit test should execute exactly the same way every time it is run
  • It Must fail - if a logic change is introduced to the code under test, the unit test must fail

Key attributes of a high quality test suite:

  • Class Scope - each test suite should cover the full set of activities of a specific class

Unit Test has Narrow Scope

What are the characteristics:

  • Single responsibility - each unit test should cover a single activity
  • Each test should define its scope using the should/do/when (and given/xx/yy) naming convention Why it’s important:
  • It simplifies the test cases and makes them easier to understand and maintain
  • Makes it easy to pinpoint what has failed
  • Minimizes test execution time How is this measured:
  • Proven by review by QE team

Unit Test is Deterministic

What are the characteristics:

  • Unit test executes passes or fails exactly the same way every time it is run
  • Mocks are created for all external dependencies in the class under test Why it’s important:
  • CI/CD execution will become unreliable if unit tests are dependent on external factors such as database availability, execution of external rest APIs or environmental configurations
  • It becomes very difficult to pin failures if unit tests start failing due to changes in dependent classes (if another class fails, only it’s unit tests should fail). How this is measured:
  • Accidental coverage: though an imperfect measure, low accidental coverage implies a low probability of non-deterministic failure. It does not guarantee it.
  • Repeated execution in an isolated environment

Unit Test must fail

What are the characteristics:

  • Any change to the logic of the code under test must cause a unit test failure.
  • It must not be brittle; code changes that preserve existing logic must not cause the test to fail (e.g. it must not break encapsulation by checking the presence or state of a specific internal variable) Why it’s important:
  • If a bug is introduced and does not cause the unit test to fail, then the unit test does not provide any value How this is measured:
  • Mutation coverage injects defects into code and measures whether unit tests fail as a result.

Techniques for writing great unit tests with - Javascript

Introduction

The article outlines best practices applied to the unit testing with javascript language and illustrates them by an example. We are going to focus on the client-side javascript code dependent on the angular js framework. The test uses Jasmine and Karma test frameworks, however, the setup of the project and framework is out of the scope of this article. The information about the setup could be found on the framework’s official pages. It is important to note that patterns and practices could be successfully applied to other testing frameworks, such as Jest.

Creation of a High-Quality Unit Test

Creation of the test file

The size of your project is going to grow over time. The same applies to the number of unit tests you have. It is important to organize your test suite in order to be able to efficiently manage and maintain the tests in the future. While the creation of the tests might look like a trivial task, there are a few best practices we suggest to follow from the beginning. First, the file name should match for the top-level test suite name. In our example we are going to write a test case for the 'team-table' angular module, so out test suite would contain the following code:

describe('team-table', function() {
	...
}

Correspondingly, we are going to name our test spec file as “team-table.spec.js”. Now, whenever we would need to make any change to the team-table suite, we would know the test spec file named the same. The second rule we precisely follow says that there is one top-level test suite per file.

Use a proper naming

It is worth spending a minute to create a meaningful name or title for a suite or a test when it is written. It helps other developers a lot to understand what the tests do and what features they assert. Together with a code coverage report, the list of the properly named test could become a profound way of figuring out the covered parts of your application. Note, that it is possible to embed one suite into another:

describe('mpmApplicationView', function() {
    describe('open', function() {
      it('should set scope.selectedMember', function() {
         . . .
      }
   }
}

Following the rule results in a nice human-readable test execution summary:
PASS  spec/team-table.spec.js
  team-table
    initialization
       should be initialized successfully (98ms)
    mpmApplicationView
      open
         should set scope.selectedMember (18ms)
      close
         should set scope.selectedMember to null (16ms)
      switchMpm
         should set scope.selectedMember to next item in assignments when dir is 1 (16ms)
         should not change scope.selectedMember to next item in assignments when dir is 1 and next item is not present (17ms)
         should set scope.selectedMember to next item in assignments when dir is 1 (17ms)
      reject
         should remove scope.selectedMember from controller.assignments (27ms)
      getCurrentTime
         should call TimeUtils.getUTC (21ms)
      $document click
         should set controller.UI.showSeatOptions to null (21ms)

Follow Arrange, Act, and Assert (AAA) methodology

"Arrange-Act-Assert” is a pattern for arranging and formatting code in UnitTest methods. It suggests that you should divide your test method into three sections: arrange, act, and assert. Each section is only responsible for the part in which they are named after: “Arrange” arranges all necessary preconditions and inputs, “Act” performs an action on the object or method under test, and finally “Assert” ensures that the expected results have occurred. Typically the sections separated using comment and a line break, for example:

    describe('open', function() {
      it('should set scope.selectedMember', function() {
        // Arrange
        var assignment = { selection: { marketplaceMember: { id: 1 } } };

        // Act
        controller.mpmApplicationView.open(assignment);

        // Assert
        expect(scope.selectedMember).toEqual(
          assignment.selection.marketplaceMember
        );
      });
    });

Arrange-Act-Assert formatted test is way more lucid. What is being tested is clearly separated from the setup and verification steps. The developer is not allowed to alternate actions and assertions as the pattern forces the single act block per test block to be followed by the single assert block. Therefore the pattern is also a form of developer self-control as it makes it harder to create cases which test too many things at once.

Assert a single functionality per test case

A spec (test case) should test only “one” expectation. Prefer a large number of small test cases over a few long and complex specs. It works very well together with following the AAA pattern and suite grouping techniques. Having a large number of tests with meaningful names allows developers to identify, understand, and localize an issue in the code much faster:

  describe('mpmApplicationView', function() {
    describe('open', function() {
      it('should set scope.selectedMember', function() {
        // Arrange
        var assignment = { selection: { marketplaceMember: { id: 1 } } };

        // Act
        controller.mpmApplicationView.open(assignment);

        // Assert
        expect(scope.selectedMember).toEqual(
          assignment.selection.marketplaceMember
        );
      });
    });
    describe('close', function() {
      it('should set scope.selectedMember to null', function() {
        // Arrange

        // Act
        controller.mpmApplicationView.close();

        // Assert
        expect(scope.selectedMember).toBe(null);
      });
    });

It would be also required to refactor existing testes during the product life cycle, for example when a new feature is developed. Having small and nicely formatted tests makes it is much easier to achieve the same

Use setup and teardown methods

Modern testing frameworks provide you a set of setup and teardown methods like beforeEach() and afterEach() that you can use to put an arrangement and cleaning code required for a group of tests. In javascript, it is allowed to define setup and teardown methods for every suite, even for embedded ones. This feature allows to define preparation and cleanup behaviour in the flexible way:

    beforeEach(function() {
      element = angular.element(
        '<team-table assignments="assignments" seat-controls="seatControls"></team-table>'
      );
      $compile(element)($scope);
      $scope.$digest();
      scope = element.isolateScope();
      controller = element.controller('teamTable');
    });

Usage of the beforeEach method allows you to simplify the content of the “Arrange”. Avoid putting the duplicate code in the “Arrange” section of multiple methods, use the method beforeEach instead.

Mock the dependencies

A good unit test is designed to test the module in isolation. However, most of the modules have a dependency on other modules. In real production projects, such dependencies could be very complex. While test-driven code is usually designed to be unit tested easily, it is of then not the case when writing the tests post factum, especially for the legacy applications. In such cases, the mocking technique comes very handily. It allows replacing real dependencies with fake, often generated instances. The fake object could be used to simulate the required behavior and validate the usage of the dependency by the class under the test. In the second case, spying a dependency could be a better option. Javascript is a weak typing language and as a result, there are a variety of ways to perform mocking. The most commonly used are provided by jasmine and jest itself:

     module(function($provide) {
      $provide.value('routePrefix', jasmine.createSpy());
      TimeUtils = {
        getUTC: jasmine.createSpy()
      };
      $provide.value('TimeUtils', TimeUtils);
    });

Unit test quality metrics

The coverage does matter

One reason why people write tests is to prevent regressions. The test suite can efficiently prevent them from happening only when it covers the functionality of the application to a sufficient degree. Coverage metrics help to keep track of the parts of the code being validated by the test suite you maintain. There are a number of them, but most of the teams focus on the line coverage and branch coverage metrics. Line coverage identifies which lines were encountered as a result of your tests. Branch coverage is a little less intuitive metric. In a nutshell, it measures the fraction of conditional operations outcomes tested by the test suite over the number of all possible outcomes. Coverage metrics for great unit tests are expected to be above the 90% threshold.

Modern testing frameworks allow you to measure the coverage out of the box and generate a nice report in a command line or in the form of the HTML page. The unit test we are using as an example has a great level of 100% line coverage. The branch coverage could, however, be improved as it is only 75%.

Evaluate your tests with a mutation framework

The code coverage metric can tell you which parts of your codebase are executed during your test run, however, it tells nothing about the quality of the assertions made. It is indeed possible to achieve 100% coverage without making a single assert statement! It is obvious that such a test is almost useless for regression detection despite its high metric value.

Luckily, a mutation framework is a way to measure the ability of the test suite to capture defects. Stryker is the example of the mutation frameworks supporting a variety of languages, including javascript. The idea of mutation testing is to generate a set of so-called mutants. Each mutant is a copy of the module under the test with the single minor code change. The changes are made automatically by the mutation framework. The test suite is applied to a codebase where the class is replaced with a mutated version of it. The mutant is considered as surviving if there are no failed test cases. As a result, the mutant survival rate is measured and used as a metric.

Our example test suite has a 68.75% mutation score. Low branch coverage values could be one of the reasons for the low mutation rate and could indicate the need to improve the number of the tests and the number of assert statements over the test suite.

The main disadvantage of mutation testing is it’s duration. A test should be executed as many times as the number of generated mutants which will take a significant amount of time, especially for the huge codebase.

Unit Testing in React

Jest and Enzyme are the main framework and testing utilities used in testing components and utility files in React. Please visit their respective documentation for detailed information on how to get started, best practices and updates.

Running unit tests

# Run all unit tests
npm run test

# Run unit tests in watch mode
npm run test:watch

Introduction to Jest

For unit tests, we use Jest with the describe/expect syntax. If you're not familiar with Jest, you can read about:

Unit test files

Configuration for Jest is in jest.config.js, support files are in src/tests, but as for the tests themselves - they're first-class citizens. That means they live alongside our source files, using the same name as the file they test, but with the extension .test.ts.

This may seem strange at first, but it makes poor test coverage obvious from a glance, even for those less familiar with the project. It also lowers the barrier to adding tests before creating a new file, adding a new feature, or fixing a bug.

Unit test mocks

Jest offers many tools for mocks, including:

Component Testing

  1. Match snapshot using default or expected props. Note that while the snapshot is convenient, we require not to rely solely on this for every test case as this is easily overlooked by initiating jest -updateSnapshot without carefully inspecting the change.

    const baseProps = {
        activeSection: 'email',
        onSubmit: jest.fn(),
        updateSection: jest.fn(),
    };
    
    test('should match snapshot, not send email notifications', () => {
        const wrapper = shallow(<EmailNotificationSetting {...baseProps}/>);
    
        // Use "toMatchInlineSnapshot" whenever possible when the snapshot consists of several lines of code only
        // It creates an easier to read snapshot, inline with the test file.
        expect(wrapper).toMatchInlineSnapshot();
    
        // Save snapshot particularly when component has other render function like "renderOption"
        // It creates a small snapshot of that particular render function instead of the entire component
        expect(wrapper.instance().renderOption()).toMatchInlineSnapshot();
    
        // Only use "toMatchSnapshot" whenever above options are not possible.
        // Limit the use to one (1) snapshot only.
        // Save snapshot if it generates an easy to inspect and identifiable HTML or components that can easily verify future change.
        expect(wrapper).toMatchSnapshot();
    });
  2. Add verification to important elements.

    expect(wrapper.find('#emailNotificationImmediately').exists()).toBe(true);
    expect(wrapper.find('h1').text()).toEqual(props.siteName);
    expect(wrapper.find('h4').text()).toEqual(props.customDescriptionText);
  3. Check CSS class.

    expect(wrapper.find('#create_post').hasClass('center')).toBe(true);
  4. Simulate the event and verify state changes accordingly.

    test('should pass handleChange', () => {
        const wrapper = mountWithIntl(<EmailNotificationSetting {...baseProps}/>);
        wrapper.find('#emailNotificationImmediately').simulate('change');
    
        expect(wrapper.state('enableEmail')).toBe('true');
        expect(wrapper.state('emailInterval')).toBe(30);
    });
  5. Ensure that all functions of a component are tested. This can be done via events, state changes or just calling it directly.

    test('should call updateSection on handleExpand', () => {
        const newUpdateSection = jest.fn();
        const wrapper = mountWithIntl(
            <EmailNotificationSetting
                {...baseProps}
                updateSection={newUpdateSection}
            />
        );
        wrapper.instance().handleExpand();
    
        expect(newUpdateSection).toBeCalled();
        expect(newUpdateSection).toHaveBeenCalledTimes(1);
        expect(newUpdateSection).toBeCalledWith('email');
    });
  6. When a function is passed to a component via props, make sure to test if it gets called for a particular event call or its state changes.

    test('should call functions on handleSubmit', () => {
        const newOnSubmit = jest.fn();
        const newUpdateSection = jest.fn();
        const wrapper = mountWithIntl(
            <EmailNotificationSetting
                {...baseProps}
                onSubmit={newOnSubmit}
                updateSection={newUpdateSection}
            />
        );
    
        wrapper.instance().handleSubmit();
    
        expect(newOnSubmit).not.toBeCalled();
        expect(newUpdateSection).toHaveBeenCalledTimes(1);
        expect(newUpdateSection).toBeCalledWith('');
    
        wrapper.find('#emailNotificationNever').simulate('change');
        wrapper.instance().handleSubmit();
    
        expect(newOnSubmit).toBeCalled();
        expect(newOnSubmit).toHaveBeenCalledTimes(1);
        expect(newOnSubmit).toBeCalledWith({enableEmail: 'false'});
    
        expect(savePreference).toHaveBeenCalledTimes(1);
        expect(savePreference).toBeCalledWith('notifications', 'email_interval', '0');
    });
  7. Test the component's internal or lifecycle methods by having different sets of props.

    test('should pass componentWillReceiveProps', () => {
        const nextProps = {
            enableEmail: true,
            emailInterval: 30
        };
        const wrapper = mountWithIntl(<EmailNotificationSetting {...baseProps}/>);
        wrapper.setProps(nextProps);
    
        expect(wrapper.state('enableEmail')).toBe(nextProps.enableEmail);
        expect(wrapper.state('emailInterval')).toBe(nextProps.emailInterval);
    
        ...
        const shouldUpdate = wrapper.instance().shouldComponentUpdate({show: true});
        expect(shouldUpdate).toBe(true);
    });
  8. Provide a mockup of a function required by the component but also pass other exported functions out of it to prevent potential error when those were used indirectly by another functions.

    jest.mock('utils/utils', () => {
        const original = require.requireActual('utils/utils');
        return {
            ...original,
            isMobile: jest.fn(() => true),
        };
    });
  9. For utility functions, list all test cases with test description, input and output.

    describe('stripMarkdown | RemoveMarkdown', () => {
    const testCases = [{
        description: 'emoji: same',
        inputText: 'Hey :smile: :+1: :)',
        outputText: 'Hey :smile: :+1: :)',
    },
    {
        description: 'at-mention: same',
        inputText: 'Hey @user and @test',
        outputText: 'Hey @user and @test',
    }];
    
    testCases.forEach((testCase) => it(testCase.description, () => {
        expect(stripMarkdown(testCase.inputText)).toEqual(testCase.outputText);
    }));
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment