e2e-summary

paragon-e2e-summary.md

Paragon Admin - E2E

The following is a summary/report from the recent refactoring of the e2e test cases. You will find recommendations, and important information on how to handle specific test scenarios.

Case: addNewLicense

This helper script had 3 pause calls, the following explains the solution for each pause:

• After .click('.e2e-license-modal-btn')
  • This click triggered a modal to be opened, instead of pausing the script we can wait for the modal to be visible, this can be done through the .licenseDialogBox class which is added to the wrapper of the modal body.
• After .click('.e2e-state-select-0')
  • This click opens a material-ui SelectField dropdown. The dropdown relys on data to generate the options list, we can fix this by appending classes depending on the state of the component (ie: append a class when the options are ready to render in the SelectField component). With this in place we can use the class and the waitForElementNotVisible command, until the condition is satisfied we go ahead and trigger the click.
  • Another approach is to use the disabled state of the dropdown, the dropdown should be disabled until the data is ready, same concept as above just uses a different attribute.
  • Animations, this component provides built in animations, for this we can use a custom command called waitForAnimation, this will allow the animation to complete. Keep in mind this should only be used in cases where we can't control the state of the animation.
• After .click('.e2e-state-nc-0')
  • The "License Type" dropdown depends on the context from the selected option in the "License Jurisdiction" dropdown. We need to make sure dropdowns that depend on context from other elements should be disabled until the required context is populated. By disabling the dropdown, or adding specific classes based on the disabled state, we can use waitForElementNotVisible or waitForElementVisible before a click is triggered.
    • Same as above for the dropdown close/open animation we can rely on waitForAnimation, since material-ui's SelectField component doesn't provide any class when the dropdown is disabled, everything is handled through inline css.

Case: clearValue

This method lost scope of the test chain

This method clears the value of an input (TextField component), the problem here was that the test chain was broken everytime this helper was called.

Original implementation:

export function clearValue(browser, selector) {
  browser.getValue(selector, result => {
    for (const c in result.value) {
      if (c) {
        browser.setValue(selector, '\\u0008');
      }
    }
  });
}

Original use case:

export function editCustomer(browser, licenseNumber) {
  browser
    .waitForElementVisible('.e2e-edit-customer-btn')
    .click('.e2e-edit-customer-btn')
    .pause(500)
    .waitForElementVisible('.e2e-license-modal-btn');
  clearValue(browser, 'input[name="e2e-customer-first-name"]');
  clearValue(browser, 'input[name="e2e-customer-middle-name"]');
  clearValue(browser, 'input[name="e2e-customer-last-name"]');
  clearValue(browser, 'input[name="e2e-customer-address"]');
  clearValue(browser, 'input[name="e2e-customer-phone"]');
  browser
    .pause(500)
    .setValue('input[name="e2e-customer-first-name"]', 'Leo')
    .setValue('input[name="e2e-customer-last-name"]', 'Qiu')
  clearValue(browser, 'input\[name="e2e-license-form-number"\]');
  ...

Refactored:

export function clearValue(browser, selector, done) {
  browser.getValue(selector, result => {
    let count = 0;
    for (const character in result.value) {
      if (character) {
        browser.setValue(selector, '\u0008');
      }
      count++;
    }
    if (count === result.value.length) {
      done();
    }
  });
}

async function clearBatchValues(browser, selectors, done) {
  await Promise.all(
    selectors.map(selector => {
      return new Promise(resolve => {
        clearValue(browser, selector, resolve);
      });
    })
  );
  done();
}

Refactored use case:

export function editCustomer(browser, licenseNumber) {
  return browser
    .waitForElementVisible('.e2e-edit-customer-btn')
    .click('.e2e-edit-customer-btn')
    .waitForElementVisible('.e2e-license-modal-btn', 5500)
    .perform((client, done) => {
      const selectors = [
        'input[name="e2e-customer-first-name"]',
        'input[name="e2e-customer-middle-name"]',
        'input[name="e2e-customer-last-name"]',
        'input[name="e2e-customer-address"]',
        'input[name="e2e-customer-phone"]',
      ];
      clearBatchValues(client, selectors, done);
    })
    .waitForElementVisible('input[name="e2e-customer-first-name"]')
    .setValue('input[name="e2e-customer-first-name"]', 'Leo')
    .setValue('input[name="e2e-customer-last-name"]', 'Qiu')
    .perform((client, done) => {
      clearValue(client, 'input[name="e2e-customer-address"]', done);
    })
    ...

As you can see the test chain is not broken, in order to avoid breaking it and still being able to execute custom scripts we can use the perform command, which provides us a done callback that can be triggered once the process finalizes.

The clearBatchValues method was created in order to clear multiple inputs in a batch, this allows us to reduce lines of code by not having to call perform and clearValue invidually within a test script.

Case calling browser.url multiple times

The editCustomer, newCustomer, searchCustomer, and login scripts where each calling browser.url individually after login. This caused an effect similar to a full page refresh, it slowed down and cause tests to be unstable due to the fact that some resources take longer to load and certain test scripts had to wait for longer than the default time. In order to avoid this we can use the navigation UI, by clicking the customers menu item from the sidebar we are still able to access the customers UI without having to do a "full page refresh".

browser.url should be restricted unless a redirect is required, or if it is not possible to navigate to the desired page through the navigation UI.

Final Notes:

It is very important to pay attention to what your test results are giving you, if a test is failing there is always a reason behind it, look at the way you are implementing your UI.

Keeping in mind best practices and UI standards will guide you to the solution of your failing e2e tests.

Avoid at all cost using pauses

Investigate and learn about your e2e tool, nightwatch provides good documentation, but remember documentation will guide you but won't make the solution for you.

Understand how the command queue works, here is a good article that explains it https://github.com/nightwatchjs/nightwatch/wiki/Understanding-the-Command-Queue