playwright-snippets.md

Create custom fixtures to bypass bot detection algorithms in 3rd party applications for integration testing

import { Browser, test } from "@playwright/test";
import { chromium } from "playwright-extra";
import stealth from "puppeteer-extra-plugin-stealth";

export default test.extend<Browser>({
  browser: async ({ browser }, use) => {
    await browser.close();

    chromium.use(stealth());
    const stealthBrowser = await chromium.launch();
    await use(stealthBrowser);

    await stealthBrowser.close();
  },
});

Defining custom Typescript types for automation specific usecases

Example 1

type AtLeastOne<T, U = { [K in keyof T]: Pick<T, K> }> = Partial<T> &
  U[keyof U];

This code defines a custom utility type called AtLeastOne which is used to make sure that the type of an Object is a subset of another interface. This custom util is similar to the Partial util provided by Typescript with one key improvement. Partial accepts an empty object to be a valid type, but AtLeastOne requires at least one key-value pair of the superset to be present on the object.

Eg:

interface SuperSet {
  key1: string;
  key2: string;
  key3: string;
}

const object1: Partial<SuperSet> = { key1: "value1" }; // Matching type ✅
const object1: AtLeastOne<SuperSet> = { key1: "value1" }; // Matching type ✅

const object3: Partial<SuperSet> = { key4: "value4" }; // Type mismatch ❌
const object3: AtLeastOne<SuperSet> = { key4: "value4" }; // Type mismatch ❌

const object3: Partial<SuperSet> = {}; // Matching type ✅
const object3: AtLeastOne<SuperSet> = {}; // Type mismatch ❌

In this implementation we do the following to generate a custom type:

1. <T, U = { [K in keyof T]: Pick<T, K> }>: In this section, T is the interface we pass in. We define an additional generic type K which acts as an iterator going through all key value pairs of the interface T.
2. Partial<T> & U[keyof U];: In this section, Partial<T> takes all the subsets of the object includng, the empty object ({}). We use the union operator(&) to combine the partial interface with the individual key-value pairs we get from the iterator on the first step.

Example 2

Fixtures<T extends KeyValue = {}, W extends KeyValue = {}, PT extends KeyValue = {}, PW extends KeyValue = {}> = {
  [K in keyof PW]?: WorkerFixtureValue<PW[K], W & PW> | [WorkerFixtureValue<PW[K], W & PW>, { scope: 'worker', timeout?: number | undefined }];
} & {
  [K in keyof PT]?: TestFixtureValue<PT[K], T & W & PT & PW> | [TestFixtureValue<PT[K], T & W & PT & PW>, { scope: 'test', timeout?: number | undefined }];
} & {
  [K in keyof W]?: [WorkerFixtureValue<W[K], W & PW>, { scope: 'worker', auto?: boolean, option?: boolean, timeout?: number | undefined }];
} & {
  [K in keyof T]?: TestFixtureValue<T[K], T & W & PT & PW> | [TestFixtureValue<T[K], T & W & PT & PW>, { scope?: 'test', auto?: boolean, option?: boolean, timeout?: number | undefined }];
};

This is a utility type which extends the existing test fixtures with custom fixtures. The above code accepts:

• T - Additional test fixture entries
• W - Additional Worker fixture entries
• PT - Existing Playwright test fixture entries
• PW - Existing Playwright worker fixture entries

This utility merges and combines all the exsting and new fixture entries and returns it a single type when create a new fixture to be used in Playwright tests.

Implement custom solution for the limitations in Playwright while tracking multiple responses parallely

// Reference https://www.bigbinary.com/blog/tackling-flaky-tests-in-cypress-and-playwright

this.responses = [];

interceptMultipleResponses = ({
    responseUrl = "",
    responseStatus = 200,
    times = 1,
    baseUrl,
    customPageContext,
    timeout = 35_000,
  }: Partial<InterceptMultipleResponsesParams> = {}): Promise<Response[]> => {
    const pageContext = customPageContext ?? this.page;
    return Promise.all(
      [...new Array(times)].map(() =>
        pageContext.waitForResponse(
          (response: Response) => {
            if (
              response.request().resourceType() === "xhr" &&
              response.status() === responseStatus &&
              response.url().includes(responseUrl) &&
              response.url().startsWith(baseUrl ?? this.baseURL ?? "") &&
              !this.responses.includes(response.headers()?.["x-request-id"])
            ) {
              this.responses.push(response.headers()?.["x-request-id"]);

              return true;
            }

            return false;
          },
          { timeout }
        )
      )
    );
  };

Use TOTP to skip 2FA in 3rd party applications like GitHub

loginToGithubInNewTabAndClose = async () => {
    const totp = new TOTP({
      issuer: "GitHub",
      label: process.env.GITHUB_ACCOUNT_USERNAME,
      algorithm: "SHA1",
      digits: 6,
      period: 30,
      secret: process.env.GITHUB_ACCOUNT_TOTP_SECRET,
    });

    const githubPage = await this.context.newPage();
    await githubPage.goto(THIRD_PARTY_ROUTES.github.loginUrl);
    await githubPage
      .getByLabel("Username or email address")
      .pressSequentially(process.env.GITHUB_ACCOUNT_USERNAME, { delay: 5 });

    await githubPage
      .getByLabel("Password")
      .pressSequentially(process.env.GITHUB_ACCOUNT_PASSWORD, { delay: 5 });

    await githubPage
      .getByRole("button", { name: GITHUB_TEXTS.signIn, exact: true })
      .click();

    await githubPage.getByPlaceholder("XXXXXX").click();
    await githubPage
      .getByPlaceholder("XXXXXX")
      .pressSequentially(totp.generate(), { delay: 5 });

    if (
      await githubPage
        .getByText(GITHUB_TEXTS.confirmAccountRecovery)
        .isVisible()
    ) {
      await githubPage
        .getByRole("button", { name: GITHUB_TEXTS.remindMeLater })
        .click();
    }

    const skipPasskeyPrompt = githubPage.getByRole("link", {
      name: GITHUB_TEXTS.askMeLater,
    });

    if (await skipPasskeyPrompt.isVisible()) {
      await skipPasskeyPrompt.click();
    }

    await githubPage.close();
  };

Create dynamic iframes on webpages to test embed functionality

  initializeEmbedPage = async ({
  embedType,
  embedCode,
}: initializeEmbedPageParams) => {
  this.embedPage = await this.context.newPage();
  this.embedType = embedType;

  await this.embedPage.setContent(
    basicHTMLContent(
      this.embedType === "elementClick"
        ? `${embedCode}<a href='#' id='open-popup-button'>Click here to book</a>`
        : embedCode
    )
  );

  this.embeddedFrame = this.embedPage.frameLocator(
    this.embedType === "inline" ? "iframe" : EMBED_SELECTORS.neetoCalIFrame
  );

  return this.embedPage;
};

  copyEmbedScript = async ({ embedLabel }: { embedLabel: string }) => {
  await this.page.getByTestId(`${hyphenize(embedLabel)}-radio-label`).click();
  await expect(
    this.page.getByTestId(`${hyphenize(embedLabel)}-radio-label`)
  ).toBeChecked();

  await this.page.getByTestId(CAL_COMMON_SELECTORS.copyButton).click();

  return await this.page.evaluate(() => navigator.clipboard.readText());
};

Use Twilio and Mailosaur APIs to test Email and SMS delivery in a cost-effective manner

const [customerReminderEmail, hostReminderEmail] = await Promise.all([
        remindersPage.verifyReminderEmail(customerEmail, receivedAfter),
        remindersPage.verifyReminderEmail(credentials.email, receivedAfter),
      ]);

      expect(customerReminderEmail.subject).toContain(
        REMINDER_TEXTS.emailReminderBody(
          `${credentials.firstName} ${credentials.lastName}`
        )
      );

      expect(hostReminderEmail.subject).toContain(
        REMINDER_TEXTS.emailReminderBody(customerName)
      );

      await expect(async () => {
        const messages = await twilioClient.messages.list({
          from: process.env.TWILIO_DEFAULT_PHONE_NUMBER,
          to: process.env.TWILIO_INCOMING_PHONE_NUMBER,
          dateSent: receivedAfter,
        });

        const filteredMessages = messages.filter(
          message =>
            message.dateUpdated > receivedAfter &&
            message.status === "received" &&
            message.body.includes(REMINDER_TEXTS.smsReminderBody()) &&
            message.direction === "inbound"
        );

        expect(
          filteredMessages.find(message =>
            message.body.includes(
              REMINDER_TEXTS.smsReminderBody(
                `${credentials.firstName} ${credentials.lastName}`
              )
            )
          )
        ).toBeDefined();

        expect(
          filteredMessages.find(message =>
            message.body.includes(REMINDER_TEXTS.smsReminderBody(customerName))
          )
        ).toBeDefined();
      }).toPass({ timeout: 2 * 60 * 1000 });

Modify the user-agents to simulate different Operating Systems to test various behaviors

test("should verify keyboard shortcuts in non-mac environment", async ({
    browser,
    request,
  }) => {
    const context = await browser.newContext({
      userAgent: USER_AGENTS.windows,
      storageState: STORAGE_STATE,
    });
    const page = await context.newPage();
    const neetoPlaywrightUtilities = new CustomCommands(page, request);
    const helpAndProfilePage = new HelpAndProfilePage({
      page,
      neetoPlaywrightUtilities,
      chatApiBaseURL: CHAT_API_BASE_URL,
      kbDocsBaseURL: KB_DOCS_BASE_URL,
      changelogBaseURL: CHANGELOG_BASE_URL,
    });

    await page.goto("/", { waitUntil: "load" });
    await helpAndProfilePage.openAndVerifyKeyboardShortcutsPaneV2(
      [],
      "windows"
    );
  });

  test("should verify keyboard shortcuts in mac environment", async ({
    browser,
    request,
  }) => {
    const context = await browser.newContext({
      userAgent: USER_AGENTS.mac,
      storageState: STORAGE_STATE,
    });
    const page = await context.newPage();
    const neetoPlaywrightUtilities = new CustomCommands(page, request);
    const helpAndProfilePage = new HelpAndProfilePage({
      page,
      neetoPlaywrightUtilities,
      chatApiBaseURL: CHAT_API_BASE_URL,
      kbDocsBaseURL: KB_DOCS_BASE_URL,
      changelogBaseURL: CHANGELOG_BASE_URL,
    });

    await page.goto("/", { waitUntil: "load" });
    await helpAndProfilePage.openAndVerifyKeyboardShortcutsPaneV2([], "mac");
  });

  test("should verify app switcher", async ({ helpAndProfilePage }) => {
    await helpAndProfilePage.openAppSwitcherAndVerifyV2();
  });

  test("should verify profile links", async ({ helpAndProfilePage }) => {
    skipTest.forAllExceptStagingEnv();
    await helpAndProfilePage.verifyProfileAndOrganizationLinksV2();
  });
});

Use application i18n translations to avoid test failures due to minor text changes

import {
  GLOBAL_TRANSLATIONS_PATTERN,
  PROJECT_TRANSLATIONS_PATH,
} from "@constants/common";
import { Page } from "@playwright/test";
import { readFileSyncIfExists } from "@utils/common";
import { sync } from "fast-glob";
import { getI18nInstance } from "playwright-i18next-fixture";
import { mergeDeepLeft } from "ramda";

type GenericObject = Record<string, unknown>;

declare global {
  interface Window {
    globalProps: GenericObject;
  }
}

export const LOWERCASED = "__LOWERCASED__";

export const FORMATS = {
  boldList: "boldList",
  list: "list",
  anyCase: "anyCase",
};
export const LIST_FORMATS = [FORMATS.boldList, FORMATS.list];


export const readTranslations = () => {
  let translations: GenericObject = readFileSyncIfExists(
    PROJECT_TRANSLATIONS_PATH
  );
  const paths = sync(GLOBAL_TRANSLATIONS_PATTERN);
  paths.forEach(path => {
    const packageTranslation = readFileSyncIfExists(path);
    translations = mergeDeepLeft(translations, packageTranslation);
  });

  return translations;
};

const getter = (key: string) => () =>
  getI18nInstance().t(`taxonomyDefaultLabels.${key}`);

export const replaceNullValuesWithGetter = (
  inputObject: object,
  parentKey = ""
) => {
  const result: GenericObject = {};

  for (const [key, value] of Object.entries(inputObject)) {
    const transKey = parentKey ? `${parentKey}.${key}` : key;

    if (value === null) {
      Object.defineProperty(result, key, {
        get: getter(transKey),
      });
    } else if (typeof value === "object") {
      result[key] = replaceNullValuesWithGetter(value, transKey);
    } else {
      result[key] = value;
    }
  }

  return result;
};

export const mergeTaxonomies = async (
  translations: GenericObject,
  page: Page
) => {
  const defaultTaxonomyKeys = Object.keys(
    translations.taxonomyDefaultLabels || {}
  );

  const defaultTaxonomies = Object.fromEntries(
    defaultTaxonomyKeys.map(key => [key, { singular: null, plural: null }])
  );

  const hostTaxonomies = (await page.evaluate(
    () => window.globalProps?.taxonomies
  )) as object;

  return replaceNullValuesWithGetter(
    mergeDeepLeft(hostTaxonomies, defaultTaxonomies)
  );
};
export const i18nFixture: Fixtures<
  I18nPlaywrightFixture,
  PlaywrightWorkerArgs & PlaywrightWorkerOptions,
  PlaywrightTestArgs & PlaywrightTestOptions,
  PlaywrightWorkerArgs & PlaywrightWorkerOptions
> = {
  i18n: [
    async ({ page }, use) => {
      const translation = readTranslations();

      const taxonomies = await mergeTaxonomies(translation, page);
      const options: InitOptions = {
        debug: false,
        fallbackLng: "en",
        resources: { en: { translation } },
        interpolation: {
          defaultVariables: { taxonomies },
          escapeValue: false,
          skipOnVariables: false,
          alwaysFormat: true,
          format: (value, format, lng, options) => {
            let newValue = value;

            if (LIST_FORMATS.includes(format ?? "")) {
              newValue = listFormatter({
                value: newValue,
                format,
                lng,
                options,
              });

              return newValue;
            }

            return lowerCaseDynamicTextFormatter(newValue, format);
          },
        },
        postProcess: ["removeTagsProcessor", "sentenceCaseProcessor"],
      };
      const i18nInitialized = await initI18n({
        plugins: [removeTagsProcessor, sentenceCaseProcessor],
        options,
        // Fetch translations in every test or fetch once
        cache: true,
      });
      await use(i18nInitialized);
    },
    // Run as auto fixture to be available through all tests by getI18nInstance()
    { auto: true },
  ],
  t: async ({ i18n }, use) => {
    await use(i18n.t);
  },
};

Use JMAP and fastmail APIs to send instant emails to test sending and receiving emails

  hostname = "api.fastmail.com";
  username = process.env.FASTMAIL_USERNAME || "automation@example.com";
  token = process.env.FASTMAIL_TOKEN;

  authUrl = `https://${this.hostname}/.well-known/jmap`;
  headers = {
    "Content-Type": "application/json",
    Authorization: `Bearer ${this.token}`,
  };

  getSession = async () => {
    const response = await this.neetoPlaywrightUtilities.apiRequest({
      url: this.authUrl,
      method: "get",
      headers: this.headers,
    });

    return response.json();
  };

  getSentId = async ({ apiUrl, accountId }) => {
    const response = await this.neetoPlaywrightUtilities.apiRequest({
      url: apiUrl,
      method: "post",
      headers: this.headers,
      body: JSON.stringify({
        using: ["urn:ietf:params:jmap:core", "urn:ietf:params:jmap:mail"],
        methodCalls: [
          ["Mailbox/query", { accountId, filter: { name: "Sent" } }, "a"],
        ],
      }),
    });
    const data = await response.json();

    return data["methodResponses"][0][1].ids[0];
  };

  getIdentityId = async ({ apiUrl, accountId }) => {
    const response = await this.neetoPlaywrightUtilities.apiRequest({
      url: apiUrl,
      method: "post",
      headers: this.headers,
      body: JSON.stringify({
        using: [
          "urn:ietf:params:jmap:core",
          "urn:ietf:params:jmap:mail",
          "urn:ietf:params:jmap:submission",
        ],
        methodCalls: [["Identity/get", { accountId, ids: null }, "a"]],
      }),
    });
    const data = await response.json();
    const identities = data["methodResponses"][0][1].list;

    return identities.find(({ email }) => email === this.username).id;
  };

  composeEmail = async ({
    apiUrl,
    accountId,
    sentId,
    identityId,
    emailDetails,
    replyValue = null,
    emailId = null,
    messageId = null,
  }) => {
    const { subject, body, to, senderName } = emailDetails;

    const sentObject = {
      from: [{ email: this.username, name: senderName }],
      to: [{ email: to }],
      subject,
      keywords: { $sent: true },
      mailboxIds: { [sentId]: true },
      bodyValues: { body: { value: body, charset: "utf-8" } },
      textBody: [{ partId: "body", type: "text/plain" }],
      inReplyTo: replyValue,
      references: replyValue,
    };

    const response = await this.neetoPlaywrightUtilities.apiRequest({
      url: apiUrl,
      method: "post",
      headers: this.headers,
      body: JSON.stringify({
        using: [
          "urn:ietf:params:jmap:core",
          "urn:ietf:params:jmap:mail",
          "urn:ietf:params:jmap:submission",
        ],
        methodCalls: [
          [
            "Email/set",
            {
              accountId,
              create: { sent: sentObject },
              destroy: emailId,
            },
            "a",
          ],
          [
            "EmailSubmission/set",
            {
              accountId,
              onSuccessDestroyEmail: messageId,
              create: { sendIt: { emailId: "#sent", identityId } },
            },
            "b",
          ],
        ],
      }),
    });
    const data = await response.json();

    return data["methodResponses"][0][1].created.sent.id;
  };

  getMessageId = async ({ apiUrl, accountId, id }) => {
    const response = await this.neetoPlaywrightUtilities.apiRequest({
      url: apiUrl,
      method: "post",
      headers: this.headers,
      body: JSON.stringify({
        using: [
          "urn:ietf:params:jmap:core",
          "urn:ietf:params:jmap:mail",
          "urn:ietf:params:jmap:submission",
        ],
        methodCalls: [["Email/get", { accountId, ids: [id] }, "a"]],
      }),
    });
    const data = await response.json();

    return data["methodResponses"][0][1].list[0].messageId[0];
  };

  checkForEnvs = () => {
    if (!this.username || !this.token) {
      throw new Error(
        "Please provide FASTMAIL_USERNAME and FASTMAIL_TOKEN as environment variables"
      );
    }
  };

  sendReplyAndDeleteEmail = async emailDetails => {
    this.checkForEnvs();
    const session = await this.getSession();
    const { apiUrl, primaryAccounts } = session;
    const accountId = primaryAccounts["urn:ietf:params:jmap:mail"];
    const [sentId, identityId] = await Promise.all([
      this.getSentId({ apiUrl, accountId }),
      this.getIdentityId({ apiUrl, accountId }),
    ]);

    const emailId = await this.composeEmail({
      apiUrl,
      accountId,
      sentId,
      identityId,
      emailDetails,
    });

    const messageId = await this.getMessageId({
      apiUrl,
      accountId,
      id: emailId,
    });

    await this.composeEmail({
      apiUrl,
      accountId,
      sentId,
      identityId,
      emailDetails,
      replyValue: [messageId],
      emailId: [emailId],
      messageId: ["#sendIt"],
    });
  };
}

Add random mouse movements and keyboard delays to prevent sites from identifying bot user

When using 3rd party websites like Google, Microsoft, zoom, etc., they go through many measures to prevent bot users on their sites. For tracking bot users, they depend on the human imperfections in human interactions on the screen, like shacky mouse movements and human-level delays in type events. Such imperfections do not occur for a bot user, so we deliberately added such randomness in our interactions on the website to ensure that the website doesn't tag the test as a bot user and block the account.

Multi-user login to test role-based access

Sometimes, we need to test role-based access and other features requiring a user to be logged in as both a standard user and an admin user. In such cases, we need to utilize the multi-page features provided by Playwright to log in as an admin user in one browser and as a standard user in another. This can give us real-time feedback on the admin actions on the standard account and how they can affect certain features. We also use this multi-user login to test the chat application in neetoChat, where the chat happens between two users.

Implement tests such that they do not interfere with each other in implementing parallelism

When the number of test cases increases, they will take much time to run serially. To improve the speed of test runs, we need to implement parallelism, which will run multiple tests simultaneously. When doing this, there is a case in which the execution of one test can interfere with the other.

Consider an example where a test deletes a record and asserts the total record count after deletion. If another test creates/deletes a record in between the tests, the parallel execution will cause the assertion on the first test to fail because the record count might not be what we expect. We need to implement tests so that the assertions are independent. In the above example, instead of asserting the record count after deletion, we need to assert that the record is not present on the table after deletion.

Implement sharded and processor-based parallelism to achieve 16x speed than normal

In Playwright, we have two levels of parallelism. Processor-based parallelism runs multiple tests on the same machine on different processors. And we have sharded parallelism, where tests are split between different machines and run parallel. We have implemented a combination of these where we have four tests running parallelly per machine and four machines running parallel. This makes the effective parallelism 16. This means a test run, which takes almost 16 mins to complete serially, now completes within 1 minute.

Common NPM package.

We have multiple applications on which we write the Playwright tests. These applications have a lot of common logic which need not be repeated everywhere. For this purpose, we have created a common NPM package encapsulating all the common utils, methods, workflows and helpers in a single commons package. This package is written entirely in TypeScript and we have used rollup to efficiently compile all the package contents into JavaScript and type definitions. This compiled code is published to NPM and we use this NPM package to access all the common logic in all applications.