Test IDs: Preferred Option vs Last Resort

data-testid.md

Definitions

TestID First

To interact with a UI element from a test, apply a data-testid attribute to that element in app code, then find by test ID.

TestID Last

To interact with a UI element from a test, first choice are accessible selectors like role, label, placeholder, then HTML5 and ARIA selectors based on title or alt text. Use data-testid as a last resort. This is the view put forth by @testing-library:

In the spirit of the guiding principles, it is recommended to use [getByTestId] only after the other queries don't work for your use case. Using data-testid attributes do not resemble how your software is used and should be avoided if possible. That said, they are way better than querying based on DOM structure or styling css class names.

Benefits of TestID First

Clear test failure output

When findByTestId('search-input') fails, we get output like

Unable to find an element by: [data-testid="search-input"]

We know the element is either visually hidden or not rendered at all.

We can pair this with explicit tests for what the user sees. e.g. -

cy.findByTestId("done-button").should("have.text", "Done");

This will generate a nice error on failure that pinpoints that our text has changed.

Stable identifiers for components across refactors

If we change the visible text content or label for a UI element, but the test ID doesn't change, a test that leverages test IDs can continue to pass across that small tweak.

Drawbacks of Test ID First

Tests often do not reflect how humans interact with the app

People don't interact with the data-testids we've placed in the DOM. It's possible for a test to continue passing because it manipulates the app through test IDs, even when the app stops working for regular humans. We lose some of the confidence our tests could give us.

The stability of test IDs may be illusory

In this example, could I ever change the menu item text without also updating the data-testid?

<Button data-testid="done-button" text="Done" />

Say the button text becomes "Finish". I certainly can continue to find the menu item by done-button, but now my test ID reflects a version of the component that humans don't see today.

We'll need a naming convention for data-testids

Test IDs are as global as CSS. Consequently, we'll need a convention for avoiding name collisions. Think BEM.

Benefits of TestID Last

Tests reflect how humans interact with the app

If we change what people interact with, we will need to change test code. Since our test exercise the app similar to how humans do, we have greater confidence that our app actually works.

Avoids sprinkling test IDs throughout our components

data-testid can be visual noise, often duplicating a neighboring prop value. Less code beats more code.

Tests can be written without looking at the code under test

I think this could pass for "blackbox testing" -- avoid testing implementation details by virtue of not being able to see those details! Tests can be written by looking only at the rendered app, the way QA or a user would see it.

No naming convention

Each component essentially becomes a namespace. It's fairly easy to narrow down to the only "Complete One-off" button within a menu, or row, or modal, etc.

Drawbacks of TestID Last

Ambiguous test failure output

When findByPlaceholderText('Search products by name...') fails, we get

Unable to find an element with the placeholder text of: /Search product by name.../

This leaves us guessing: Has the text changed? Is the element visually hidden? Maybe the element isn't rendered at all.

We could preface a test that relies on the placeholder with an explicit check:

cy.findByTestId("done-button").should("have.text", "Done");

Then, if the text has changed, the first test failure will be an unambiguous one.

More app code changes require test changes

If a display value is being used for element lookup, changing it requires changing potentially many tests that rely on that lookup. (Ideally, these are straightforward to update because you know you just changed the display value and can readily identify the test failures that look like an obvious result.)

Trevor's Thoughts on Test/App Code Coupling

If we have to rewrite our tests every time we change our code, those tests give us no confidence that our code continues to produce correct results. Good tests give us freedom to refactor confidently.

For instance, if we overhaul our magicSort algorithm for a major performance gain, we want this test to keep humming right along:

expect(magicSort(myArray)).toBe([1, 2, 3]);

But we would emphatically not want that expectation to continue passing if the requirement for magicSort changed so that the array output should now be in descending order, for instance.

We'd say the assertion above avoids testing implementation details. It takes some input and expects some output, with no fuss about what happens in between.

The question when we find by button text, or by input placeholder text, is: are we testing implementation details or output? I'd argue those things are output for UX code no less than [1, 2, 3] is for a sorting algorithm. Anything we put on screen for people to rely on or navigate by--whether that's button content, input placeholder text, an aria-label, etc. could be considered part of our "contract" with the users.

The difficulty lies in deciding what is important output and what is incidental. Not all changes have an equal impact on people's ability to use our app. But I think if we're asking the user to hunt for a substantially new visual clue to exercise the same function–especially, say, if the changing label or placeholder text is the only thing they have to go on–we should have to update our tests along with our users who have to update their mental map of the app.