BLOG
BlogGuides

Tests that survive a redesign: why selector tests break

Test maintenance is the cost that kills end-to-end suites. Why tests tied to selectors fail after UI changes when the app still works, and what to write instead.

Red, and nothing is broken#

The worst failure a test suite can produce is a false one. The app works; the test fails because a class name changed, a button moved into a menu, a component library was swapped. After a few of those, people stop reading the results.

Shopify's mobile team described exactly this: their Appium suite kept failing while the app was fine. Currents, which builds reporting for Playwright suites, cites a survey of 335 developers and testers that found the main cost of unreliable tests was not the reruns but the gradual loss of confidence in the results.

Why selectors break#

A scripted test says *where* to click: the element with this id, the third button in this row, the input after this label. A redesign changes where things are without changing what the app does. The test was describing the structure, so it breaks, and somebody spends the afternoon updating selectors for a release that had no bugs.

Good practice reduces this (stable test ids, role-based locators), but it cannot remove it: a test written against structure needs updating when structure changes.

Describe the behaviour instead#

A Testsy test describes what a person would do and what they should see:

Sign in with the stored account "customer".
Open the project "Launch notes" and rename it to "Launch notes v2".
Reload the page and check the project's title reads "Launch notes v2".

There is no selector to go stale. If the rename moves from a pencil icon to a settings menu next quarter, the sentence is still true, and the tester finds the new path. The first passing run learns the path through your site and later runs replay it; when the site has changed, the run explores again and learns the new one. See learned paths.

What still needs changing#

Plain-language tests are not maintenance-free. They need editing when behaviour changes:

  • The feature is renamed in the interface ("Projects" become "Workspaces").
  • A step is added ("confirm in the dialog").
  • The expected value changes ("the free plan now allows 5 projects").

That is the right kind of maintenance: the test changes when the product does, not when the markup does.

Proof of what it decided#

A test that adapts to a new layout needs to show what it actually checked, or you are trusting a black box. Every Testsy check is kept with the value it was decided on and a crop of the screen it was read from, and the replay marks every click.

Each passed check with the value it was decided on, the screenshot it was read from, and the step.
Each passed check with the value it was decided on, the screenshot it was read from, and the step.

If a pass looks wrong, you can see exactly what the tester read and where.

Where to start#

Move the tests that break most often in your current suite: the ones on screens your designers touch every sprint. Keep your unit and API tests; they are fast and precise for what they cover. Use plain-language tests for the journeys a customer takes through the interface.

This post is part of Release testing for product teams.

Filed underWebReleases and regressionsChoosing a tool