How a run works
What happens between pressing Run and the verdict, and what you can watch while it happens.
From Run to the verdict#
- Waiting. Your plan runs a set number of tests at a time (see Plans and billing); a run waits until one of yours finishes. While it waits it shows "Waiting for a free slot".
- Starting. We find a free browser (or, for an app, a free iPhone or Android device), start it and prepare the test's files.
- Testing. The tester follows your steps on the real screen: it reads the page, clicks, types, scrolls, uploads your files and signs in with your stored accounts. It checks each of your checks against what is on the screen and records the value it decided on.
- The verdict. Once every check is decided, the verdict is recorded and the evidence is saved: the review video, the screenshots, the console and network logs.
What you can watch#
The run page shows, while the run is going:
- The live screen: the browser with its address bar and tabs, or the app on its device, with each tap.
- Checks: "N of M verified", with the reason for each verified check.
- What the tester is doing: a timed account of its last actions.
- Steps so far: a strip with the screen after each action.
A run usually takes 2 to 8 minutes. You can leave the page; the result is kept. Tick Email me the result to get it by email, or Stop run to end it (a stopped run is not counted).
Learned paths#
The first passing run of a test learns the path through your site. The next runs replay it, so passing runs are quick; if the site has changed, the run explores again and learns the new path. You choose with the run mode:
| Mode | What it does |
|---|---|
auto | Replays the learned path when it is fresh, otherwise explores. The default, and what the app uses. |
cold | Always explores, as on the first run. |
warm | Only replays the learned path. |
Where runs come from#
Runs start from the web app (Run, Run all, Save and run, Run again), from schedules, from CI, from the command-line tool, from AI agents and from the REST API. Each run records where it came from; the Runs page filters by it.
The browser and the network#
- Websites run in Chrome: on a desktop screen (1440 × 900), a tablet screen or a phone screen. Ads and trackers are blocked by default, which makes runs faster and steadier.
- Apps run on an iPhone or an Android device with fresh app data each run.
- Browsers reach your site directly from our cloud. If a site refuses cloud traffic with a bot wall on the first page, the run is retried through our static addresses and the result says so. You can allow those addresses: see Network and allow-listing.
- Every run stops by 20 minutes. A run is one run however long it takes.