BLOG
BlogGuides

What a browser test suite costs when you pay per run

No seats and no minutes: one test run once is one run. How to estimate a month of release checks, schedules and CI runs on Testsy's plans, and what is never counted.

The unit is a run#

Testsy prices by runs. One test, run once, on one platform, is one run, however long it takes. An app run counts the same as a web run. There are no seats: inviting the whole team does not change the bill, up to each plan's member limit.

These are counted: passed, failed and inconclusive runs, including scheduled runs and runs from CI, agents and the API. These are not: runs the site blocked, runs that failed because of a problem on our side, and runs you stopped. An app test that uses two phones counts as two. From plans and billing.

The plans#

Free trialStarterTeamGrowth
PriceFree$49 a month$149 a month$299 a month
Runs included10, once100 a month500 a month1,200 a month
Extra runs (packs of 50)—$30$20$15
Web runs at a time151020
App runs at a time11310
Android and iOS app testsYesYesYesYes
Members1525100
Projects12515
Tests203001,0005,000
Assistant drafts a day40300300300
Phone number and endpoint calls—YesYesYes
Invite people—YesYesYes

Prices are in US dollars a month. The free trial's runs are for the life of the workspace; paid plans' runs start again each billing period. When a paid plan's runs are used, you can buy a pack of 50 more or move up a plan. A pack lasts 90 days.

Estimate a month#

Count three sources:

  1. After each deploy. Tests in the release suite × deploys a month. Five tests × 40 deploys = 200 runs.
  2. On a schedule. Tests × runs a day × 30. Four tests nightly = 120 runs.
  3. By hand and by agents. Developers and coding agents running tests while they work. Watch it for a week on Plan and usage, which shows runs by project.
Plan and usage: runs used this period and the projects they went to.
Plan and usage: runs used this period and the projects they went to.

Five tests on 40 deploys plus four nightly is 320 runs a month: inside the Team plan's 500, with room for agents and reruns.

Ways to spend less#

  • Run the full suite where it matters. On pull requests, run a short smoke set on each push and the full suite once before merge.
  • Use learned paths. A test that passed once replays the learned path on later runs, which is faster. It does not change the run count, but it shortens CI waits.
  • Allow our addresses if bot protection blocks you. Blocked runs are not counted, but they are not useful either.
  • Archive tests you no longer run. Archived tests do not count toward the saved-test limit.

Try it before deciding#

The free trial has 10 runs and needs no card: enough to write your release suite and run it twice.

This post is part of Release testing for product teams.

Filed underWebCI, schedules and agentsChoosing a tool