# Run browser tests from any CI with one file

> CircleCI, Bitbucket, Jenkins, Buildkite or Azure Pipelines: one dependency-free script runs your saved tests, writes JUnit and exits non-zero on a failure.

By Jason Nguyen, Founder, Testsy. Published 2026-11-21. https://testsy.dev/blog/run-browser-tests-from-any-ci-with-one-file/

## No plugin needed

Not every team runs GitHub Actions or GitLab. Testsy's [command-line tool](https://testsy.dev/docs/cli/) is one JavaScript file with no dependencies, so it runs anywhere with Node 18 or later:

```sh
curl -fsSL https://app.testsy.dev/cli/testsy-ai.mjs -o testsy-ai.mjs
export TESTSY_AI_API_KEY=qa_…   # from a secret, never in the file
node testsy-ai.mjs run --project p_your_project --junit testsy-ai-junit.xml
```

It recognises GitHub Actions, GitLab, CircleCI, Bitbucket, Buildkite, Azure Pipelines and Jenkins, and sends the commit, branch, job and job link with each run, so a result points back to the build that started it.

## What `run` does

It starts every test in the project (or the ones you name with `--test`), then checks every 15 seconds until each has a verdict, printing each result as it lands. Each start sends an idempotency key and requests are retried on network errors, server errors and rate limits, so a retry never starts a run twice.

The useful flags:

| Flag | |
| --- | --- |
| `--url <address>` | Website tests start at this address: a preview, review app or staging |
| `--junit <file>` | One test case per run, with the link to its proof |
| `--fail-on` | Verdicts that make it exit 1; `failed,error` by default |
| `--timeout` | How long to wait; 45 minutes by default |
| `--build <file or link>` | App tests run this build |
| `--no-wait` | Start the runs, print their ids and exit |

## Exit codes your CI understands

| Code | Meaning |
| --- | --- |
| `0` | Every run passed, or none ended in a `--fail-on` verdict |
| `1` | A run ended in a `--fail-on` verdict |
| `2` | A setup problem: missing key, unknown project, a test that could not start |
| `3` | Timed out; the JUnit file is still written |

A missing result never makes the suite pass: a run that has no verdict is reported as an error.

## Where to put it

After the deploy step, so the tests see the new version. Pass the deployed address with `--url` if it differs from the saved one. Keep the JUnit file as an artifact, and publish it with your CI's test report feature so failures show in the build page.

## Reading a failure

The tool prints each run's verdict and a link. Runs started this way also come with a [watch link](https://testsy.dev/docs/watch-links/) that opens the run live and its result afterwards, without a Testsy account, for whoever is on call. The result shows the check that failed, what was expected and what was seen, with a replay.

This post is part of [Release testing for product teams](https://testsy.dev/blog/release-testing-for-product-teams/).
