# Let Codex check its own work in a real browser

> Add Testsy to Codex with one command. The agent drafts a plain-language test, runs it against your app, reads every check and reruns after its fix.

By Jason Nguyen, Founder, Testsy. Published 2026-11-21. https://testsy.dev/blog/let-codex-check-its-own-work-in-a-browser/

## The gap Codex cannot close alone

Codex is good at changing code and running the commands you give it. What it cannot do on its own is look at your deployed app the way a customer does: open the page, click, wait, read what is on the screen. So "the tests pass" is the strongest thing it can tell you, and the tests are often ones it wrote.

Anthropic, writing about its own agents, describes the same pattern: the agent [marked features complete without proper testing](https://anthropic.com/engineering/effective-harnesses-for-long-running-agents) until it was told to test in a browser as a user would. The fix is to give the agent a tester that does exactly that.

## Add it with one command

Create an API key in Testsy (**Settings → API keys**), store it as `TESTSY_AI_API_KEY`, then:

```sh
codex mcp add testsy --url https://app.testsy.dev/mcp --bearer-token-env-var TESTSY_AI_API_KEY
```

The skills (**testsy-check-release**, **testsy-investigate-run**, **testsy-regression-from-bug**) are in [testsy-ai/testsy-integrations](https://github.com/testsy-ai/testsy-integrations) under `skills/`. They tell the agent to keep the original expectation, read the evidence, and tell a real failure from a test that could not finish.

Codex asks before it starts a run or changes a test. For unattended use with `codex exec` and approvals off, [the MCP docs](https://testsy.dev/docs/mcp/) give the setting that approves Testsy's tools.

## A session, step by step

"Check that the invoice page shows the right total after adding two items" becomes:

1. `plan_test` drafts the steps and checks from that sentence. Codex shows you the plan; you answer any question it asks, such as which sign-in account to use.
2. `create_test` saves it; `run_test` starts it and returns a watch link.
3. `wait_run` returns the moment the run ends, with each check, its outcome, the reason and the value it was decided on ("Total: $42.00").
4. A failed check comes with what was expected and what was seen. Codex reads it, fixes the code, and calls `run_test` on the same test again.

![Each passed check with the value it was decided on, the screenshot it was read from, and the step.](https://testsy.dev/docs/img/run-checks-light.webp)

## In CI as well

The same saved tests run from your pipeline after each deploy, with no agent involved: the [command-line tool](https://testsy.dev/docs/cli/) is one file with no dependencies, writes JUnit, and exits 1 when a check fails. So the test Codex wrote on Tuesday guards every deploy after it.

## Before you let it loose

- Point runs at staging and use made-up details: a run can create accounts and change records.
- Use the narrowest key that works. "Run tests and read results" is enough for an agent that only runs saved tests.
- Every run counts toward your plan. The free trial has 10.

This post is part of [Testing an app you built with AI](https://testsy.dev/blog/test-an-app-you-built-with-ai/).
