BLOG
BlogGuides

Make Claude Code test what it just built

Your coding agent says the feature works. Connect Testsy over MCP and it has to run the feature in a real browser and bring back proof before it says done.

"Done" is a claim, not a result#

A coding agent finishes a change and reports that it works. Usually it means the code compiles and the agent's own reading of it looks right. Nobody has signed in, clicked the button and reloaded the page.

Testsy gives the agent a way to do exactly that and a reason it cannot bluff: the test runs on our side, in Chrome or on iPhone, iPad and Android, and comes back with a verdict and the value behind every check. The agent reads the result the same way you would.

This guide sets that up in Claude Code. The same steps work in Cursor, Codex and any other MCP client; their setup is at the end.

What you need#

  • A Testsy workspace. The free trial has 10 runs.
  • Your change deployed somewhere the internet can reach: a staging site or a preview deployment. Testsy's browser is not on your laptop, so an address that only works on your machine will not load.
  • A test account for your app, made for testing.

Connect Claude Code#

The quickest way is the plugin. In Claude Code:

/plugin marketplace add testsy-ai/testsy-claude-plugins
/plugin install testsy@testsy

There is no key to set. The first time Claude calls a Testsy tool, a browser window asks you to sign in and choose the workspace.

For CI or any session without a browser, connect with an API key instead. Create one in Settings → API keys, store it as the environment variable named below, and add the server:

claude mcp add --transport http testsy https://app.testsy.dev/mcp --header "Authorization: Bearer $TESTSY_AI_API_KEY"

A key that can only run tests and read results is enough once your tests exist. Choose the one that can also create and edit tests while the agent is still writing them.

Integrations in the dashboard: where runs can start from, including AI agents over MCP.
Integrations in the dashboard: where runs can start from, including AI agents over MCP.

Store the sign-in account in the app, not in the chat#

Add the test account under the project's Sign-in accounts tab and give it a name such as "member". The tester types the password into your sign-in form itself; the password never reaches the AI. If you paste it into the conversation instead, it has already passed through your agent. See Sign-in accounts.

Ask for one journey, with a visible result#

Pick the thing a customer does next with what you just built. Then give Claude a small, exact job:

Use Testsy to check the task app at https://staging.example.com.
Sign in as the stored account "member".

A member should be able to create a task named "Release smoke sample",
see it in the list, reload the page, and still see it with that title.
Do not change other tasks.

Draft the plan first and show it to me before you save or run anything.

Two things in that prompt do most of the work. The expected result is something a person could see on the screen ("still see it with that title"), and the agent has to show the plan before it runs.

What the agent does#

  1. It finds the project with list_projects and drafts the test with plan_test. You get the steps and the checks back, sometimes with a question it cannot answer alone.
  2. Once you approve, it saves the plan with create_test and starts it with run_test.
  3. run_test returns a watch link. Open it and you see the browser live, with each check ticking over as it is verified. No sign-in is needed, and the link keeps working for 24 hours after the run unless your workspace chose otherwise. See Watch links.
  4. It calls wait_run until the run is final, then reads each check's result, reason and value.

A run usually takes 2 to 8 minutes. A queued run or a drafted plan is not a result, so do not accept "the test has been started" as an answer.

A finished run: the verdict, then the replay with each step and the place a value was read.
A finished run: the verdict, then the replay with each step and the place a value was read.

Read the verdict before the summary#

Agents summarise. Read the verdict line first.

VerdictWhat to do
PassedThis journey works in this environment. That is all it says.
FailedA check did not match. Open the step it failed at before deciding what to fix.
BlockedSomething stopped the tester, often a bot wall or a sign-in barrier. Fix access; nothing was verified.
InconclusiveThe tester could not establish the answer. Make the check name something visible.
A failed check: expected, saw, the step it was judged at, and Copy fix prompt next to Share.
A failed check: expected, saw, the step it was judged at, and Copy fix prompt next to Share.

The classic bug this catches: the task appears the moment you create it and is gone after a reload, because the save never reached the server. The agent's own summary would have said the feature works.

Close the loop#

When a check fails, the agent already has what it needs. With get_run it reads the failed check, what was expected, what was seen and the steps to reproduce. Ask it to fix the cause, deploy, and run the same saved test again.

Two rules keep this honest:

  • The expectation does not move. If the agent proposes changing the check so that it passes, say no. Fix the app or decide the requirement was wrong, on purpose.
  • Keep the failed run. Ask for links to the failed result and the passing one. The pair is your evidence that the fix did something.

Working in the dashboard instead? Copy fix prompt on any result that did not pass puts the same facts on your clipboard for any coding agent. See Fix prompt.

Make it the default#

The plugin installs three skills that turn this into a habit: one to check a release, one to investigate a run, and one to turn a bug into a regression test. You can also add a line to your project's instructions for the agent:

After changing anything a user can see, deploy to staging and run the Testsy
test for that flow. Do not report the work as done until the run is final,
and include the result link.

When a check is worth running on every release, move it out of the chat and into your pipeline with GitHub Actions or a schedule.

Cursor, Codex and other clients#

The server is the same; only the way you add it differs.

claude mcp add --transport http testsy https://app.testsy.dev/mcp --header "Authorization: Bearer $TESTSY_AI_API_KEY"

Every tool, with its parameters, is listed in AI agents (MCP). A longer walkthrough of the same flow is in Test a SaaS release from Claude Code.

Filed underWebCI, schedules and agents