Test every Vercel preview before you merge
Run your saved plain-language tests against each pull request's preview, including previews behind Vercel Deployment Protection, and fail on a regression.

The preview nobody clicks#
Every pull request on Vercel gets its own preview address, built from the branch and running on real infrastructure. It is the best place to catch a regression, before it reaches main. In practice the preview gets a glance from the reviewer, if that.
Running browser tests against it is the obvious fix, and the hard part is plumbing: waiting for the deployment, finding the right address, getting past protection, and reporting back to the pull request.
Same tests, preview address#
A Testsy test is saved with your production address. For a preview you do not need a second test: start the same test at another address, and it explores from there. From the CI docs:
- GitHub Action:
target-url: ${{ github.event.deployment_status.environment_url }} - Command-line tool:
node testsy-ai.mjs run --project p_… --url "https://pr-42.preview.example.com" - API and MCP:
urlwhen you start a test or a project.
Vercel reports each finished deployment to GitHub as a deployment status, so a workflow that runs on: deployment_status starts when the preview is ready, with its address in the event. The workflow from the docs does this:
# .github/workflows/testsy-ai.yml
name: Testsy
on:
deployment_status:
workflow_dispatch:
jobs:
qa:
if: github.event_name == 'workflow_dispatch' || github.event.deployment_status.state == 'success'
runs-on: ubuntu-latest
steps:
- name: Run Testsy tests
env:
TESTSY_AI_API_KEY: ${{ secrets.TESTSY_AI_API_KEY }}
TESTSY_AI_URL: https://app.testsy.dev
TARGET: ${{ github.event.deployment_status.environment_url }}
run: |
curl -fsSL "$TESTSY_AI_URL/cli/testsy-ai.mjs" -o testsy-ai.mjs
# On a deployment event the tests run against that deploy's address (a preview or staging of the same site).
node testsy-ai.mjs run --project p_your_project --junit testsy-ai-junit.xml ${TARGET:+--url "$TARGET"}
- uses: actions/upload-artifact@v4
if: always()
with: { name: testsy-ai-junit, path: testsy-ai-junit.xml }The run's result shows the address it ran against. App tests in the same project are unaffected; they run their build.
Previews behind Deployment Protection#
If your previews ask for a Vercel sign-in, the tester will be stopped at that page like any stranger. Vercel's answer for automation is Protection Bypass for Automation: a secret, created in the project settings, that can be sent as a header or as a query parameter named x-vercel-protection-bypass. Adding x-vercel-set-bypass-cookie=true makes Vercel set a cookie, so the pages that follow load too.
Testsy's browser does not send custom headers, so use the query parameter in the address you pass:
target-url: ${{ github.event.deployment_status.environment_url }}/?x-vercel-protection-bypass=${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }}&x-vercel-set-bypass-cookie=trueTreat the address as a secret: it carries the bypass. The result shows the address the run started at, so keep the result inside your workspace rather than sharing it, and rotate the secret if it leaks. A deployment built before you rotate keeps the old value, as Vercel's page explains.
What a preview run cannot share with production#
- Sign-in accounts and endpoints belong to the test. A preview that needs different credentials needs its own test.
- Learned paths belong to the saved address, so a preview run always explores. That makes it a little slower than a replayed production run.
- Data. A preview usually talks to a staging database. Tests that create records should use made-up details.
Fail the check, keep the evidence#
By default the job fails when a run fails or does not finish; fail-on changes that. Each run prints a watch link anyone can open, live and afterwards, and on GitHub a table of results goes into the job summary. Make the Testsy job a required check on main, and a regression stops at the pull request with its replay attached.
Each run counts toward your plan. Five tests on every push of a busy pull request adds up; many teams run the full set on deployment_status for the final push only, or a short smoke set on every push.
Filed underWebReleases and regressionsCI, schedules and agents