BLOG
BlogGuides

Test every Vercel preview before you merge

Run your saved plain-language tests against each pull request's preview, including previews behind Vercel Deployment Protection, and fail on a regression.

The preview nobody clicks#

Every pull request on Vercel gets its own preview address, built from the branch and running on real infrastructure. It is the best place to catch a regression, before it reaches main. In practice the preview gets a glance from the reviewer, if that.

Running browser tests against it is the obvious fix, and the hard part is plumbing: waiting for the deployment, finding the right address, getting past protection, and reporting back to the pull request.

Same tests, preview address#

A Testsy test is saved with your production address. For a preview you do not need a second test: start the same test at another address, and it explores from there. From the CI docs:

  • GitHub Action: target-url: ${{ github.event.deployment_status.environment_url }}
  • Command-line tool: node testsy-ai.mjs run --project p_… --url "https://pr-42.preview.example.com"
  • API and MCP: url when you start a test or a project.

Vercel reports each finished deployment to GitHub as a deployment status, so a workflow that runs on: deployment_status starts when the preview is ready, with its address in the event. The workflow from the docs does this:

# .github/workflows/testsy-ai.yml
name: Testsy
on:
  deployment_status:
  workflow_dispatch:
jobs:
  qa:
    if: github.event_name == 'workflow_dispatch' || github.event.deployment_status.state == 'success'
    runs-on: ubuntu-latest
    steps:
      - name: Run Testsy tests
        env:
          TESTSY_AI_API_KEY: ${{ secrets.TESTSY_AI_API_KEY }}
          TESTSY_AI_URL: https://app.testsy.dev
          TARGET: ${{ github.event.deployment_status.environment_url }}
        run: |
          curl -fsSL "$TESTSY_AI_URL/cli/testsy-ai.mjs" -o testsy-ai.mjs
          # On a deployment event the tests run against that deploy's address (a preview or staging of the same site).
          node testsy-ai.mjs run --project p_your_project --junit testsy-ai-junit.xml ${TARGET:+--url "$TARGET"}
      - uses: actions/upload-artifact@v4
        if: always()
        with: { name: testsy-ai-junit, path: testsy-ai-junit.xml }

The run's result shows the address it ran against. App tests in the same project are unaffected; they run their build.

Previews behind Deployment Protection#

If your previews ask for a Vercel sign-in, the tester will be stopped at that page like any stranger. Vercel's answer for automation is Protection Bypass for Automation: a secret, created in the project settings, that can be sent as a header or as a query parameter named x-vercel-protection-bypass. Adding x-vercel-set-bypass-cookie=true makes Vercel set a cookie, so the pages that follow load too.

Testsy's browser does not send custom headers, so use the query parameter in the address you pass:

target-url: ${{ github.event.deployment_status.environment_url }}/?x-vercel-protection-bypass=${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }}&x-vercel-set-bypass-cookie=true

Treat the address as a secret: it carries the bypass. The result shows the address the run started at, so keep the result inside your workspace rather than sharing it, and rotate the secret if it leaks. A deployment built before you rotate keeps the old value, as Vercel's page explains.

What a preview run cannot share with production#

  • Sign-in accounts and endpoints belong to the test. A preview that needs different credentials needs its own test.
  • Learned paths belong to the saved address, so a preview run always explores. That makes it a little slower than a replayed production run.
  • Data. A preview usually talks to a staging database. Tests that create records should use made-up details.

Fail the check, keep the evidence#

By default the job fails when a run fails or does not finish; fail-on changes that. Each run prints a watch link anyone can open, live and afterwards, and on GitHub a table of results goes into the job summary. Make the Testsy job a required check on main, and a regression stops at the pull request with its replay attached.

Each run counts toward your plan. Five tests on every push of a busy pull request adds up; many teams run the full set on deployment_status for the final push only, or a short smoke set on every push.

Filed underWebReleases and regressionsCI, schedules and agents