Preview vs Production Workflow

The gold standard for modern visual regression: Catching bugs before they merge by directly comparing temporary Preview environments to the Live Production site.

The Strategy

Traditional visual testing requires you to manage a "golden image" baseline in your git repository. This is painful—it bloats your repo, creates merge conflicts, and the baselines are often out of date.

RegressionBot recommendation: Use your live Production site as the baseline. When a developer opens a PR, compare their Preview URL against the Live URL. No images in git, zero maintenance.

Zero Baseline Bloat

Stop storing thousands of PNGs in your Git history. Let the cloud handle it.

Self-Healing

Baselines naturally update automatically as your production site evolves.

The 3-Step Lifecycle

Automated Check CI/CD

On every push, GitHub Actions waits for your preview deployment to finish and then triggers a RegressionBot job. It renders screenshots of both the preview URL and production URL to compare them.

Visual Review Developer

If regressions are detected, the PR check fails. The developer follows the link in the PR comment to inspect the diff images.

Approval ChatOps

If the change is intentional (e.g., a new UI feature), the developer comments /approve-visual on the PR. RegressionBot approves the job, and the PR check turns green.

GitHub Actions Configuration

Add this workflow to your repository to automatically test on every PR.

name: Visual Check
on:
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write # Required to post comments
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: 20

      # Replace this with your preview provider's output.
      - name: Set preview URL
        run: echo "PREVIEW_URL=https://preview.myapp.com" >> "$GITHUB_ENV"

      - name: Install RegressionBot SDK
        run: npm install regressionbot

      - name: Run RegressionBot and write report
        run: |
          node --input-type=module <<'EOF'
          import fs from 'node:fs';
          import { RegressionBot } from 'regressionbot';

          const client = new RegressionBot(process.env.REGRESSIONBOT_API_KEY);
          const job = await client
            .test(process.env.PREVIEW_URL)
            .against('https://myapp.com')
            .forProject('my-web-app')
            .on(['Desktop Chrome', 'iPhone 12'])
            .sitemap(process.env.PREVIEW_URL + '/sitemap.xml')
            .scan('/pages/**')
            .run();

          await job.waitForCompletion();
          const summary = await job.getSummary();

          const lines = [
            '## Visual Regression Results',
            '',
            '**Job:** ' + job.jobId,
            '**Score:** ' + summary.overallScore + '/100',
            '**Regressions:** ' + summary.regressionCount,
            ''
          ];

          for (const regression of summary.regressions || []) {
            const diff = (regression.diffPercentage ?? 0).toFixed(3);
            lines.push(
              '### ' + regression.url,
              '',
              '**Variant:** ' + regression.variantName,
              '**Diff:** ' + diff + '%',
              '**Visual Match:** ' + regression.visualMatchScore + '/100',
              ''
            );

            if (regression.regressionbotSummary) {
              lines.push(
                '> **RegressionBot Summary:**',
                '> ' + regression.regressionbotSummary.replace(/\n/g, '\n> '),
                ''
              );
            }

            if (regression.diffUrl) {
              lines.push('[Open diff image](' + regression.diffUrl + ')', '');
            }
          }

          fs.writeFileSync('regressionbot-report.md', lines.join('\n'));
          if (summary.regressionCount > 0) process.exit(1);
          EOF
        env:
          REGRESSIONBOT_API_KEY: ${{ secrets.REGRESSIONBOT_API_KEY }}

      - name: Comment visual summary on PR
        if: always()
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const body = fs.existsSync('regressionbot-report.md')
              ? fs.readFileSync('regressionbot-report.md', 'utf8')
              : 'RegressionBot did not produce a report. Check the workflow logs.';

            await github.rest.issues.createComment({
              owner: context.repo.owner,
              repo: context.repo.repo,
              issue_number: context.issue.number,
              body
            });

Enable ChatOps Approval

To allow developers to approve changes directly from comments, add this "Approval" workflow.

on:
  issue_comment:
    types: [created]

jobs:
  approve:
    if: github.event.issue.pull_request && startsWith(github.event.comment.body, '/approve-visual')
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: 20

      - name: Approve RegressionBot Job
        run: |
          JOB_ID=$(echo "${{ github.event.comment.body }}" | awk '{print $2}')
          npx regressionbot approve "$JOB_ID"
        env:
          REGRESSIONBOT_API_KEY: ${{ secrets.REGRESSIONBOT_API_KEY }}