Screenshot Diff Testing

Commit screenshots to git. Tests overwrite with new captures — git diff determines pass/fail, requiring human approval to commit changes.

The pattern

CRITICAL: Test Behavior

Tests ALWAYS overwrite screenshot files with new captures, then use git diff to determine pass/fail. Tests never skip writing files or do image comparison.

CRITICAL: Agent Restrictions

Coding agents must NEVER commit screenshot files unless the user explicitly instructs. Committing bypasses the human approval mechanism.

Writing a screenshot test

Each test captures a screenshot with Playwright and passes it to compareOrCreateSnapshot. The function overwrites the snapshot file and checks git diff to determine whether the visual output changed:

import test from "node:test";
import { chromium } from "playwright";

test("home-page.png", async () => {
  const browser = await chromium.launch({
    headless: true,
  });
  const page = await browser.newPage({
    viewport: { width: 1280, height: 720 },
  });
  await page.goto("http://localhost:3000");
  await page.waitForLoadState("networkidle");
  const screenshot = await page.screenshot({
    fullPage: true,
  });
  await compareOrCreateSnapshot(
    "home-page.png",
    screenshot,
  );
  await browser.close();
});

Implementing compareOrCreateSnapshot

The compareOrCreateSnapshot function always writes the new screenshot to disk, then shells out to git diff --quiet to check whether the file changed. If git reports a difference, the test throws with instructions to review the diff:

import { execSync } from "child_process";
import { writeFileSync } from "fs";
import { join } from "path";

const SNAPSHOTS_DIR = "test/snapshots";

async function compareOrCreateSnapshot(filename, screenshot) {
  const snapshotPath = join(SNAPSHOTS_DIR, filename);
  
  writeFileSync(snapshotPath, screenshot);
  
  try {
    execSync(
      `git diff --quiet HEAD -- "${snapshotPath}"`,
      { stdio: "ignore" },
    );
    console.log(`✅ Visual test passed: ${filename}`);
  } catch (error) {
    throw new Error(oneLine`
      Screenshot diff test failed: visual change detected in ${filename}. Review
      with 'git diff ${snapshotPath}', then commit.
    `);
  }
}

First Run Behavior

If the screenshot doesn't exist in git, test fails with: "No baseline snapshot found - commit this file to establish baseline".

Locating changed pixels

When a screenshot differs, don't eyeball two images. Generate a diff image that highlights exactly which pixels changed and where:

import { readFileSync, writeFileSync } from "fs";
import { execSync } from "child_process";
import { PNG } from "pngjs";
import pixelmatch from "pixelmatch";

function diffSnapshot(filename) {
  const snapshotPath = join(SNAPSHOTS_DIR, filename);
  const before = PNG.sync.read(
    execSync(`git show HEAD:${snapshotPath}`),
  );
  const after = PNG.sync.read(
    readFileSync(snapshotPath),
  );
  const { width, height } = before;
  const diff = new PNG({ width, height });

  const changed = pixelmatch(
    before.data, after.data, diff.data,
    width, height, { threshold: 0 },
  );

  if (changed === 0) return null;

  // Find bounding box of changed region
  let minX = width, minY = height;
  let maxX = 0, maxY = 0;
  for (let y = 0; y < height; y++) {
    for (let x = 0; x < width; x++) {
      const i = (y * width + x) * 4;
      if (
        diff.data[i] === 255
        && diff.data[i + 1] === 0
      ) {
        minX = Math.min(minX, x);
        minY = Math.min(minY, y);
        maxX = Math.max(maxX, x);
        maxY = Math.max(maxY, y);
      }
    }
  }

  const diffPath = snapshotPath.replace(
    ".png", "-diff.png",
  );
  writeFileSync(diffPath, PNG.sync.write(diff));

  return {
    changed,
    total: width * height,
    bbox: { minX, minY, maxX, maxY },
    diffPath,
  };
}

The diff image renders changed pixels in red against a faded version of the original. Open it to see exactly what moved — a 1px border shift, a focus ring change, a layout reflow. The bounding box tells you where to look without scanning the entire image.

This turns "the screenshot changed by 19 bytes" into "126,000 pixels differ in the grid area at y:128–888 because box-sizing changed the sticky header height by 1px."

Never assume a small file-size difference is safe to ignore. The entire point of screenshot diff testing is that we usually expect no change whatsoever. Any difference — even a few bytes — means something moved. Find out what.