Executive Definition
Regression testing is the practice of re-executing existing tests against a new build to confirm that previously working behaviour still works. The name reflects the failure it exists to catch — a regression, meaning software that has moved backwards in quality by breaking a feature that used to pass.
Every non-trivial change carries regression risk because software modules share memory, database rows, network calls, and shared configuration. A cosmetic CSS change can regress accessibility. A refactored SQL query can regress an unrelated report. Regression testing exists to catch those blast-radius failures before customers do.
The naive regression strategy is 'run every test on every commit', which is what most teams start with. It works up to about 500 tests, becomes painful around 2,000, and becomes impossible past 10,000. At that scale the total suite runtime exceeds the merge cadence — engineers merge faster than the suite finishes — so the pipeline either becomes a bottleneck or gets bypassed. The mature answer is test selection: running only the subset of tests affected by the change under review.
Selection strategies range from crude to sophisticated. The crude version tags tests by feature area and runs the tags matching the changed directory. The sophisticated version tracks per-test code coverage during previous runs, computes a diff against the pending change, and executes only tests whose covered lines intersect the diff. Tools like Playwright's --only-changed flag, pytest-testmon, and Launchable implement this pattern out of the box.
Regression coverage grows monotonically over time — a new test is added for every fixed defect, and rarely deleted. This growth is desirable but requires discipline. Teams that never prune duplicate coverage end up with 8,000 tests that provide the same signal as 3,000, at three times the runtime. Regular deletion of low-value tests is as important to a healthy regression suite as writing new ones. In 2026, the leading benchmark for regression suite health is coverage-per-minute — how much unique behavioural coverage each minute of runtime buys.
Architecture & Production Code
A modern regression pipeline separates the always-on subset from the periodic full sweep. This layered design keeps merge feedback fast without sacrificing the safety net.
┌──────────────────────┐
│ Change-impact model │
│ (git diff + coverage│
│ graph → test set) │
└──────────┬───────────┘
│
┌────────────────────────────┼────────────────────────────┐
│ │ │
▼ ▼ ▼
┌───────────────┐ ┌──────────────────┐ ┌──────────────────┐
│ PR pipeline │ │ Nightly full │ │ Weekly release │
│ Selected │ │ regression │ │ regression + │
│ regression │ │ (all tests) │ │ perf + security │
│ (~5–15 min) │ │ (~2–4 hours) │ │ (~6–10 hours) │
└───────┬───────┘ └────────┬─────────┘ └────────┬─────────┘
│ │ │
▼ ▼ ▼
Merge gate Owner triage Release sign-offThe PR-level track is the one developers feel every day. It must stay under fifteen minutes to preserve merge velocity, which is only possible with a change-impact model that filters the total suite down to the ~10% actually affected by the diff.
The nightly track catches what selection missed. Selection is a probabilistic optimisation — high recall, imperfect precision — and periodic full runs are the belt-and-braces guarantee that no test rots undetected. When a test in the nightly track fails but the corresponding PR passed, the selection model owes you an update.
The weekly track exists for concerns that need a longer window or a production-shaped environment: performance regression, security scans, and cross-browser matrices. Isolating them from the daily pipeline preserves speed for the common case without abandoning the broader coverage.
import { execSync } from "node:child_process";
import { readFileSync } from "node:fs";
// 1. Get files changed against the base branch
const changed = execSync("git diff --name-only origin/main...HEAD")
.toString()
.split("\n")
.filter(Boolean);
// 2. Load the per-test coverage map from the last green run
type CoverageMap = Record<string, string[]>; // testId -> [file, file, ...]
const map: CoverageMap = JSON.parse(
readFileSync(".coverage/test-map.json", "utf8"),
);
// 3. Select every test whose covered files intersect the diff
const selected = new Set<string>();
for (const [testId, files] of Object.entries(map)) {
if (files.some((f) => changed.includes(f))) selected.add(testId);
}
// 4. Emit a grep pattern Playwright / pytest can consume
if (selected.size === 0) {
console.log("@smoke"); // fallback to smoke if nothing intersects
} else {
console.log([...selected].map((t) => `^${t}$`).join("|"));
}Regression vs Smoke vs Retesting
| Aspect | Regression | Smoke | Retesting |
|---|---|---|---|
| Question answered | Did anything else break? | Is the build usable? | Is this specific defect fixed? |
| Scope | Broad, wide | Broad, shallow | Narrow, deep |
| Trigger | Every merge / nightly | Every build | Every defect fix |
| Test data | Existing suite | Existing subset | Reproduction steps from the bug ticket |
| Automation share | 80–95% | 100% | 50–80% |
| Runtime budget | 5 min – 4 hr | ≤ 10 min | Minutes |
Retesting and regression are frequently confused. Retesting proves a specific bug is dead; regression proves nothing else died with it. Both are needed after any non-trivial fix — one without the other is half a job.
Production Debugging Scenarios
Regression pain almost never comes from the tests themselves — it comes from the pipeline around them. These three scenarios cover the vast majority of production regression incidents.
Selected regression passes but nightly full regression fails on the same commit
- Symptom
- PR merged on green, night pipeline reports a broken invoice PDF generator.
- Root cause
- Coverage map was stale — the failing test had been added after the last coverage-collection run.
- Fix
- Rebuild the coverage map on every full nightly run and expire entries older than seven days.
Regression runtime doubled after a Playwright upgrade
- Symptom
- Full suite that ran in 90 minutes now runs in 180, with no obvious slow test.
- Root cause
- New default trace collection wrote large artefacts to slow disk in CI.
- Fix
- Set trace: 'retain-on-failure' and screenshot: 'only-on-failure' in the config; move artefacts to a RAM disk in CI.
Regression suite flags 30 failures after a shared-fixture rewrite
- Symptom
- All failures share the same setup step and vanish on rerun.
- Root cause
- The rewritten fixture no longer waits for seed data to commit before the first test starts.
- Fix
- Await the seed transaction and add a readiness assertion before yielding the fixture; run the fixture once per worker, not once per test.
Practice this concept in a real QA interview
Run a live mock with our AI Interview Coach, tune your resume with the ATS Resume Reviewer, and screen live listings on the QA Jobs Radar.