ISTQB Glossary — Chapter 6: Test Tools

In plain English

Show version A to some users and version B to others, then compare metrics.

From real testing work

Two weeks before a release the test manager compares planned versus executed cases, open defects by severity, and remaining effort. A/B Testing is where that number comes from — and the exam cares that it is used to inform a decision, not just published.

In plain English

Showing version A to some users and B to others to see which wins.

From real testing work

A signup form passes the automated scanner with zero violations, then fails the moment a tester unplugs the mouse: focus jumps from the email field straight to the footer. A/B Testing work catches exactly that gap between tool output and a real person completing the task.

const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
// then: keyboard-only pass, Tab order recorded in the charter notes

In plain English

Selenium’s API for complex interactions like drag-and-drop and hovering.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Actions Class is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

A polished test report with charts, history, and step-level detail.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Allure Report is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

A YAML-based automation tool for configuring servers.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Ansible is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

The most widely used open-source performance testing tool.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Apache JMeter is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

A high-throughput system for streaming events between services.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Apache Kafka is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

A front door for your APIs handling auth, routing, and rate limits.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under api gateway: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

The Selenium equivalent for mobile apps — one API, iOS and Android.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a appium that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

A visual testing platform that uses AI to filter out irrelevant diffs.

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so applitools is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

A GitOps tool that keeps your Kubernetes cluster in sync with what’s in Git.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Argo CD shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

The check inside a test that decides pass or fail.

From real testing work

Where you meet this in real work: during a sprint, assertion is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Prefer one logical assertion per test — makes failures easier to diagnose.

In plain English

The most widely used engine for automated accessibility checks.

From real testing work

A signup form passes the automated scanner with zero violations, then fails the moment a tester unplugs the mouse: focus jumps from the email field straight to the footer. axe-core work catches exactly that gap between tool output and a real person completing the task.

const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
// then: keyboard-only pass, Tab order recorded in the charter notes

In plain English

Microsoft’s test management inside Azure DevOps.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Azure DevOps Test Plans is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Cucumber for Python.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Behave is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Speed, repeatability, wider coverage, faster feedback, fewer manual errors.

From real testing work

After a sprint of bug fixes the team runs the unit suite with coverage on and finds the discount calculator sits at 62%. Benefits of Test Automation tells them which lines or branches never executed — usually the error paths nobody wrote a test for.

npx vitest run --coverage
# ---------------------|---------|----------|
# File                 | % Stmts | % Branch |
# discount.ts          |   62.5  |   41.6   |

In plain English

A cloud service for running JMeter-style perf tests at massive scale.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. BlazeMeter is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

Record actions once, replay them as an automated test.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Capture / Replay Tool is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

A bot that kills random servers in production so the system is forced to survive failure.

From real testing work

A tester files: "Checkout returns HTTP 500 when the cart contains a gift card and a subscription." Steps, expected, actual, environment, build number. Chaos Monkey is one of the fields or states in that workflow, and getting the term right is what makes the report actionable for the developer picking it up at 9am.

In plain English

A GUI proxy for inspecting HTTP traffic during mobile and web testing.

From real testing work

The Android app works perfectly on the team's Pixel and crashes on a three-year-old budget device on a 3G connection. Charles Proxy is what keeps that from reaching production: a device matrix chosen from real analytics, not from what is on people's desks.

In plain English

A visual testing platform designed for Storybook-based component libraries.

From real testing work

A requirements review on a "forgot password" story raises that the acceptance criteria never say what happens to an unverified email address. That is chromatic in practice — an issue found in a document, before a single line of code exists, at a fraction of the cost of finding it in UAT.

In plain English

The automated pipeline that builds, tests, and ships code on every change.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. CI/CD shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

A popular cloud CI/CD platform.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. CircleCI shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

Measures which lines/branches your tests actually ran.

From real testing work

After a sprint of bug fixes the team runs the unit suite with coverage on and finds the discount calculator sits at 62%. Code Coverage Tool tells them which lines or branches never executed — usually the error paths nobody wrote a test for.

npx vitest run --coverage
# ---------------------|---------|----------|
# File                 | % Stmts | % Branch |
# discount.ts          |   62.5  |   41.6   |

In plain English

A recorder that writes Playwright test code as you click around.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Codegen is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

UFT/QTP, TestComplete, LoadRunner — paid tools with vendor support.

From real testing work

Where you meet this in real work: during a sprint, commercial tool is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Commercial tools give you vendor support and SLA — open-source gives you flexibility.

In plain English

A diff engine used to compare actual vs expected output automatically.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Comparator is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

A portable, isolated bundle of an app and its dependencies.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Container shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

Jenkins, GitHub Actions, GitLab CI, CircleCI — build, test and report on every commit.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Continuous Integration Tool is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

JaCoCo, Istanbul, Coverage.py — reports which lines/branches your tests actually hit.

From real testing work

After a sprint of bug fixes the team runs the unit suite with coverage on and finds the discount calculator sits at 62%. Coverage Tool tells them which lines or branches never executed — usually the error paths nobody wrote a test for.

npx vitest run --coverage
# ---------------------|---------|----------|
# File                 | % Stmts | % Branch |
# discount.ts          |   62.5  |   41.6   |

In plain English

A syntax for finding elements the way CSS styles them.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. CSS Selector is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Cucumber for Java projects.

From real testing work

Where you meet this in real work: during a sprint, cucumber-jvm is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Cucumber-JVM integrates with JUnit, TestNG, and Spring — dominant in Java BDD.

In plain English

Cypress’s network-mocking command.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a cy.intercept that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

A developer-friendly E2E test framework built for modern JavaScript apps.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Cypress is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Cypress feature to auto-retry failing tests.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Cypress Retries is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Ship the code live but keep it hidden so you can test it in production without users seeing it.

From real testing work

Staging has last quarter's anonymised production snapshot, seeded with fifteen deliberately broken accounts. Dark Launch covers how that is created, versioned and restored — because a suite that passes only on Tuesday's data is not a suite you can trust.

make db-reset && make seed-fixtures ENV=staging
# fixtures are versioned with the tests, in the same commit

In plain English

Security scanning of a live system — finds issues SAST can't see.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under dast: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

Take real data and scramble the sensitive bits (names, SSNs) so the shape is real but the values are fake.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a data masking that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

One test script + a table of inputs and expected outputs = many test runs.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Data-Driven Testing is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Chrome DevTools, gdb, IDE debuggers — step through code and inspect state.

From real testing work

A tester files: "Checkout returns HTTP 500 when the cart contains a gift card and a subscription." Steps, expected, actual, environment, build number. Debugging Tool is one of the fields or states in that workflow, and getting the term right is what makes the report actionable for the developer picking it up at 9am.

In plain English

JIRA, Bugzilla, Mantis — where bug reports live and move through their state model.

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so defect tracking tool is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

Ansible, Terraform, ArgoCD, Octopus — push builds into environments consistently.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Deployment Tool is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

An E2E testing framework built specifically for React Native apps.

From real testing work

The Android app works perfectly on the team's Pixel and crashes on a three-year-old budget device on a 3G connection. Detox is what keeps that from reaching production: a device matrix chosen from real analytics, not from what is on people's desks.

In plain English

A tool that packages an app and everything it needs into a portable container.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Docker shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

A YAML file that starts many containers together as one app.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Docker Compose shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

The recipe file that tells Docker how to build an image.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Dockerfile is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Fake caller code that invokes the module under test when the real caller doesn't exist yet.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a driver that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

A piece of code that calls a low-level module so it can be tested in isolation.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a driver that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

An empty placeholder object used to satisfy a signature.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a dummy object that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

Valgrind, AddressSanitizer — watches a running program for memory leaks, race conditions, undefined behavior.

From real testing work

Where you meet this in real work: during a sprint, dynamic analysis tool is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Dynamic analysis catches defects only when the code actually runs — pair with high-coverage tests.

In plain English

Software that pretends to be a different device or system for testing.

From real testing work

The Android app works perfectly on the team's Pixel and crashes on a three-year-old budget device on a 3G connection. Emulator is what keeps that from reaching production: a device matrix chosen from real analytics, not from what is on people's desks.

In plain English

A throw-away environment created for one PR or one test cycle.

From real testing work

Staging has last quarter's anonymised production snapshot, seeded with fifteen deliberately broken accounts. Ephemeral Environment covers how that is created, versioned and restored — because a suite that passes only on Tuesday's data is not a suite you can trust.

make db-reset && make seed-fixtures ENV=staging
# fixtures are versioned with the tests, in the same commit

In plain English

Google’s official framework for testing Android apps from inside the app.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Espresso is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Waiting for a specific condition, like "button is clickable", before continuing.

From real testing work

Where you meet this in real work: during a sprint, explicit wait is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Explicit waits (WebDriverWait) are the recommended synchronization approach.

In plain English

A popular Java library that generates rich HTML test reports.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. ExtentReports is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

A working but simplified stand-in — e.g. an in-memory DB.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a fake that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

A switch that turns features on or off without redeploying.

From real testing work

Where you meet this in real work: during a sprint, feature toggle is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. LaunchDarkly, Split, and Unleash are common flag platforms — testers verify variant delivery and rollback.

In plain English

A classic Windows HTTP debugging proxy from Progress (formerly Telerik).

From real testing work

A tester files: "Checkout returns HTTP 500 when the cart contains a gift card and a subscription." Steps, expected, actual, environment, build number. Fiddler is one of the fields or states in that workflow, and getting the term right is what makes the report actionable for the developer picking it up at 9am.

In plain English

A test that sometimes passes and sometimes fails on the same build — usually a timing or environment issue.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Flaky Test is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

A customizable wait that polls at set intervals and can ignore certain errors.

From real testing work

A tester files: "Checkout returns HTTP 500 when the cart contains a gift card and a subscription." Steps, expected, actual, environment, build number. Fluent Wait is one of the fields or states in that workflow, and getting the term right is what makes the report actionable for the developer picking it up at 9am.

In plain English

A Scala-based performance testing tool known for beautiful reports.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Gatling is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

The standard version control system used everywhere.

From real testing work

Two weeks before a release the test manager compares planned versus executed cases, open defects by severity, and remaining effort. Git is where that number comes from — and the exam cares that it is used to inform a decision, not just published.

In plain English

A separate line of development in Git.

From real testing work

After a bad release the team runs a root-cause session and changes the definition of done rather than adding more tests at the end. Git Branch sits on that side of the fence: preventive and process-oriented, distinct from executing tests against a build.

In plain English

Combining changes from one branch into another.

From real testing work

Where you meet this in real work: during a sprint, git merge is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Merge preserves history; rebase rewrites it. Know both.

In plain English

Rewriting your branch to sit on top of another branch for a clean history.

From real testing work

Where you meet this in real work: during a sprint, git rebase is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Never rebase shared branches — only local, unpushed work.

In plain English

Built-in CI/CD for GitHub repos, configured with YAML workflows.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. GitHub Actions is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

GitLab’s built-in CI/CD system, configured in .gitlab-ci.yml.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. GitLab CI/CD shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

Managing your infrastructure and deployments through Git commits.

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so gitops is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

Testing GraphQL APIs — one endpoint, flexible queries.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. GraphQL Testing is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Testing gRPC APIs — the high-performance protocol used between microservices.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. gRPC Testing is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

A browser with no visible window — perfect for running tests in CI.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Headless Browser is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

The package manager for Kubernetes.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Helm shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

Bringing a server-rendered page to life on the client with JavaScript.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Hydration is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Link checkers like broken-link-checker or W3C Link Checker.

From real testing work

Where you meet this in real work: during a sprint, hyperlink testing tool is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Broken links hurt SEO and UX — cheap to catch, expensive to leave.

In plain English

A test suite that is heavy on UI and manual, light on unit — the opposite of the pyramid.

From real testing work

The same discount rule is verified three times: in a unit test on the calculator, in an API test against the pricing service, and once through the UI at checkout. Ice-Cream Cone Anti-Pattern names one of those layers — different test basis, different owner, different defects found.

// component level
expect(calcDiscount(100, "SAVE10")).toBe(90);
// system level
const res = await api.post("/cart/apply", { code: "SAVE10" });
expect(res.body.total).toBe(90);

In plain English

Doing the same thing twice has the same effect as doing it once.

From real testing work

Where you meet this in real work: during a sprint, idempotency is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Test idempotency by replaying the same request and verifying no duplicate side effects — critical for payments and webhooks.

In plain English

A global wait telling Selenium to keep trying to find elements for N seconds.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Implicit Wait is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

The most widely used service mesh, built on Envoy.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under istio: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

A Selenium hook to run JavaScript inside the tested page.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. JavaScript Executor is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

The most widely used open-source CI/CD server.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Jenkins is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

The most popular JavaScript test framework, batteries included.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a jest that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

Atlassian’s issue and workflow tracker.

From real testing work

Where you meet this in real work: during a sprint, jira is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Jira is the default backlog/defect tracker for most QA orgs.

In plain English

The standard unit testing framework for Java.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. JUnit is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

A developer-friendly JS-based load testing tool.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. k6 is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

A Gherkin-based framework that tests APIs, mocks, and performance from one tool.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a karate dsl that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

A commercial low-code automation platform covering web, mobile, and API.

From real testing work

The Android app works perfectly on the team's Pixel and crashes on a three-year-old budget device on a 3G connection. Katalon Studio is what keeps that from reaching production: a device matrix chosen from real analytics, not from what is on people's desks.

In plain English

Write tests as spreadsheets of keywords ('login', 'add to cart') instead of code — non-coders can maintain them.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Keyword-Driven Testing is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

The standard system for running and scaling containers in production.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Kubernetes shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

A K8s resource that keeps a set of pods running and updates them safely.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Kubernetes Deployment shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

The Kubernetes resource that exposes services to outside traffic.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Kubernetes Ingress shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

A stable network address for a group of pods, even as they come and go.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Kubernetes Service shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

Google’s built-in tool that grades a page on perf, a11y, and SEO.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Lighthouse is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

A lightweight alternative to Istio.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Linkerd shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

The nagging tool that catches typos, unused vars, and style issues before they reach review.

From real testing work

A requirements review on a "forgot password" story raises that the acceptance criteria never say what happens to an unverified email address. That is linter in practice — an issue found in a document, before a single line of code exists, at a fraction of the cost of finding it in UAT.

In plain English

The classic enterprise performance testing tool.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. LoadRunner is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

How your test finds a button, link, or field on the page.

From real testing work

A signup form passes the automated scanner with zero violations, then fails the moment a tester unplugs the mouse: focus jumps from the email field straight to the footer. Locator work catches exactly that gap between tool output and a real person completing the task.

const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
// then: keyboard-only pass, Tab order recorded in the charter notes

In plain English

A Python load testing tool where each virtual user is a class you write.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Locust is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

A new mobile test framework that uses simple YAML flows.

From real testing work

The Android app works perfectly on the team's Pixel and crashes on a three-year-old budget device on a 3G connection. Maestro is what keeps that from reaching production: a device matrix chosen from real analytics, not from what is on people's desks.

In plain English

A buffer that holds messages while producers and consumers work at different speeds.

From real testing work

Where you meet this in real work: during a sprint, message queue is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Testing queues covers ordering, at-least-once vs. exactly-once delivery, and dead-letter handling.

In plain English

An open-source proxy for capturing and tweaking HTTPS requests during testing.

From real testing work

Where you meet this in real work: during a sprint, mitmproxy is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. mitmproxy is scriptable in Python — great for API test debugging and security testing.

In plain English

A popular JS test runner, often paired with Chai for assertions.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Mocha is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

A fake dependency you configure to expect specific calls and assert on them afterwards.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a mock object that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

A fake API you control so tests aren’t blocked by real backends.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a mock server that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

A library that fakes API responses in browser and Node tests.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a mock service worker that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

A GUI tool to spin up fake REST APIs in minutes.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a mockoon that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

New Relic, Datadog, Prometheus — watch the app in real time.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Monitoring Tool is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

The unit test framework built into Visual Studio.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. MSTest is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

A commercial perf testing platform used in enterprise DevOps pipelines.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. NeoLoad is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

The CLI that runs Postman collections in your build pipeline.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Newman shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

A JS E2E framework built on WebDriver.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a nightwatch.js that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

JUnit for .NET.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. NUnit is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Testing that the app still works when there’s no internet.

From real testing work

Where you meet this in real work: during a sprint, offline testing is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Test offline creates, offline reads from cache, and online reconciliation of queued writes.

In plain English

Selenium, JMeter, Playwright, JUnit — free tools with community support.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Open-Source Tool is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

A standard format for describing what an API can do.

From real testing work

A story reads "As a returning customer I can reuse a saved card." Before estimating it, the three amigos add acceptance criteria for an expired card and a card removed from the account. OpenAPI is the artefact or link involved, and it is what lets you prove later that every requirement has at least one test.

In plain English

A CLI tool for running accessibility checks in CI.

From real testing work

A signup form passes the automated scanner with zero violations, then fails the moment a tester unplugs the mouse: focus jumps from the email field straight to the footer. Pa11y work catches exactly that gap between tool output and a real person completing the task.

const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
// then: keyboard-only pass, Tab order recorded in the charter notes

In plain English

The most popular consumer-driven contract testing tool.

From real testing work

A story reads "As a returning customer I can reuse a saved card." Before estimating it, the three amigos add acceptance criteria for an expired card and a card removed from the account. Pact is the artefact or link involved, and it is what lets you prove later that every requirement has at least one test.

In plain English

A Selenium pattern that wires up page elements via annotations.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Page Factory is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Wrap each page/screen in a class so tests interact with methods (login()) not raw selectors.

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so page object model is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

Playwright’s network interception API.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. page.route is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Running tests at the same time instead of one after another.

From real testing work

Where you meet this in real work: during a sprint, parallel execution is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Requires test isolation — flaky tests explode in parallel runs.

In plain English

A tool that catches unintended visual changes across your app’s pages.

From real testing work

A requirements review on a "forgot password" story raises that the acceptance criteria never say what happens to an unverified email address. That is percy in practice — an issue found in a document, before a single line of code exists, at a fraction of the cost of finding it in UAT.

In plain English

JMeter, Gatling, k6, LoadRunner — drive high load and record response times.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Performance Testing Tool is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

The pipeline lives in your repo as a YAML or DSL file.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Pipeline as Code shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

A modern, fast browser automation tool that competes with Selenium and Cypress.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Playwright is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Playwright’s way to inject browser context, pages, and helpers into tests.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Playwright Fixtures is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

The smallest deployable unit in Kubernetes — usually a single container.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Pod shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

The most popular API testing and exploration tool.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a postman that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

A saved bundle of API requests you can share and run together.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Postman Collection is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

The last environment before production — mirrors it as closely as possible.

From real testing work

Staging has last quarter's anonymised production snapshot, seeded with fifteen deliberately broken accounts. Pre-Production Environment covers how that is created, versioned and restored — because a suite that passes only on Tuesday's data is not a suite you can trust.

make db-reset && make seed-fixtures ENV=staging
# fixtures are versioned with the tests, in the same commit

In plain English

Measures where your code spends its time and memory.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Profiler is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

Rolling out features to small groups first before everyone gets them.

From real testing work

Staging has last quarter's anonymised production snapshot, seeded with fifteen deliberately broken accounts. Progressive Delivery covers how that is created, versioned and restored — because a suite that passes only on Tuesday's data is not a suite you can trust.

make db-reset && make seed-fixtures ENV=staging
# fixtures are versioned with the tests, in the same commit

In plain English

A small pilot to prove a tool actually works in your environment before you commit budget.

From real testing work

Two weeks before a release the test manager compares planned versus executed cases, open defects by severity, and remaining effort. Proof of Concept is where that number comes from — and the exam cares that it is used to inform a decision, not just published.

In plain English

The GitHub/GitLab mechanism used to propose and review code changes.

From real testing work

A requirements review on a "forgot password" story raises that the acceptance criteria never say what happens to an unverified email address. That is pull request in practice — an issue found in a document, before a single line of code exists, at a fraction of the cost of finding it in UAT.

In plain English

Google’s Node library for automating Chrome.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Puppeteer is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Testing web apps that install and work offline like native apps.

From real testing work

Where you meet this in real work: during a sprint, pwa testing is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Verify service worker cache strategies, offline flows, and add-to-home-screen.

In plain English

The de-facto Python test framework.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. pytest is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Tricentis’ enterprise test management platform.

From real testing work

Where you meet this in real work: during a sprint, qtest is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. qTest integrates tightly with Tosca and Jira.

In plain English

A pass/fail check the build must clear before moving to the next stage.

From real testing work

After a sprint of bug fixes the team runs the unit suite with coverage on and finds the discount calculator sits at 62%. Quality Gate tells them which lines or branches never executed — usually the error paths nobody wrote a test for.

npx vitest run --coverage
# ---------------------|---------|----------|
# File                 | % Stmts | % Branch |
# discount.ts          |   62.5  |   41.6   |

In plain English

A popular open-source message broker for queueing work between services.

From real testing work

Where you meet this in real work: during a sprint, rabbitmq is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. RabbitMQ is queue-first; Kafka is log-first — know the difference.

In plain English

Modern name for capture/replay — great for smoke tests, poor for maintainable suites.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Record & Playback is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Selenium IDE, Katalon Recorder — record a browser session, replay it as a test.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Record and Playback Tool is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

JIRA, DOORS, Jama — where requirements are stored, versioned and traced.

From real testing work

A story reads "As a returning customer I can reuse a saved card." Before estimating it, the three amigos add acceptance criteria for an expired card and a card removed from the account. Requirements Management Tool is the artefact or link involved, and it is what lets you prove later that every requirement has at least one test.

In plain English

Testing REST APIs — the URLs and JSON payloads behind most web apps.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under rest api testing: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

A Java library for writing readable REST API tests.

From real testing work

Where you meet this in real work: during a sprint, rest assured is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Given/When/Then chaining and JSONPath validation are core REST Assured skills.

In plain English

Upfront cost, maintenance burden, false confidence, brittle scripts, wrong tests automated.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Risks of Test Automation is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

A keyword-driven, human-readable automation framework.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Robot Framework is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Replacing old servers with new ones a few at a time.

From real testing work

Staging has last quarter's anonymised production snapshot, seeded with fifteen deliberately broken accounts. Rolling Deployment covers how that is created, versioned and restored — because a suite that passes only on Tuesday's data is not a suite you can trust.

make db-reset && make seed-fixtures ENV=staging
# fixtures are versioned with the tests, in the same commit

In plain English

The standard BDD-style test framework for Ruby.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. RSpec is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

A safe playground environment isolated from production data.

From real testing work

The same discount rule is verified three times: in a unit test on the calculator, in an API test against the pricing service, and once through the UI at checkout. Sandbox Environment names one of those layers — different test basis, different owner, different defects found.

// component level
expect(calcDiscount(100, "SAVE10")).toBe(90);
// system level
const res = await api.post("/cart/apply", { code: "SAVE10" });
expect(res.body.total).toBe(90);

In plain English

Security scanning of source code — catches vulnerabilities before running.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under sast: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

Burp Suite, OWASP ZAP, Nessus — scan for vulnerabilities and simulate attacks.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under security testing tool: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

The most widely used open-source browser automation toolkit.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a selenium that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

A cluster that runs Selenium tests on many browsers and machines at once.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Selenium Grid is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

A Selenium record-and-playback browser extension.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Selenium IDE is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

A layer under microservices that handles their talking-to-each-other.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under service mesh: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

Fake entire services (payments, third-party APIs) so tests aren't blocked by them.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a service virtualization that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

A script that runs behind a webpage to enable offline mode and push notifications.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Service Worker is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

The before-each and after-each hooks that prep and clean up around tests.

From real testing work

Staging has last quarter's anonymised production snapshot, seeded with fifteen deliberately broken accounts. Setup and Teardown covers how that is created, versioned and restored — because a suite that passes only on Tuesday's data is not a suite you can trust.

make db-reset && make seed-fixtures ENV=staging
# fixtures are versioned with the tests, in the same commit

In plain English

Sending live traffic to a new version silently to test it under real load.

From real testing work

Staging has last quarter's anonymised production snapshot, seeded with fifteen deliberately broken accounts. Shadow Deployment covers how that is created, versioned and restored — because a suite that passes only on Tuesday's data is not a suite you can trust.

make db-reset && make seed-fixtures ENV=staging
# fixtures are versioned with the tests, in the same commit

In plain English

Testing early AND in production — quality everywhere in the pipeline.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Shift Everywhere shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

Test as early as possible — reviews, unit tests, static analysis, dev-time checks.

From real testing work

A requirements review on a "forgot password" story raises that the acceptance criteria never say what happens to an unverified email address. That is shift-left in practice — an issue found in a document, before a single line of code exists, at a fraction of the cost of finding it in UAT.

In plain English

Keep testing after release — with monitoring, feature flags, A/B tests and canaries.

From real testing work

Two weeks before a release the test manager compares planned versus executed cases, open defects by severity, and remaining effort. Shift-Right is where that number comes from — and the exam cares that it is used to inform a decision, not just published.

In plain English

A model that mimics part of a real system for testing.

From real testing work

The Android app works perfectly on the team's Pixel and crashes on a three-year-old budget device on a 3G connection. Simulator is what keeps that from reaching production: a device matrix chosen from real analytics, not from what is on people's desks.

In plain English

Testing older XML-based web services.

From real testing work

A tester files: "Checkout returns HTTP 500 when the cart contains a gift card and a subscription." Steps, expected, actual, environment, build number. SOAP API Testing is one of the fields or states in that workflow, and getting the term right is what makes the report actionable for the developer picking it up at 9am.

In plain English

Cucumber for .NET/C#.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. SpecFlow is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

A stand-in that records calls made to it for later inspection.

From real testing work

A requirements review on a "forgot password" story raises that the acceptance criteria never say what happens to an unverified email address. That is spy in practice — an issue found in a document, before a single line of code exists, at a fraction of the cost of finding it in UAT.

In plain English

Testing that server-rendered pages match what the client shows after hydration.

From real testing work

Where you meet this in real work: during a sprint, ssr testing is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. SSR bugs typically show as hydration mismatches — HTML from server disagrees with client React tree.

In plain English

SonarQube, ESLint, SpotBugs — analyzes code without running it and flags problems.

From real testing work

A requirements review on a "forgot password" story raises that the acceptance criteria never say what happens to an unverified email address. That is static analysis tool in practice — an issue found in a document, before a single line of code exists, at a fraction of the cost of finding it in UAT.

In plain English

A tool for developing and testing UI components on their own.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Storybook is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Testing Stripe payment flows with test cards and sandbox webhooks.

From real testing work

The same discount rule is verified three times: in a unit test on the calculator, in an API test against the pricing service, and once through the UI at checkout. Stripe Testing names one of those layers — different test basis, different owner, different defects found.

// component level
expect(calcDiscount(100, "SAVE10")).toBe(90);
// system level
const res = await api.post("/cart/apply", { code: "SAVE10" });
expect(res.body.total).toBe(90);

In plain English

Fake code you drop in for a not-yet-built dependency so your module can be tested.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a stub that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

A Node library for writing HTTP tests against Express-style apps.

From real testing work

Where you meet this in real work: during a sprint, supertest is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. SuperTest is the standard for Node/Express API integration tests.

In plain English

The toolset that grew into OpenAPI — docs, editor, codegen.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Swagger is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Fake but realistic data used in testing so you don't touch real PII.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a synthetic data that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

The most popular tool for declaring cloud infrastructure as code.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Terraform shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

Letting tools do the repetitive parts of testing — usually execution, but also data generation and reporting.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Test Automation is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

The reusable structure around your automated tests: base classes, utilities, reporting, configuration.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Test Automation Framework is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Bulk of tests at unit level, fewer at API level, fewest at UI — cheap, fast, stable.

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so test automation pyramid is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

Software that generates realistic, compliant test data (Faker, Mockaroo, GenRocket).

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a test data preparation tool that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

Tools that read a model, decision table or classification tree and generate test cases.

From real testing work

A loan-approval rule that depends on income, credit score and existing debt has eight input combinations. Modelling it with test design tool turns an argument in a refinement meeting into a table the BA, the developer and the tester all sign off on, and each row becomes one test case.

Income OK | Score >700 | Debt <40% | Expected
    Y     |     Y      |    Y     | APPROVE
    Y     |     Y      |    N     | REFER
    Y     |     N      |    Y     | REFER
    N     |     -      |    -     | DECLINE

In plain English

Any stand-in object used in place of a real dependency.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a test double that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

Spinning up a clean test environment on demand instead of hand-crafting one.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Test Environment Provisioning is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Selenium, Cypress, Playwright — runs your automated scripts and reports pass/fail.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Test Execution Tool is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

The setup and data a test needs to run reliably.

From real testing work

Where you meet this in real work: during a sprint, test fixture is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Pytest fixtures and Playwright fixtures are common examples — reusable, composable, scoped.

In plain English

The scaffolding around the code under test — the drivers that call it and the stubs that fake its dependencies.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a test harness that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

Each test stands on its own — no leftover data or order dependency.

From real testing work

After a bad release the team runs a root-cause session and changes the definition of done rather than adding more tests at the end. Test Isolation sits on that side of the fence: preventive and process-oriented, distinct from executing tests against a build.

In plain English

Zephyr, TestRail, Xray, qTest — where test cases and results live.

From real testing work

A tester files: "Checkout returns HTTP 500 when the cart contains a gift card and a subscription." Steps, expected, actual, environment, build number. Test Management Tool is one of the fields or states in that workflow, and getting the term right is what makes the report actionable for the developer picking it up at 9am.

In plain English

A red flag in test code — like duplicate setup or unclear assertions.

From real testing work

Where you meet this in real work: during a sprint, test smell is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Meszaros’ xUnit Test Patterns catalogs the canonical test smells.

In plain English

Any software that helps a tester — from JIRA to Selenium to JMeter.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Test Tool is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

A commercial UI automation tool from SmartBear covering web, desktop, and mobile.

From real testing work

The Android app works perfectly on the team's Pixel and crashes on a three-year-old budget device on a 3G connection. TestComplete is what keeps that from reaching production: a device matrix chosen from real analytics, not from what is on people's desks.

In plain English

A library that spins up real databases/queues in Docker for your integration tests.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Testcontainers shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

Mostly integration tests, few unit and E2E — the microservices distribution.

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so testing honeycomb is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

Push code to real users behind flags, watch metrics and roll back fast if something breaks.

From real testing work

Two weeks before a release the test manager compares planned versus executed cases, open defects by severity, and remaining effort. Testing in Production is where that number comes from — and the exam cares that it is used to inform a decision, not just published.

In plain English

A library that pushes you to test the app the way a user would use it.

From real testing work

Where you meet this in real work: during a sprint, testing library is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Prefer role-based queries (getByRole) over data-testids for maintainable UI tests.

In plain English

A Java test framework with more advanced features than classic JUnit.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. TestNG is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

A standalone commercial test case management platform.

From real testing work

Where you meet this in real work: during a sprint, testrail is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. TestRail is a favorite when teams want a test tool independent of Jira.

In plain English

How you get people actually using the tool: pilot, champions, training, integration.

From real testing work

The same discount rule is verified three times: in a unit test on the calculator, in an API test against the pricing service, and once through the UI at checkout. Tool Adoption names one of those layers — different test basis, different owner, different defects found.

// component level
expect(calcDiscount(100, "SAVE10")).toBe(90);
// system level
const res = await api.post("/cart/apply", { code: "SAVE10" });
expect(res.body.total).toBe(90);

In plain English

Structured evaluation of test tools against needs, budget, integration, learning curve and vendor viability.

From real testing work

Two weeks before a release the test manager compares planned versus executed cases, open defects by severity, and remaining effort. Tool Selection is where that number comes from — and the exam cares that it is used to inform a decision, not just published.

In plain English

A debugger that lets you scrub through a failed test like a movie.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Trace Viewer is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

A commercial model-based enterprise test automation tool.

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so tricentis tosca is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

A checklist of 12 principles for building cloud-native SaaS apps.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Twelve-Factor App is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

The classic enterprise GUI automation tool — formerly QTP.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. UFT is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

JUnit, pytest, NUnit — lets developers run and assert on unit tests.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a unit test framework tool that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

A Vite-native, Jest-compatible test runner — faster in modern JS setups.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Vitest is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

The standard protocol tools use to drive real browsers.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a webdriver that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

Testing outbound event notifications your system sends to third parties.

From real testing work

Where you meet this in real work: during a sprint, webhook testing is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Test signature verification, retry behavior, and idempotency of consumers.

In plain English

A tool to fake HTTP APIs so tests don’t hit real backends.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a wiremock that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

Apple’s official iOS UI testing framework, part of Xcode.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. XCUITest is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

A syntax for finding elements by traversing the page structure.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. XPath is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

A Jira add-on that turns Jira into a full test management tool.

From real testing work

A story reads "As a returning customer I can reuse a saved card." Before estimating it, the three amigos add acceptance criteria for an expired card and a card removed from the account. Xray for Jira is the artefact or link involved, and it is what lets you prove later that every requirement has at least one test.

In plain English

A modern .NET test framework used by many ASP.NET Core projects.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. xUnit.net is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Another Jira-integrated test management tool.

From real testing work

Two weeks before a release the test manager compares planned versus executed cases, open defects by severity, and remaining effort. Zephyr is where that number comes from — and the exam cares that it is used to inform a decision, not just published.