ISTQB Glossary — Advanced and Specialist Testing

In plain English

Common abbreviation for accessibility (a + 11 letters + y).

From real testing work

A signup form passes the automated scanner with zero violations, then fails the moment a tester unplugs the mouse: focus jumps from the email field straight to the footer. Accessibility work catches exactly that gap between tool output and a real person completing the task.

const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
// then: keyboard-only pass, Tab order recorded in the charter notes

In plain English

The percentage of predictions the model got right overall.

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so accuracy is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

The four properties that make database transactions safe.

From real testing work

After a bad release the team runs a root-cause session and changes the definition of done rather than adding more tests at the end. ACID sits on that side of the fence: preventive and process-oriented, distinct from executing tests against a build.

In plain English

How easily the app adapts to a new environment without code changes.

From real testing work

Staging has last quarter's anonymised production snapshot, seeded with fifteen deliberately broken accounts. Adaptability covers how that is created, versioned and restored — because a suite that passes only on Tuesday's data is not a suite you can trust.

make db-reset && make seed-fixtures ENV=staging
# fixtures are versioned with the tests, in the same commit

In plain English

Testing apps built on AI models — including whether the model itself gives good answers.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. AI Testing shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

Using AI/ML in the test process itself — auto-generating tests, self-healing selectors, predicting risky areas.

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so ai-based testing is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

Amazon’s managed NoSQL database.

From real testing work

Where you meet this in real work: during a sprint, amazon dynamodb is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Test partition-key design, throttling, and eventually-consistent reads.

In plain English

Amazon’s object storage service.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Amazon S3 is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

Amazon’s managed message queue service.

From real testing work

Where you meet this in real work: during a sprint, amazon sqs is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. SQS standard is at-least-once with best-effort ordering; FIFO queues guarantee order.

In plain English

How easy it is to figure out why something broke or what a change will affect.

From real testing work

A tester files: "Checkout returns HTTP 500 when the cart contains a gift card and a subscription." Steps, expected, actual, environment, build number. Analysability is one of the fields or states in that workflow, and getting the term right is what makes the report actionable for the developer picking it up at 9am.

In plain English

A long secret string used to identify the caller of an API.

From real testing work

Where you meet this in real work: during a sprint, api key is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. API keys prove identity, not user identity; never commit them to source control.

In plain English

Cisco’s enterprise APM platform.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. AppDynamics is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

Extra attributes that tell assistive tech what a widget is and does.

From real testing work

A signup form passes the automated scanner with zero violations, then fails the moment a tester unplugs the mouse: focus jumps from the email field straight to the footer. ARIA work catches exactly that gap between tool output and a real person completing the task.

const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
// then: keyboard-only pass, Tab order recorded in the charter notes

In plain English

Verifying login, password rules, MFA, session, and lockout behaviour.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under authentication testing: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

Verifying role- and permission-based access controls.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under authorization testing: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

Run code in AWS without managing servers.

From real testing work

Where you meet this in real work: during a sprint, aws lambda is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Cold-start latency, timeouts, and IAM permissions are the top Lambda test concerns.

In plain English

Azure’s serverless functions — the equivalent of AWS Lambda.

From real testing work

Where you meet this in real work: during a sprint, azure functions is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Cold starts, bindings, and Durable Functions are common test focuses.

In plain English

The first clean run whose numbers become the yardstick for future runs.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Baseline Performance Test is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

Checking whether an app kills your phone battery.

From real testing work

Where you meet this in real work: during a sprint, battery testing is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Excessive wake locks, GPS use, and background sync are the top battery-drain culprits.

In plain English

A repeatable test used to compare versions, releases, or products head-to-head.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Benchmark Test is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

Checking whether an AI treats different groups of people fairly.

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so bias testing is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

Testing pipelines that process huge datasets across many machines.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Big Data Testing shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

A safe, no-blame incident review that focuses on process and system fixes.

From real testing work

A requirements review on a "forgot password" story raises that the acceptance criteria never say what happens to an unverified email address. That is blameless postmortem in practice — an issue found in a document, before a single line of code exists, at a fraction of the cost of finding it in UAT.

In plain English

The single slowest step that caps overall throughput.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Bottleneck is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

Walling off resources so one failure can’t sink the whole system.

From real testing work

A tester files: "Checkout returns HTTP 500 when the cart contains a gift card and a subscription." Steps, expected, actual, environment, build number. Bulkhead Pattern is one of the fields or states in that workflow, and getting the term right is what makes the report actionable for the developer picking it up at 9am.

In plain English

Making sure the business keeps running even when tech, people, or facilities are hit.

From real testing work

A tester files: "Checkout returns HTTP 500 when the cart contains a gift card and a subscription." Steps, expected, actual, environment, build number. Business Continuity is one of the fields or states in that workflow, and getting the term right is what makes the report actionable for the developer picking it up at 9am.

In plain English

A network of servers around the world that speeds up static content delivery.

From real testing work

Where you meet this in real work: during a sprint, cdn is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. CDN test focus: cache-hit ratio, invalidation, and origin failover.

In plain English

A pattern that stops hammering a broken service so it can recover.

From real testing work

Where you meet this in real work: during a sprint, circuit breaker is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Resilience4j and Hystrix are canonical implementations — test open, half-open, and closed states.

In plain English

Testing apps that run in the cloud — AWS, Azure, GCP — including scaling and cost behavior.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Cloud Testing is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

Simulate a first-time user step by step and note where they'd get stuck.

From real testing work

A requirements review on a "forgot password" story raises that the acceptance criteria never say what happens to an unverified email address. That is cognitive walkthrough in practice — an issue found in a document, before a single line of code exists, at a fraction of the cost of finding it in UAT.

In plain English

Checking that text stands out enough from its background to be readable.

From real testing work

A signup form passes the automated scanner with zero violations, then fails the moment a tester unplugs the mouse: focus jumps from the email field straight to the footer. Color Contrast Testing work catches exactly that gap between tool output and a real person completing the task.

const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
// then: keyboard-only pass, Tab order recorded in the charter notes

In plain English

Does it play well with other systems and share resources cleanly?

From real testing work

Staging has last quarter's anonymised production snapshot, seeded with fifteen deliberately broken accounts. Compatibility covers how that is created, versioned and restored — because a suite that passes only on Tuesday's data is not a suite you can trust.

make db-reset && make seed-fixtures ENV=staging
# fixtures are versioned with the tests, in the same commit

In plain English

Test that the system behaves correctly when many things happen at the exact same time.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Concurrency Testing is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

Only the right people can read the data.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under confidentiality: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

A 2×2 table showing what the model got right and wrong for each class.

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so confusion matrix is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

Consumers publish what they need; providers verify they still meet it.

From real testing work

Where you meet this in real work: during a sprint, consumer-driven contract testing is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Pact is the reference tool for consumer-driven contracts.

In plain English

A browser policy that blocks scripts and resources from unapproved sources.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under content security policy: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

Test that each API consumer and provider still honor the shared schema — great for microservices.

From real testing work

Where you meet this in real work: during a sprint, contract testing is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Pact and Spring Cloud Contract are the industry-standard contract testing tools.

In plain English

Producer and consumer each publish and verify a shared contract so integrations don't silently break.

From real testing work

The same discount rule is verified three times: in a unit test on the calculator, in an API test against the pricing service, and once through the UI at checkout. Contract Testing names one of those layers — different test basis, different owner, different defects found.

// component level
expect(calcDiscount(100, "SAVE10")).toBe(90);
// system level
const res = await api.post("/cart/apply", { code: "SAVE10" });
expect(res.body.total).toBe(90);

In plain English

Google’s three metrics for real user page experience: LCP, INP, CLS.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Core Web Vitals is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

A shared ID that lets you follow one request across many services.

From real testing work

Where you meet this in real work: during a sprint, correlation id is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Correlation IDs are the poor-man’s distributed tracing — essential for microservices debugging.

In plain English

Splitting the write side and read side of your data model into separate services.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. CQRS is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

Run the app across Chrome, Firefox, Safari, Edge and old versions to confirm consistent behavior.

From real testing work

Staging has last quarter's anonymised production snapshot, seeded with fifteen deliberately broken accounts. Cross-Browser Testing covers how that is created, versioned and restored — because a suite that passes only on Tuesday's data is not a suite you can trust.

make db-reset && make seed-fixtures ENV=staging
# fixtures are versioned with the tests, in the same commit

In plain English

Test the same app across Windows, macOS, Linux, iOS, Android — behavior should match.

From real testing work

The Android app works perfectly on the team's Pixel and crashes on a three-year-old budget device on a 3G connection. Cross-Platform Testing is what keeps that from reaching production: a device matrix chosen from real analytics, not from what is on people's desks.

In plain English

Malicious page makes your browser send a real, authenticated request to another site.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under cross-site request forgery: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

Attacker gets their JavaScript to run in another user's browser session.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under cross-site scripting: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

A short customer satisfaction rating.

From real testing work

Two weeks before a release the test manager compares planned versus executed cases, open defects by severity, and remaining effort. CSAT is where that number comes from — and the exam cares that it is used to inform a decision, not just published.

In plain English

How much the page jumps around while loading.

From real testing work

Where you meet this in real work: during a sprint, cumulative layout shift is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Target: CLS under 0.1. Caused by images without dimensions, injected ads, late-loading fonts.

In plain English

Making sure nothing is lost or corrupted when moving data to a new system.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Data Migration Testing shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

How trustworthy your data is — right values, no gaps, no duplicates, up to date.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Data Quality shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

Copying just a slice of production data — with all relationships intact — for testing.

From real testing work

Staging has last quarter's anonymised production snapshot, seeded with fifteen deliberately broken accounts. Data Subsetting covers how that is created, versioned and restored — because a suite that passes only on Tuesday's data is not a suite you can trust.

make db-reset && make seed-fixtures ENV=staging
# fixtures are versioned with the tests, in the same commit

In plain English

Testing that data moving through pipelines and dashboards is accurate, complete, and on time.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Data Testing shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

A lookup structure that makes SELECTs fast.

From real testing work

Where you meet this in real work: during a sprint, database index is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Missing indexes are the #1 cause of slow queries in production QA investigations.

In plain English

Keeping copies of the database on multiple servers.

From real testing work

Where you meet this in real work: during a sprint, database replication is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Primary-replica lag is a common test target — reads may return stale data.

In plain English

Testing the database directly — schemas, queries, procedures — not just through the UI.

From real testing work

Where you meet this in real work: during a sprint, database testing is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. DB testing catches issues that unit and UI tests miss, especially around constraints, indexes, and concurrency.

In plain English

A cloud platform for big data engineering and ML on top of Spark.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Databricks shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

A commercial all-in-one monitoring and observability platform.

From real testing work

Two weeks before a release the test manager compares planned versus executed cases, open defects by severity, and remaining effort. Datadog is where that number comes from — and the exam cares that it is used to inform a decision, not just published.

In plain English

Two processes each hold a lock the other needs — both freeze forever.

From real testing work

Where you meet this in real work: during a sprint, deadlock is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Prevention: consistent lock ordering, timeouts, and lock-free structures.

In plain English

Scan your libraries for known CVEs, usually as part of CI.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under dependency scanning: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

Making sure upgrades to libraries don't break your app.

From real testing work

Where you meet this in real work: during a sprint, dependency testing is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Combine with dependency scanning to catch both security and behaviour issues.

In plain English

A cloud service that lets you run tests on hundreds of real phones at once.

From real testing work

A signup form passes the automated scanner with zero violations, then fails the moment a tester unplugs the mouse: focus jumps from the email field straight to the footer. Device Farm work catches exactly that gap between tool output and a real person completing the task.

const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
// then: keyboard-only pass, Tab order recorded in the charter notes

In plain English

The plan for bringing systems back after major outages, region failures, or data loss.

From real testing work

A tester files: "Checkout returns HTTP 500 when the cart contains a gift card and a subscription." Steps, expected, actual, environment, build number. Disaster Recovery is one of the fields or states in that workflow, and getting the term right is what makes the report actionable for the developer picking it up at 9am.

In plain English

A tool that shows how a single request travels across all your microservices.

From real testing work

Where you meet this in real work: during a sprint, distributed tracing is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. OpenTelemetry is the standard; Jaeger and Zipkin are common backends.

In plain English

The avionics software safety standard.

From real testing work

After a sprint of bug fixes the team runs the unit suite with coverage on and finds the discount calculator sits at 62%. DO-178C tells them which lines or branches never executed — usually the error paths nobody wrote a test for.

npx vitest run --coverage
# ---------------------|---------|----------|
# File                 | % Stmts | % Branch |
# discount.ts          |   62.5  |   41.6   |

In plain English

Deployment frequency, lead time, change failure rate, mean time to restore — the four DevOps performance KPIs.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. DORA Metrics is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

A commercial observability platform known for its AI (Davis) root-cause engine.

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so dynatrace is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

Running code closer to the user for lower latency.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Edge Computing is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

Test whether the system delivers its output using a reasonable amount of resources (CPU, memory, network).

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Efficiency Testing is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

A distributed search engine powering many app search bars and log platforms.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Elasticsearch shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

A popular log collection, storage, and search stack.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. ELK Stack shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

Testing software that runs on tiny chips inside cars, appliances, and medical devices.

From real testing work

The Android app works perfectly on the team's Pixel and crashes on a three-year-old budget device on a 3G connection. Embedded Testing is what keeps that from reaching production: a device matrix chosen from real analytics, not from what is on people's desks.

In plain English

Testing the vector representations AI systems use to find similar items.

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so embeddings testing is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

How much unreliability you're allowed before you must slow down releases.

From real testing work

A tester files: "Checkout returns HTTP 500 when the cart contains a gift card and a subscription." Steps, expected, actual, environment, build number. Error Budget is one of the fields or states in that workflow, and getting the term right is what makes the report actionable for the developer picking it up at 9am.

In plain English

The rules for what happens when a service uses up its error budget — e.g. release freeze.

From real testing work

A tester files: "Checkout returns HTTP 500 when the cart contains a gift card and a subscription." Steps, expected, actual, environment, build number. Error Budget Policy is one of the fields or states in that workflow, and getting the term right is what makes the report actionable for the developer picking it up at 9am.

In plain English

Testing the pipes that move and reshape data from one system to another.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. ETL Testing shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

Storing every change as an event instead of overwriting rows.

From real testing work

Where you meet this in real work: during a sprint, event sourcing is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Event sourcing gives auditability and time-travel but complicates schema evolution.

In plain English

A single score that balances precision and recall.

From real testing work

Where you meet this in real work: during a sprint, f1 score is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. F1 is preferred over accuracy for imbalanced datasets.

In plain English

Crash immediately on bad input instead of limping along.

From real testing work

A tester files: "Checkout returns HTTP 500 when the cart contains a gift card and a subscription." Steps, expected, actual, environment, build number. Fail Fast is one of the fields or states in that workflow, and getting the term right is what makes the report actionable for the developer picking it up at 9am.

In plain English

If something fails, default to a safe outcome.

From real testing work

A tester files: "Checkout returns HTTP 500 when the cart contains a gift card and a subscription." Steps, expected, actual, environment, build number. Fail Safe is one of the fields or states in that workflow, and getting the term right is what makes the report actionable for the developer picking it up at 9am.

In plain English

Keeps working even when parts fail.

From real testing work

A tester files: "Checkout returns HTTP 500 when the cart contains a gift card and a subscription." Steps, expected, actual, environment, build number. Fault Tolerance is one of the fields or states in that workflow, and getting the term right is what makes the report actionable for the developer picking it up at 9am.

In plain English

Usability testing done while the UI is being designed — feedback goes straight into the next iteration.

From real testing work

A signup form passes the automated scanner with zero violations, then fails the moment a tester unplugs the mouse: focus jumps from the email field straight to the footer. Formative Usability Evaluation work catches exactly that gap between tool output and a real person completing the task.

const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
// then: keyboard-only pass, Tab order recorded in the charter notes

In plain English

Does the software cover every promised feature and user goal?

From real testing work

Where you meet this in real work: during a sprint, functional completeness is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Missing features = incomplete, even if what's there works perfectly.

In plain English

Does the function produce the right output for the given input?

From real testing work

Where you meet this in real work: during a sprint, functional correctness is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Sub-characteristic of Functional Suitability in ISO 25010.

In plain English

Does the software actually do what it should? Correctness, completeness, appropriateness.

From real testing work

After a bad release the team runs a root-cause session and changes the definition of done rather than adding more tests at the end. Functional Suitability sits on that side of the fence: preventive and process-oriented, distinct from executing tests against a build.

In plain English

Tweaking Java garbage collector settings to reduce pauses.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. GC Tuning is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

The EU privacy law that governs how personal data is collected and processed.

From real testing work

Where you meet this in real work: during a sprint, gdpr is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Testers verify consent flows, data export, and right-to-be-forgotten deletion.

In plain English

Google Cloud’s serverless container platform.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Google Cloud Run shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

Google Cloud’s managed messaging service.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Google Pub/Sub shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

The system keeps working, just with fewer features, when something breaks.

From real testing work

Where you meet this in real work: during a sprint, graceful degradation is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Test graceful degradation by disabling non-critical dependencies and verifying core flows still work.

In plain English

The standard dashboard tool for metrics and logs.

From real testing work

Two weeks before a release the test manager compares planned versus executed cases, open defects by severity, and remaining effort. Grafana is where that number comes from — and the exam cares that it is used to inform a decision, not just published.

In plain English

A commercial tool for running safe chaos experiments in production.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under gremlin: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

Testing the safety filters that stop the AI from saying harmful or off-topic things.

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so guardrail testing is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

When the AI makes stuff up but sounds sure about it.

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so hallucination is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

Plugging real hardware into a simulated world to test the software driving it.

From real testing work

Staging has last quarter's anonymised production snapshot, seeded with fifteen deliberately broken accounts. Hardware-in-the-LoopTesting covers how that is created, versioned and restored — because a suite that passes only on Tuesday's data is not a suite you can trust.

make db-reset && make seed-fixtures ENV=staging
# fixtures are versioned with the tests, in the same commit

In plain English

A snapshot of JVM memory used to hunt memory leaks.

From real testing work

Where you meet this in real work: during a sprint, heap dump is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Tools: jmap + Eclipse MAT, VisualVM — heap dump analysis is a senior perf-tester skill.

In plain English

Experts walk the UI against a checklist of usability rules — usually Nielsen's 10.

From real testing work

A requirements review on a "forgot password" story raises that the acceptance criteria never say what happens to an unverified email address. That is heuristic evaluation in practice — an issue found in a document, before a single line of code exists, at a fraction of the cost of finding it in UAT.

In plain English

The US health-data privacy and security law.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under hipaa: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

A middle ground between SAST and DAST that watches the app from inside as tests run.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under iast: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

The lifecycle standard for medical device software.

From real testing work

The Android app works perfectly on the team's Pixel and crashes on a three-year-old budget device on a 3G connection. IEC 62304 is what keeps that from reaching production: a device matrix chosen from real analytics, not from what is on people's desks.

In plain English

Define servers, networks and configs in code (Terraform, CloudFormation) so envs are versioned and reproducible.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Infrastructure as Code shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

How smoothly the software installs, upgrades, and uninstalls.

From real testing work

Staging has last quarter's anonymised production snapshot, seeded with fifteen deliberately broken accounts. Installability covers how that is created, versioned and restored — because a suite that passes only on Tuesday's data is not a suite you can trust.

make db-reset && make seed-fixtures ENV=staging
# fixtures are versioned with the tests, in the same commit

In plain English

Data can't be tampered with by unauthorized parties.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under integrity: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

How quickly the page responds to user clicks and taps — replaces FID.

From real testing work

Where you meet this in real work: during a sprint, interaction to next paint is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. INP replaced FID in March 2024 as a Core Web Vital.

In plain English

Test that the codebase is ready to be translated: date/number formats, RTL text, character encoding.

From real testing work

Where you meet this in real work: during a sprint, internationalization testing is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. i18n happens once in code; l10n happens per locale. i18n test failures block all localization.

In plain English

Testing smart devices — thermostats, wearables, sensors — and how they talk to each other and the cloud.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under iot testing: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

The functional safety standard for cars.

From real testing work

After a bad release the team runs a root-cause session and changes the definition of done rather than adding more tests at the end. ISO 26262 sits on that side of the fence: preventive and process-oriented, distinct from executing tests against a build.

In plain English

The reference model with 8 product quality characteristics and 5 quality-in-use characteristics.

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so iso/iec 25010 is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

The main international standard for information security programs.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under iso/iec 27001: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

A signed JSON blob used to prove who you are between services.

From real testing work

Where you meet this in real work: during a sprint, json web token is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Never store JWTs in localStorage for XSS-exposed apps; know the header/payload/signature structure.

In plain English

Unplug the mouse and try to reach every control — tab, shift+tab, enter, space.

From real testing work

Two weeks before a release the test manager compares planned versus executed cases, open defects by severity, and remaining effort. Keyboard Navigation Testing is where that number comes from — and the exam cares that it is used to inform a decision, not just published.

In plain English

How fast the biggest thing on the page finishes loading.

From real testing work

Where you meet this in real work: during a sprint, largest contentful paint is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Target: under 2.5 seconds for a good user experience.

In plain English

The wait time before data starts arriving — network-focused delay.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Latency is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

Using a language model to grade the answers of another language model.

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so llm as judge is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

Testing ChatGPT-style apps — accuracy, safety, cost, and speed.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. LLM Testing is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

Increase load in a controlled way to find how many users/transactions the system supports.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Load Testing is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

Test each localized version: strings translated, dates/currency correct, layout unbroken.

From real testing work

Staging has last quarter's anonymised production snapshot, seeded with fifteen deliberately broken accounts. Localization Testing covers how that is created, versioned and restored — because a suite that passes only on Tuesday's data is not a suite you can trust.

make db-reset && make seed-fixtures ENV=staging
# fixtures are versioned with the tests, in the same commit

In plain English

Test ML systems on data, model behavior, fairness and drift — not just standard functional tests.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Machine Learning Testing shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

How high up the first correct answer tends to appear in a ranked list.

From real testing work

Two weeks before a release the test manager compares planned versus executed cases, open defects by severity, and remaining effort. Mean Reciprocal Rank is where that number comes from — and the exam cares that it is used to inform a decision, not just published.

In plain English

How long, on average, a bug lives in the system before someone notices it.

From real testing work

A tester files: "Checkout returns HTTP 500 when the cart contains a gift card and a subscription." Steps, expected, actual, environment, build number. Mean Time to Detect is one of the fields or states in that workflow, and getting the term right is what makes the report actionable for the developer picking it up at 9am.

In plain English

Instead of asserting an exact answer, assert that changing the input in a known way changes the output in a known way.

From real testing work

Where you meet this in real work: during a sprint, metamorphic testing is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Useful for ML models and complex calculations where exact expected outputs are unknown.

In plain English

An app built from many small services, each with its own database and deployment pipeline.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Microservices shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

Testing a system where dozens of small services talk over HTTP or messaging.

From real testing work

Where you meet this in real work: during a sprint, microservices testing is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Rely on contract tests + service virtualization; end-to-end tests do not scale.

In plain English

Testing how the app behaves on slow, spotty, or dropped connections.

From real testing work

The Android app works perfectly on the team's Pixel and crashes on a three-year-old budget device on a 3G connection. Mobile Network Testing is what keeps that from reaching production: a device matrix chosen from real analytics, not from what is on people's desks.

In plain English

Testing apps on real phones: functionality + performance + battery + network + interruptions.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Mobile Testing is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

When an AI model gets worse over time because the real world changed.

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so model drift is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

How safely you can change the code without breaking other things.

From real testing work

A tester files: "Checkout returns HTTP 500 when the cart contains a gift card and a subscription." Steps, expected, actual, environment, build number. Modifiability is one of the fields or states in that workflow, and getting the term right is what makes the report actionable for the developer picking it up at 9am.

In plain English

Well-separated pieces that don't cause ripple effects when changed.

From real testing work

In sprint planning, payment processing scores high likelihood and high impact while the marketing footer scores low on both. Modularity is how the team justifies spending 60% of the test effort on 10% of the codebase — to a stakeholder, in one sentence.

In plain English

A popular document-oriented NoSQL database.

From real testing work

Where you meet this in real work: during a sprint, mongodb is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. MongoDB tests focus on schema flexibility, indexing, and eventual consistency in replica sets.

In plain English

Logging in with more than just a password.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under multi-factor authentication: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

TLS where both sides prove who they are with certificates.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under mutual tls: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

A widely used open-source relational database, popular with LAMP-stack apps.

From real testing work

Where you meet this in real work: during a sprint, mysql is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. MySQL and Postgres are the two dominant open-source RDBMSes.

In plain English

A metric for how well a ranked list puts the best results at the top.

From real testing work

Two weeks before a release the test manager compares planned versus executed cases, open defects by severity, and remaining effort. NDCG is where that number comes from — and the exam cares that it is used to inform a decision, not just published.

In plain English

A single-question customer loyalty score.

From real testing work

Two weeks before a release the test manager compares planned versus executed cases, open defects by severity, and remaining effort. Net Promoter Score is where that number comes from — and the exam cares that it is used to inform a decision, not just published.

In plain English

A commercial APM and observability platform.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. New Relic is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

The classic checklist for judging UI usability.

From real testing work

A signup form passes the automated scanner with zero violations, then fails the moment a tester unplugs the mouse: focus jumps from the email field straight to the footer. Nielsen's 10 Usability Heuristics work catches exactly that gap between tool output and a real person completing the task.

const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
// then: keyboard-only pass, Tab order recorded in the charter notes

In plain English

You can prove who did what, so they can't deny it later.

From real testing work

Where you meet this in real work: during a sprint, non-repudiation is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Audit logs and digital signatures support non-repudiation.

In plain English

The standard "Sign in with Google" style delegation protocol.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under oauth 2.0: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

OAuth plus a standardized way to know who the user is.

From real testing work

After a bad release the team runs a root-cause session and changes the definition of done rather than adding more tests at the end. OpenID Connect sits on that side of the fence: preventive and process-oriented, distinct from executing tests against a build.

In plain English

The standard SDK for producing observability data in any language.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. OpenTelemetry is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

The industry-standard hit list of the most dangerous web vulnerabilities.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under owasp top 10: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

A cryptographic replacement for passwords, synced across your devices.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under passkey: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

The security standard for anyone handling credit card data.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under pci dss: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

The ethical hacker who tries to break in on purpose.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under penetration tester: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

Simulate a real attack on the system to find exploitable holes.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under penetration testing: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

p95 = 95% of requests are faster than this number.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Percentile Latency is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

How well the software performs given the CPU, memory, and bandwidth it consumes.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Performance Efficiency is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

Data that identifies a specific person — must be protected.

From real testing work

Two weeks before a release the test manager compares planned versus executed cases, open defects by severity, and remaining effort. Personally Identifiable Information is where that number comes from — and the exam cares that it is used to inform a decision, not just published.

In plain English

How easily the software moves to new OSes, browsers, devices, or clouds.

From real testing work

The Android app works perfectly on the team's Pixel and crashes on a three-year-old budget device on a 3G connection. Portability is what keeps that from reaching production: a device matrix chosen from real analytics, not from what is on people's desks.

In plain English

A leading open-source relational database.

From real testing work

After a bad release the team runs a root-cause session and changes the definition of done rather than adding more tests at the end. PostgreSQL sits on that side of the fence: preventive and process-oriented, distinct from executing tests against a build.

In plain English

Of everything the model called positive, how many actually are?

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so precision is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

Verify the app follows GDPR, CCPA, HIPAA etc. — consent, minimization, retention, deletion.

From real testing work

Where you meet this in real work: during a sprint, privacy testing is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Privacy testing overlaps security testing but focuses on data-handling rules, not attacks.

In plain English

The standard open-source metrics database in cloud-native stacks.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Prometheus shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

Testing whether your prompts reliably produce the answers you want.

From real testing work

A signup form passes the automated scanner with zero violations, then fails the moment a tester unplugs the mouse: focus jumps from the email field straight to the footer. Prompt Testing work catches exactly that gap between tool output and a real person completing the task.

const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
// then: keyboard-only pass, Tab order recorded in the charter notes

In plain English

Instead of writing example tests, describe rules that should always hold and let the tool try 1000s of inputs.

From real testing work

A requirements review on a "forgot password" story raises that the acceptance criteria never say what happens to an unverified email address. That is property-based testing in practice — an issue found in a document, before a single line of code exists, at a fraction of the cost of finding it in UAT.

In plain English

A structured checklist of what 'quality' means, so requirements and tests can target it.

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so quality model is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

Two threads or requests interfere because their timing wasn't controlled.

From real testing work

A tester files: "Checkout returns HTTP 500 when the cart contains a gift card and a subscription." Steps, expected, actual, environment, build number. Race Condition is one of the fields or states in that workflow, and getting the term right is what makes the report actionable for the developer picking it up at 9am.

In plain English

Testing AI systems that look up documents and then answer questions.

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so rag testing is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

Bring users on slowly instead of hitting the system with full load instantly.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Ramp-Up is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

Capping how many requests a client can make per minute.

From real testing work

Two weeks before a release the test manager compares planned versus executed cases, open defects by severity, and remaining effort. Rate Limiting is where that number comes from — and the exam cares that it is used to inform a decision, not just published.

In plain English

Running tests on actual phones instead of emulators to catch real-world issues.

From real testing work

The Android app works perfectly on the team's Pixel and crashes on a three-year-old budget device on a 3G connection. Real Device Testing is what keeps that from reaching production: a device matrix chosen from real analytics, not from what is on people's desks.

In plain English

Making sure a system responds within a guaranteed time — every time, no exceptions.

From real testing work

Two weeks before a release the test manager compares planned versus executed cases, open defects by severity, and remaining effort. Real-Time Testing is where that number comes from — and the exam cares that it is used to inform a decision, not just published.

In plain English

Measure what real users actually experience, in the wild.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Real-User Monitoring is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

Of everything that is actually positive, how many did the model catch?

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so recall is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

Can the system bounce back cleanly after a crash without losing data?

From real testing work

A tester files: "Checkout returns HTTP 500 when the cart contains a gift card and a subscription." Steps, expected, actual, environment, build number. Recoverability is one of the fields or states in that workflow, and getting the term right is what makes the report actionable for the developer picking it up at 9am.

In plain English

Deliberately crash it and time how cleanly it comes back.

From real testing work

Where you meet this in real work: during a sprint, recovery testing is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Overlaps with disaster recovery and business-continuity testing.

In plain English

An extremely fast in-memory key-value store, used for caching and sessions.

From real testing work

Where you meet this in real work: during a sprint, redis is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Redis is the default cache in most stacks — test eviction, expiration, and persistence modes.

In plain English

How consistently the system stays up and correct over time.

From real testing work

After a bad release the team runs a root-cause session and changes the definition of done rather than adding more tests at the end. Reliability sits on that side of the fence: preventive and process-oriented, distinct from executing tests against a build.

In plain English

How easily this component can swap in for another one doing the same job.

From real testing work

Staging has last quarter's anonymised production snapshot, seeded with fifteen deliberately broken accounts. Replaceability covers how that is created, versioned and restored — because a suite that passes only on Tuesday's data is not a suite you can trust.

make db-reset && make seed-fixtures ENV=staging
# fixtures are versioned with the tests, in the same commit

In plain English

How much CPU, memory, disk and bandwidth the software chews through under load.

From real testing work

Where you meet this in real work: during a sprint, resource utilization is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Track resource utilization side-by-side with response time — bottlenecks show up as saturated resources.

In plain English

Time from click to visible result — the classic user-perceived performance metric.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Response Time is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

Automatically re-attempting failed calls, usually with a growing delay.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Retry Pattern is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

How easily a component can be used again elsewhere.

From real testing work

A signup form passes the automated scanner with zero violations, then fails the moment a tester unplugs the mouse: focus jumps from the email field straight to the footer. Reusability work catches exactly that gap between tool output and a real person completing the task.

const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
// then: keyboard-only pass, Tab order recorded in the charter notes

In plain English

RTO = how long you can be down. RPO = how much data you can lose.

From real testing work

Where you meet this in real work: during a sprint, rto / rpo is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Both are business-defined — testing must confirm the system meets them.

In plain English

A way to keep data consistent across microservices without a global transaction.

From real testing work

A tester files: "Checkout returns HTTP 500 when the cart contains a gift card and a subscription." Steps, expected, actual, environment, build number. Saga Pattern is one of the fields or states in that workflow, and getting the term right is what makes the report actionable for the developer picking it up at 9am.

In plain English

An older XML-based SSO protocol, common in enterprise IdPs.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under saml: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

Actually navigating the app with NVDA, JAWS, or VoiceOver as a real user would.

From real testing work

A signup form passes the automated scanner with zero violations, then fails the moment a tester unplugs the mouse: focus jumps from the email field straight to the footer. Screen Reader Testing work catches exactly that gap between tool output and a real person completing the task.

const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
// then: keyboard-only pass, Tab order recorded in the charter notes

In plain English

How you safely store and rotate passwords, keys, and tokens used by code.

From real testing work

Where you meet this in real work: during a sprint, secrets management is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Vault, AWS Secrets Manager, and Doppler are common secret stores — test rotation and access controls.

In plain English

The US federal accessibility law — requires WCAG-level access for federal IT.

From real testing work

A signup form passes the automated scanner with zero violations, then fails the moment a tester unplugs the mouse: focus jumps from the email field straight to the footer. Section 508 work catches exactly that gap between tool output and a real person completing the task.

const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
// then: keyboard-only pass, Tab order recorded in the charter notes

In plain English

Confidentiality, integrity, non-repudiation, authenticity, accountability.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under security: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

Testing Lambda-style functions where there’s no server to log into.

From real testing work

Where you meet this in real work: during a sprint, serverless testing is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Cold-start latency, event payload contracts, and IAM permissions are the top serverless test targets.

In plain English

Contract that promises the customer a specific level of service.

From real testing work

Where you meet this in real work: during a sprint, service level agreement is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. SLA breaches usually trigger financial penalties.

In plain English

A specific metric that shows how well the service is doing.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Service Level Indicator is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

The internal target for reliability or performance — e.g. 99.9% availability.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Service Level Objective is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

Testing cookies, tokens, timeouts, and logout behaviour.

From real testing work

Where you meet this in real work: during a sprint, session management testing is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Look for session fixation, missing timeouts, and tokens exposed in URLs.

In plain English

Splitting a big database across many servers by some key.

From real testing work

On a checkout form that accepts a quantity between 1 and 99, sharding is what stops you writing 99 test cases. You pick one value from inside the valid range and the values sitting either side of each edge, then let the rest of the range go untested on purpose.

// quantity field: valid 1..99
test.each([0, 1, 2, 98, 99, 100])("quantity %i", (qty) => {
  const res = validateQuantity(qty);
  expect(res.valid).toBe(qty >= 1 && qty <= 99);
});

In plain English

Test earlier — in design and dev, not just after code is done.

From real testing work

Where you meet this in real work: during a sprint, shift-left testing is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Practices: TDD, static analysis, code review, early test design.

In plain English

An iPhone stand-in that runs on macOS to test iOS apps without hardware.

From real testing work

The Android app works perfectly on the team's Pixel and crashes on a three-year-old budget device on a 3G connection. Simulator is what keeps that from reaching production: a device matrix chosen from real analytics, not from what is on people's desks.

In plain English

Log in once, use many apps.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under single sign-on: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

Google-invented discipline that treats ops as a software problem.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Site Reliability Engineering is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

A modern cloud data warehouse for analytics.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. Snowflake shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

Run heavy but normal load for hours or days to reveal memory leaks and slow degradation.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Soak Testing is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

An audit report proving a SaaS company handles data safely.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under soc 2: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

User input ends up as executable SQL and lets attackers read or change the database.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under sql injection: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

Combining rows from two tables using a shared column.

From real testing work

Where you meet this in real work: during a sprint, sql join is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Know INNER, LEFT, RIGHT, FULL OUTER, and CROSS joins — asked in nearly every QA interview.

In plain English

Logging in JSON with named fields, not free-form text.

From real testing work

Where you meet this in real work: during a sprint, structured logging is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Structured logs enable querying, alerting, and analytics — unstructured logs don’t.

In plain English

Usability testing on the near-finished product to see if it hit the design goals.

From real testing work

A signup form passes the automated scanner with zero violations, then fails the moment a tester unplugs the mouse: focus jumps from the email field straight to the footer. Summative Usability Evaluation work catches exactly that gap between tool output and a real person completing the task.

const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
// then: keyboard-only pass, Tab order recorded in the charter notes

In plain English

Fake users running scripts against production 24/7 to detect issues.

From real testing work

The payment gateway sandbox is down for maintenance, but the checkout service still has to be tested. The team wires in a synthetic monitoring that returns canned success and decline responses, so the tests exercise their own code instead of a third party's uptime.

vi.mock("./gateway", () => ({
  charge: vi.fn(async (amount: number) =>
    amount > 5000 ? { status: "declined" } : { status: "approved" }),
}));

In plain English

A quick 10-question survey turned into one usability score.

From real testing work

A signup form passes the automated scanner with zero violations, then fails the moment a tester unplugs the mouse: focus jumps from the email field straight to the footer. System Usability Scale work catches exactly that gap between tool output and a real person completing the task.

const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
// then: keyboard-only pass, Tab order recorded in the charter notes

In plain English

Fowler's 2x2 for debt: reckless/prudent, deliberate/inadvertent — helps teams talk about which debt is OK.

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so technical debt quadrant is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

The discipline of keeping test environments healthy, isolated, versioned and available on demand.

From real testing work

Two weeks before a release the test manager compares planned versus executed cases, open defects by severity, and remaining effort. Test Environment Management is where that number comes from — and the exam cares that it is used to inform a decision, not just published.

In plain English

Rich telemetry from your test runs (timings, flake rate, root causes) so you can improve the suite over time.

From real testing work

A tester files: "Checkout returns HTTP 500 when the cart contains a gift card and a subscription." Steps, expected, actual, environment, build number. Test Observability is one of the fields or states in that workflow, and getting the term right is what makes the report actionable for the developer picking it up at 9am.

In plain English

The struggle of knowing whether the test passed when the correct answer is hard to compute.

From real testing work

Where you meet this in real work: during a sprint, test oracle problem is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Metamorphic testing and property-based testing are two approaches to the oracle problem.

In plain English

Kent C. Dodds' variant of the pyramid that puts integration testing in the middle spotlight.

From real testing work

A requirements review on a "forgot password" story raises that the acceptance criteria never say what happens to an unverified email address. That is test trophy in practice — an issue found in a document, before a single line of code exists, at a fraction of the cost of finding it in UAT.

In plain English

How easy it is to write good tests for the software.

From real testing work

Where you meet this in real work: during a sprint, testability is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Poor testability is a design smell — push feedback back to developers.

In plain English

The pause a real user takes between clicks — models human pacing.

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so think time is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

A snapshot of every thread in the JVM — used to find deadlocks and hangs.

From real testing work

Where you meet this in real work: during a sprint, thread dump is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Take three thread dumps a few seconds apart to see which threads are stuck.

In plain English

Sit down early and map what could go wrong and who might attack.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under threat modeling: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

How fast the system responds and how much it processes per second.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Time Behaviour is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

A hard limit on how long an operation is allowed to run.

From real testing work

Where you meet this in real work: during a sprint, timeout is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Every network call needs a timeout — missing timeouts are a top cause of outages.

In plain English

The encryption that protects HTTPS connections.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under tls: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

Boring, repetitive ops work that could and should be automated away.

From real testing work

The regression pack runs on every merge to main and blocks the deploy when it fails. Toil is the piece of that setup being named here — and the cost is not writing the tests, it is keeping them green when the UI changes next sprint.

- name: Regression suite
  run: npx playwright test --grep @regression
- name: Publish report
  if: always()
  uses: actions/upload-artifact@v4
  with: { name: playwright-report, path: playwright-report/ }

In plain English

Swapping real sensitive values for meaningless tokens that map back only in a vault.

From real testing work

Where you meet this in real work: during a sprint, tokenization is the piece of vocabulary a tester reaches for when explaining a decision to a developer or a stakeholder — for example, justifying in a stand-up why a specific check belongs in this build rather than the next. Tokenization is preferred over encryption for PCI scope reduction.

In plain English

The 6-digit rotating code from Google/Microsoft Authenticator apps.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under totp: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

A group of database operations that all succeed or all fail together.

From real testing work

The same discount rule is verified three times: in a unit test on the calculator, in an API test against the pricing service, and once through the UI at checkout. Transaction names one of those layers — different test basis, different owner, different defects found.

// component level
expect(calcDiscount(100, "SAVE10")).toBe(90);
// system level
const res = await api.post("/cart/apply", { code: "SAVE10" });
expect(res.body.total).toBe(90);

In plain English

How many transactions the system handles each second.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Transactions Per Second is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

How easy and pleasant the software is to use for its intended users.

From real testing work

A signup form passes the automated scanner with zero violations, then fails the moment a tester unplugs the mouse: focus jumps from the email field straight to the footer. Usability work catches exactly that gap between tool output and a real person completing the task.

const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
// then: keyboard-only pass, Tab order recorded in the charter notes

In plain English

Numeric measures of how easy or effective a product is to use.

From real testing work

A signup form passes the automated scanner with zero violations, then fails the moment a tester unplugs the mouse: focus jumps from the email field straight to the footer. Usability Metric work catches exactly that gap between tool output and a real person completing the task.

const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
// then: keyboard-only pass, Tab order recorded in the charter notes

In plain English

Broader than usability testing — includes emotion, satisfaction, and journey.

From real testing work

A signup form passes the automated scanner with zero violations, then fails the moment a tester unplugs the mouse: focus jumps from the email field straight to the footer. UX Testing work catches exactly that gap between tool output and a real person completing the task.

const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
// then: keyboard-only pass, Tab order recorded in the charter notes

In plain English

Compare screenshots against baselines to catch UI regressions that functional tests miss.

From real testing work

A résumé-screening model scores candidates from 0 to 1. There is no single expected value to assert against, so visual testing is checked statistically: accuracy on a held-out set, score distribution compared across demographic slices, and an alert when the live distribution drifts from training.

assert accuracy_score(y_true, y_pred) >= 0.87
gap = abs(tpr(group_a) - tpr(group_b))
assert gap < 0.05, f"fairness gap too wide: {gap:.3f}"

In plain English

A private, isolated network inside a public cloud.

From real testing work

A nightly ETL job loads 40 million rows into the reporting warehouse. VPC shows up as the reconciliation step: row counts, checksum comparisons per partition, and null-rate thresholds that fail the pipeline before an analyst builds a dashboard on bad data.

SELECT load_date,
       COUNT(*) AS rows_loaded,
       SUM(CASE WHEN customer_id IS NULL THEN 1 ELSE 0 END) AS null_keys
FROM warehouse.orders
WHERE load_date = CURRENT_DATE
GROUP BY load_date;

In plain English

A flaw that attackers can use to break in or cause damage.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under vulnerability: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

Scan hosts, containers, dependencies against known CVE databases and misconfigurations.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under vulnerability scanning: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

The W3C standard for accessibility — three levels A, AA, AAA.

From real testing work

A signup form passes the automated scanner with zero violations, then fails the moment a tester unplugs the mouse: focus jumps from the email field straight to the footer. WCAG work catches exactly that gap between tool output and a real person completing the task.

const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
// then: keyboard-only pass, Tab order recorded in the charter notes

In plain English

The three WCAG conformance tiers — most laws require AA.

From real testing work

A signup form passes the automated scanner with zero violations, then fails the moment a tester unplugs the mouse: focus jumps from the email field straight to the footer. WCAG A / AA / AAA work catches exactly that gap between tool output and a real person completing the task.

const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
// then: keyboard-only pass, Tab order recorded in the charter notes

In plain English

A firewall specifically for web app traffic that blocks known attack patterns.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under web application firewall: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

The browser standard behind passkeys and hardware-key logins.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under webauthn: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding

In plain English

A description of the mix of transactions, users, and think times used in a load test.

From real testing work

Before a Black Friday sale, the team models 5,000 concurrent shoppers against staging. Workload Model is the part of that exercise that answers a specific question — not "is it fast", but "at what point does it stop being fast, and what breaks first".

// k6 scenario
export const options = {
  stages: [
    { duration: "2m", target: 500 },
    { duration: "5m", target: 5000 },
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"] },
};

In plain English

A brand-new vulnerability with no patch available yet.

From real testing work

During a pre-release security pass, a tester submits `' OR 1=1 --` into a search box and watches the response. Work like that sits under zero-day: deliberately behaving like an attacker against your own system, in a controlled environment, with written permission.

curl -s "https://staging.example.com/api/search?q=%27%20OR%201%3D1%20--" \
  -H "Authorization: Bearer $TOKEN" | jq '.results | length'
# a jump from 12 to 4,812 results is the finding