Executive Definition
Data-driven testing, sometimes abbreviated DDT, is a technique in which a single test procedure is executed repeatedly against a table of input values and expected results. The test body is written once; the framework iterates over the data set and reports one pass or fail per row. The goal is to increase coverage of input variation without duplicating scenario code.
The pattern shows up under different names in every major framework: @ParameterizedTest in JUnit 5, @pytest.mark.parametrize in pytest, test.each in Jest and Vitest, [TestCase] in NUnit, and Examples: tables in Cucumber. All of them share the same shape — one function, many rows, one report line per row — because the underlying problem is universal: input variation is where bugs live, and copy-pasting a test to cover ten inputs is worse than useless.
Data can live in several places. Inline arrays are best for small, curated cases where the values are self-documenting. External files (CSV, JSON, YAML) suit large data sets or cross-team ownership, where a business analyst edits the rows and the engineer maintains the logic. Property-based generators (Hypothesis, fast-check, jqwik) go further: they synthesise inputs from constraints and shrink failing cases to a minimal reproduction.
The typical mistake is to conflate data-driven with keyword-driven testing. Data-driven varies the inputs of a fixed scenario; keyword-driven varies the scenario itself, sequencing steps declared in a table. Mixing them into one framework tends to produce something that does neither well. Keep the scenario logic in code, keep the input variation in tables, and let the framework do the iteration.
Data-driven suites in 2026 are the fastest way to raise a suite's defect-detection power without adding new scenarios. A checkout total function tested with 40 tax-boundary rows finds rounding errors that a five-example unit test will never see. A login test parameterised over 12 email formats catches Unicode edge cases before a customer does. The discipline is to pick the rows deliberately: boundary values, empty and null inputs, locale variants, and known past-defect regressions.
Architecture & Production Code
A data-driven test has three moving parts: the data source, the parameter binding, and the assertion. The framework wires them together and produces one result per row.
┌───────────────────────┐
│ Data source │
│ - inline array │
│ - CSV / JSON / YAML │
│ - property generator │
└──────────┬────────────┘
│ rows
▼
┌───────────────────────┐ ┌──────────────────────────┐
│ Parameter binding │ ─────▶ │ Test body (runs N times) │
│ (@ParameterizedTest, │ │ assert compute(a) == b │
│ test.each, ...) │ └────────────┬─────────────┘
└───────────────────────┘ │
▼
┌──────────────────────┐
│ Report: 1 line/row │
│ ✓ 12 passed │
│ ✗ 3 failed (row #7) │
└──────────────────────┘The report is the payoff. When a framework reports 'test failed', you dig; when it reports 'test failed on row 7 with input (amount=-0.01, currency=JPY)', you fix. Always give each row a display name so the CI log tells the whole story without a debugger.
The data source is chosen by ownership. If the tester picks the values, inline them in code — that keeps them versioned with the logic. If a domain expert curates cases (tax boundaries, insurance rate tables, medical coding fixtures), a CSV or spreadsheet with a checked-in schema is friendlier. Never load data from a shared network mount; that turns your CI into a fragile mess.
Property-based generators are the level up. Instead of listing rows, you declare invariants ('for any positive amount and any supported currency, computeTotal(a,c) >= a') and the generator shrinks any failing input to its minimal form. Use it for pure functions with clear invariants; overkill for UI flows.
# tests/test_discount.py
import pytest
from cart import apply_discount
CASES = [
# (subtotal, code, expected)
(100.00, "SAVE10", 90.00),
(100.00, "SAVE20", 80.00),
(100.00, "SAVE100", 0.00),
(100.00, "SAVE101", 100.00), # invalid, no discount
(0.00, "SAVE10", 0.00),
(49.99, "FREESHIP", 49.99), # code applies to shipping only
]
@pytest.mark.parametrize("subtotal, code, expected", CASES,
ids=[f"{s}-{c}" for s, c, _ in CASES])
def test_apply_discount(subtotal, code, expected):
assert apply_discount(subtotal, code) == pytest.approx(expected)
// JUnit 5 equivalent
// @ParameterizedTest(name = "{0} + {1} = {2}")
// @CsvSource({
// "100.00, SAVE10, 90.00",
// "100.00, SAVE20, 80.00",
// "0.00, SAVE10, 0.00"
// })
// void applyDiscount(double subtotal, String code, double expected) {
// assertEquals(expected, cart.applyDiscount(subtotal, code), 0.001);
// }Data-Driven vs Keyword-Driven vs Hard-Coded Tests
| Aspect | Data-driven | Keyword-driven | Hard-coded |
|---|---|---|---|
| Varies | Inputs | Steps | Nothing |
| Author profile | Engineer + analyst | Analyst / manual QA | Engineer |
| Framework support | Every major framework | Robot Framework, Cucumber | Native test runner |
| Report granularity | 1 result per row | 1 result per scenario | 1 result per function |
| Best when | Same logic, many inputs | Same steps, different order | Small suite, prototypes |
| Common risk | Data drift from prod | Scenario sprawl | Copy-paste explosion |
Choose data-driven when the scenario is stable but the inputs vary. Choose keyword-driven when non-engineers need to compose new scenarios from a fixed vocabulary. Hard-coded tests are fine for prototypes but should not survive the second sprint.
Production Debugging Scenarios
Data-driven suites fail in ways that look weird until you notice the pattern.
One failing row hides behind a summarised report
- Symptom
- CI says 'test_apply_discount failed' with no indication of which input.
- Root cause
- The parametrize decorator was missing ids, so the runner logged only the function name.
- Fix
- Always provide ids=[...] or a name= template. Every row deserves a human-readable label in the report.
Shared fixture mutates between rows
- Symptom
- Row 1 passes, row 2 fails; running row 2 in isolation passes.
- Root cause
- A module-scope list or DB row was mutated by row 1 and reused by row 2.
- Fix
- Move mutable fixtures to function scope. For DB tests, wrap each row in a transaction that rolls back after assertion.
CSV data drifts from production schema
- Symptom
- Suite has been green for weeks; production returns 500 on the exact case the CSV was supposed to cover.
- Root cause
- A column was renamed in production but nobody updated the CSV.
- Fix
- Validate CSV headers against a versioned schema at test startup. Fail loudly when the schema drifts.
Practice this concept in a real QA interview
Run a live mock with our AI Interview Coach, tune your resume with the ATS Resume Reviewer, and screen live listings on the QA Jobs Radar.