High coverage hides failures: strengthen the test oracle

Understand the likely causes

• Coverage records execution against an instrumented scope. It does not judge assertion strength, identify missing requirements, or guarantee realistic data. Shared actual/expected calculations create a circular oracle. A narrow model may omit duplicate requests, boundaries, or harmful persistent side effects even when all its lines execute.

1. Define what the percentage measures

Record measured files, exclusions, build, and whether the figure reports statements or branches. Branch coverage can expose an unvisited decision outcome, but a visited branch can still produce an unchecked wrong value. Compare the metric with requirements and risk mapping. If refund retry behavior is absent from the suite, write that gap explicitly rather than assuming a high global percentage covers it. Coverage is a useful searchlight for missing execution, not a product correctness percentage.

2. Replace circular expectations

For a reviewed policy that refunds fully at least 48 hours before a workshop and half otherwise, use independent expected cents at 48 and 47 hours. Do not compute expected = refund_cents(...) and compare the function with itself. Name rounding and currency assumptions. A fixed reviewed example is often clearer than a second complicated calculator that can reproduce the original mistake. The snippet is illustrative and does not cover payment integration.

def refund_cents(fee, hours_before):
    return fee if hours_before >= 48 else fee // 2

assert refund_cents(12000, 48) == 12000
assert refund_cents(12000, 47) == 6000

3. Assert outcomes and forbidden side effects

For a rejected enrollment, assert the intended validation or authorization category and verify no row or seat change. For a repeated refund request, inspect one refund identity and one financial effect. A test that accepts any non-200 response could pass while the service is unavailable. Include a valid baseline operation in the controlled environment and keep unrelated fields valid, ensuring the negative case actually reaches the targeted rule.

4. Challenge weak implementations deliberately

Ask whether returning zero, returning an empty list, skipping a write, or changing a boundary operator would still pass. For sorting, order alone accepts an empty result; add multiset preservation and a known exact example with duplicates. Controlled mutation testing can help find weak assertions, but its result also has scope and equivalent-mutant limitations. Do not chase a perfect mutation score while neglecting user risks that were never modeled.

Keep it from recurring

• Review assertions during code review, require independent oracles for important calculations, and keep boundary and retry cases tied to risk. Publish metric definitions and exclusions. Use coverage to guide investigation while explaining confidence through evidence quality and remaining uncertainty.