Regression tests on demand

Once you've localized a bug and seen the fix, there's one more prompt that closes the loop: write the regression test. If you don't pin the fix with a test, the same bug will resurface in six months when someone refactors the function for an unrelated reason.

LLMs are excellent at this. The bug specification you wrote during localization is already a test specification — you just need to ask for it.

A regression test prompt has four parts:

1. The buggy input that produced wrong output
2. The expected correct output
3. The framework you use (pytest, vitest, go test)
4. A constraint on test naming so it's clear what the test is pinning

Here's the template:

Write a pytest regression test for the bug we just fixed.

• Buggy input: second_largest([5, 5])
• Expected return value: None (no second-largest distinct number exists)
• Test function name: test_returns_none_when_only_duplicates
• Output ONLY the test code in a single Python block, no prose, no imports beyond pytest.

You'll get back something like:

def test_returns_none_when_only_duplicates():
    assert second_largest([5, 5]) is None

One line. That's the regression test you would have written yourself if you weren't tired. The model is doing two seconds of work that protects you for years.

The naming constraint matters. test_returns_none_when_only_duplicates reads as documentation: anyone hitting that test failure six months from now knows immediately that the function is supposed to return None for duplicate-only input. Compare to test_second_largest_1 — same code, no documentation value.

You can chain this prompt with the localization prompt to make it cheaper. Add to the localization prompt:

After identifying the fix, output a single pytest regression test that would have caught this bug.

Now one prompt produces both: the diagnosis, the fix, and a test that pins the fix. Three turns of work in one round-trip.

A useful pattern for teams: keep a snippet called pin-the-fix that contains the regression test prompt template. When a teammate Slacks you "I just fixed the timezone bug in formatDate," your reply is "Add the regression test using pin-the-fix." The model writes it. The bug never comes back.

A subtle warning: the model will sometimes write tests that pass against the buggy code if you're not careful. Always run the test against the unfixed code to confirm it fails first. A regression test that passes before the fix is a placebo. This is the same red-green discipline you'd apply if you were writing the test by hand.

The red-green discipline visualised:

Module 3 next: refactor prompts that don't silently regress behaviour. :::