Bug localization prompts

The wrong way to debug with an LLM is to paste the broken function and say "this is wrong, fix it." The model rewrites the entire thing, you lose the existing structure, and you end up reviewing a brand new implementation instead of a one-line fix.

The right pattern is what working engineers actually do: localize first, fix second. A localization prompt asks the model to find the bug — it does not ask the model to rewrite the function.

This module's flow:

The full localize-then-fix loop:

Here's a localization prompt against a buggy second_largest:

The following Python function is supposed to return the second-largest distinct number, but it sometimes returns the largest. Identify the bug, give the exact line, and the one-line fix. Do not rewrite the whole function.

def second_largest(nums):
    if len(nums) < 2:
        return None
    largest = float("-inf")
    second = float("-inf")
    for n in nums:
        if n > largest:
            second = largest
            largest = n
        elif n > second:
            second = n
    return second

Test that fails: second_largest([5, 5]) returns -inf, expected None.

Claude Sonnet 4.5 returned:

## Bug Identification

Line with bug: Line 10 (the `elif n > second:` condition)

Problem: When all numbers are the same (like `[5, 5]`), the function doesn't
check if `n` is distinct from `largest` before updating `second`. This allows
duplicate values to become the "second largest."

One-line fix:
        elif n > second and n != largest:

This ensures that `second` only gets updated if the number is both greater than
the current second AND different from the largest, guaranteeing we get the
second-largest distinct number.

Captured from Claude Sonnet 4.5 (claude-sonnet-4-5) on 2026-04-27. Re-runs may differ slightly. See the model-lifecycle note in Module 1, lesson 1.

Now run it. Apply that one-line fix and call the exact test from the prompt:

>>> second_largest([5, 5])
-inf

Unchanged. The prompt supplied a failing test — second_largest([5, 5]) returns -inf, expected None — and the fix the model proposed does not make that test pass.

The diagnosis is not worthless, which is what makes this worth studying. Trace [5, 3, 5] and the fix earns its keep: without n != largest, the trailing 5 would overwrite second with 5; with it, second correctly stays 3. The model found a real duplicate-handling bug. It just did not find the bug the failing test describes, and it presented the one it found as if it had.

The bug the test is actually pointing at is a different one: when every value is identical, second is never assigned at all, so the sentinel float("-inf") leaks out as a return value. No elif condition can fix that, because the branch never runs. The function needs to stop returning its sentinel:

    return second if second != float("-inf") else None

The lesson inside the lesson. A localization prompt returns a hypothesis, formatted with the confidence of a conclusion — headings, a line number, a fix, and a paragraph explaining why it works. None of that formatting is evidence. The failing test you pasted into the prompt is the only thing in the exchange that can settle the question, and it takes about four seconds to re-run.

So add one step to the loop, permanently:

Re-run the failing test against the proposed fix before you read the explanation. If the test still fails, the explanation is describing a different bug.

This costs nothing and it is the only check that catches a confident, well-argued, wrong answer. Everything else about this exchange — the prompt shape, the constraint on output, the refusal to rewrite — worked exactly as intended. A good prompt buys you a good hypothesis. It does not buy you a correct one.

Three things make this prompt work:

1. The expected behaviour is stated in one sentence — "supposed to return the second-largest distinct number." The model now knows what correctness means.
2. A failing test case is included — second_largest([5, 5]) returns -inf, expected None. This anchors the bug to a reproducible input.
3. The output shape is constrained — "exact line, one-line fix, do not rewrite." The model can't escape into a refactor.

The "do not rewrite" instruction is the single most important word in this prompt. Without it, the model defaults to the helpful behaviour of returning a clean rewrite — which makes it harder to review, harder to land as a small PR, and harder to learn from.

When you can't include a failing test (the bug is intermittent, or you don't have a repro), describe the symptom precisely instead. "Returns 0 when input is empty" beats "doesn't work for empty inputs." Concrete inputs and outputs give the model a target. Vague prose gives it room to invent.

Next up: making the model explain a real traceback in plain language. :::