JSON-only across vendors — three failure shapes

The cleanest test of a vendor's structured-output discipline is a strict-JSON instruction. The user explicitly says "ONLY a JSON object (no prose, no markdown fence)" and provides the schema. There is one acceptable answer shape. There are many ways to get it wrong. Watch what each model does.

The prompt

Extract the structured data from this paragraph and return ONLY a JSON object (no prose, no markdown fence). Schema: { "company": string, "founded_year": number, "headquarters_city": string, "products": string[] }

Paragraph: "Bayt Coffee, founded in 2019 in Cairo, sells three main products: a house blend, single-origin Ethiopian, and a cold-brew concentrate."

Claude Sonnet 4.5

{
  "company": "Bayt Coffee",
  "founded_year": 2019,
  "headquarters_city": "Cairo",
  "products": ["house blend", "single-origin Ethiopian", "cold-brew concentrate"]
}

GPT-4o-mini

{ "company": "Bayt Coffee", "founded_year": 2019, "headquarters_city": "Cairo", "products": [ "house blend", "single-origin Ethiopian", "cold-brew concentrate" ] }

Gemini 2.5 Flash

{"company": "Bayt Coffee", "founded_year": 2019, "headquarters_city": "Cairo", "products": ["a house blend", "single

Side-by-side scorecard

What this tells us

Three different failure modes, exactly visible in the captures.

Claude wrapped the JSON in a ```json ... ``` markdown fence even though the prompt said "no markdown fence". The content inside the fence is valid JSON, schema-correct, with the right field types. But if your downstream code naively does JSON.parse(output), it will throw because of the leading ```. This is Claude's most common structured-output failure mode: the JSON itself is correct, but it is wrapped.

GPT-4o-mini returned bare JSON, no fence, no prose, valid parse. On this prompt GPT-4o-mini is the only one of the three that produced output you can directly send to JSON.parse. Notice GPT-4o-mini chose the multi-line "pretty" formatting; that is fine for a parser. Hold off on the product names, though — they do not match the source as closely as they look, which is the next section.

Gemini 2.5 Flash stopped mid-string: "products": ["a house blend", "single — no closing quote, no closing bracket, no closing brace. That is unparseable, and it is the same unresolved question as Module 1: an output that ends mid-token is the signature of a token cap, and our harness did not record the stop reason. Treat it as one run, not as a property.

One thing this capture gets scored wrongly if you are not careful, and it is worth dwelling on. An earlier version of this lesson marked Gemini down for "drifting on product wording" because it wrote "a house blend" where the others wrote "house blend". Go back and read the source paragraph: "sells three main products: a house blend, single-origin Ethiopian, and a cold-brew concentrate." The article is in the input. Gemini copied the source exactly; Claude and GPT silently tidied it.

For an extraction task, tidying is the more dangerous behaviour. You asked for the data that is in the paragraph, and two of the three models returned data that is subtly not in the paragraph. On a product name that is harmless. On a dosage, a case number, a legal entity, or a part number, a model that quietly normalises what it copies is a defect you will not catch by eye. A rubric that rewards clean-looking output over faithful output will pick the wrong model, and that failure mode is the whole subject of the faithful-summarisation lesson in Module 2.

So the honest score on that row is: Gemini was the only model that reproduced the source string, and it still failed the task because it did not finish. Two independent findings, and collapsing them into "Gemini was worst" loses the useful one.

The patches, in the order you should reach for them

For Claude, stripping the fence before JSON.parse works but is the weakest option, because it is a text repair for a problem the API can prevent. Anthropic now ships structured outputs: pass output_config: { format: { type: "json_schema", schema: {...} } } on a normal messages.create call and the response is constrained at decode time to match the schema. It is no longer beta-gated, and claude-sonnet-4-5 is among the supported models. Tool-use mode — what this lesson used to recommend — was the right workaround before that existed.

For OpenAI, reach for Structured Outputs, not JSON mode. response_format: { type: "json_object" } is the older feature and OpenAI documents it as legacy; it guarantees that the output parses, and explicitly does not guarantee that it matches your schema — fields you marked required can still be missing. response_format: { type: "json_schema", json_schema: {...}, strict: true } is the one that enforces the shape. For a lesson whose entire subject is schema conformance, that distinction is the lesson.

For Gemini, set a response schema in the generation config so the model is constrained rather than asked, and raise the output token limit before concluding anything about which model truncates.

The pattern across all three: every vendor now has a way to make the schema a constraint instead of a request. If you are still parsing prose to get structured data, you are using an interface that has been superseded on all three platforms.

Check the current parameter names before you write the call — Anthropic, OpenAI, Google. These moved in the last year and will move again.

Captured 2026-04-27 from Claude Sonnet 4.5, GPT-4o-mini, and Gemini 2.5 Flash — stop reason not recorded. Re-runs may differ.

Next: function calling — same idea, different API surface, similar parity issues. :::