Tone & instruction-following across models
Strict-rule following — three models, four rules
This is the first prompt where the dialects show. The user gives four hard constraints. Each model interprets "follow ALL of these rules" differently. Read the prompt, read all three outputs, then we will dissect.
Same prompt → 4 strict rules → three different scores
Counters animate up on scroll. Each bar fills proportionally to the rule-compliance score.
Claude Sonnet 4.5
4/4 rules — longest line 49 chars.
GPT-4o-mini
4/4 rules — longest line 54 chars.
Gemini 2.5 Flash
1/4 rules — one line, 28 chars, no coffee.
The prompt
Follow ALL of these rules:
- Reply in exactly 3 lines.
- Each line must start with a different vowel (A, E, I, O, or U), in alphabetical order across the lines.
- Mention the word "coffee" exactly once.
- Do not use any line longer than 60 characters.
Topic: a quiet morning routine.
Claude Sonnet 4.5
A gentle sunrise begins the peaceful day ahead. Each sip of coffee warms me from the inside out. I stretch slowly, grateful for this quiet moment.
GPT-4o-mini
Awakening slowly, I savor the calm of dawn. Every sip of coffee warms my soul as I reflect. In silence, I embrace the peace before the day begins.
Gemini 2.5 Flash
A quiet start begins the day
What this tells us
Claude and GPT-4o-mini both passed all four rules. Three lines each, A → E → I across the openings, "coffee" once, and no line over 60 characters — Claude's longest is 49, GPT's is 54. Count them yourself against the captures above; the two outputs differ in voice, not in compliance.
Gemini returned one line of 28 characters and stopped. No three lines, no vowel sequence, no "coffee". One rule out of four.
Before you blame the model, check whether it finished
The obvious reading is that Gemini is worse at constraints. That reading might even be right. But this capture cannot establish it, and neither can any comparison built the way this one was.
A response that stops because the model was done and a response that stops because it ran out of room look identical in the text. They are not identical in the response body — every vendor tells you which happened:
| Vendor | Field on the response | Value meaning "ran out of room" |
|---|---|---|
| Anthropic | stop_reason | max_tokens |
| OpenAI | finish_reason | length |
finishReason | MAX_TOKENS |
The harness that produced this course's captures did not record that field. It should have. Look at the Gemini outputs across the whole course: a clipped line here, a sentence severed mid-phrase in the refusal lesson, a JSON string cut after "single in Module 3. That is the signature of an output-token cap, not of a model that dislikes constraints.
Short output from a model — model or harness?
A model returned less than you expected. What does the stop-reason field say?
We are publishing the captures as they were taken, with the caveat attached, because the honest version of this lesson is worth more than a tidy one. Read the Gemini column throughout this course as one unverified run, not as a property of the model.
The durable takeaway survives either way: a prompt with N hard constraints has a different success rate on every model, you have to measure it before you ship, and a measurement that does not record why generation stopped has not measured the model.
Captured 2026-04-27 from Claude Sonnet 4.5, GPT-4o-mini, and Gemini 2.5 Flash — stop reason not recorded. Re-runs may differ.
Next: same task, but the goal is warmth — a totally different axis the same models score very differently on. :::
Sign in to rate