GPT-5.6 Sets a New High on Agents' Last Exam (2026)

An agent that scores 82% on one benchmark can drop to 25% on another — same model, same week, wildly different verdict on whether it can actually do the job. That gap is the entire reason Agents' Last Exam (ALE) exists, and on July 9, 2026, OpenAI's new GPT‑5.6 Sol posted the highest score anyone has recorded on it.¹ The number is real. So is the reason nobody should read it as "agents can now do real work."

In one line: OpenAI's GPT‑5.6 Sol, released July 9, 2026, set a new high on Agents' Last Exam — a UC Berkeley RDI benchmark built from real professional projects across 55 industries — scoring 53.6/100 with a 30.6% full-pass rate under its Codex-harness, XHigh-reasoning configuration, and edging out Claude Fable 5, whose own score is complicated by a safety system that quietly rerouted roughly a third of its test tasks to a weaker model.¹²³

TL;DR

• What happened: OpenAI's GPT‑5.6 family (Sol, Terra, Luna) went generally available July 9, 2026, and OpenAI made Agents' Last Exam its headline external benchmark. Sol's Codex-harness, XHigh-reasoning configuration hit a new record: 53.6/100 average score and a 30.6% full-pass rate on the 152-task public split.¹²
• What ALE is: A benchmark built by UC Berkeley's Center for Responsible, Decentralized Intelligence (Berkeley RDI) from real projects that 300+ working professionals actually completed — not toy tasks — spanning 55 non-physical industries, graded by deterministic scripts rather than an LLM judge.³⁴
• Why scores collapse: Agents that score 80%+ on benchmarks like Terminal-Bench routinely fall to the 20-30% range on ALE. Before GPT-5.6, the best full-pass rate recorded was roughly 25%; three quarters of failures trace to agents not understanding the job, not to broken tool calls.²⁴
• The number OpenAI itself reports two ways: OpenAI's own comparison table lists Sol's ALE score as 52.7, while its accompanying text calls out "a new high of 53.6" — a discrepancy on OpenAI's own page that matters more than it should, because the 13.1-point lead OpenAI advertises over Claude Fable 5 only works out arithmetically against the 53.6 figure.¹
• The Claude complication: On ALE specifically, Claude Fable 5's own safety classifiers rerouted about 35% of tasks — mostly life-sciences and biology-adjacent work flagged as "cybersecurity or biology" — to the weaker Claude Opus 4.8 mid-task. On the untouched 101 tasks, Fable 5 nearly matched GPT-5.5; on the 51 affected ones, it tracked Opus 4.8 almost exactly.⁵⁶
• The bigger picture: Even the new record — 30.6% full-pass — means GPT-5.6 Sol still fails roughly 7 in 10 real professional tasks in the benchmark. ALE's hardest tier averages a 2.6% full-pass rate across all tested systems.⁴

What You'll Learn

• What Agents' Last Exam actually measures, and how it's graded
• Why the same agents that ace other benchmarks collapse on this one
• The full trajectory: GPT-5.5's "surprise" June win, and GPT-5.6's July record
• Why Claude Fable 5's ALE score isn't a clean read on the model's capability
• What a 30.6% full-pass rate should (and shouldn't) tell you about deploying agents

What is Agents' Last Exam?

Agents' Last Exam is a benchmark built by UC Berkeley's Center for Responsible, Decentralized Intelligence (Berkeley RDI), co-led with the RDI Foundation, to test whether AI agents can complete the kind of long, economically valuable work that professionals are actually paid to do.³ The project, led by researchers including Yiyou Sun and Dawn Song, published its methodology in a June 2026 paper and now runs a live public leaderboard.⁴⁷

The design principle is unusual: every task started as somebody's real project. More than 300 practitioners — spanning academic affiliations like MIT, Harvard, and Stanford alongside industry partners including Goldman Sachs, JPMorgan, Morgan Stanley, and Adobe — contributed past work that historically took them days or weeks to finish, complete with the original input files and the software they used to do it.³⁴

Those projects were organized into a taxonomy built on O*NET/SOC 2018, the U.S. government's own occupational classification system, and filtered down to 55 subdomains across finance, law, visual media, computational mathematics, life sciences, and more.⁴ As of this month, the project has collected over 1,500 tasks toward an eventual 5,000-task target.³

Grading is the other unusual choice. Each task runs in an isolated virtual machine with read-only input files, pre-installed software, an output folder, and a hidden reference folder holding the expert's original deliverable.⁴ Most tasks use what the paper calls a "gate-and-score" pattern: a binary precondition has to pass first — a file has to parse without error, a 3D part can't collide with a milling head — and failing that gate zeroes the score regardless of partial progress.⁴

Wherever a deterministic check is possible, ALE uses one instead of asking another model to judge whether the output "looks right." Runs are capped at five hours and cost $3 to $10 each to grade.⁴

Why scores collapse: ALE versus everything else

The benchmark's own creators are direct about why this matters. Compared with Terminal-Bench and SWE-bench-Pro, ALE is broader (spanning 40 of its 55 industry subdomains, versus 6 and 5 respectively), longer-horizon (human time-to-complete runs from hours to weeks, not minutes), and harder — before GPT-5.6, the best full-pass rate recorded on ALE sat around 25%, compared with 82.0% on Terminal-Bench and 59.1% on SWE-bench-Pro for the same class of systems.²

That collapse isn't a story about tools breaking mid-run. When the ALE team categorized failures, roughly three-quarters traced back to agents not understanding the domain or choosing the wrong approach — not to crashed scripts or lost files.⁴

One concrete symptom: about a third of public ALE tasks hand the agent professional GUI software (SolidWorks-class CAD tools, DaVinci Resolve, that kind of thing), and agents routinely refuse to use it, instead improvising ad hoc terminal scripts that approximate — but don't replicate — what a domain expert would actually do.⁴ On the hardest tier of the full task pool, the average full-pass rate across tested harness-and-model combinations is just 2.6%.⁴

GPT-5.5's June win over Claude Fable 5 — and the catch

Before GPT-5.6 existed, the ALE leaderboard already had its first upset. In June 2026, Berkeley RDI published an interactive comparison running Codex (GPT-5.5), Claude Code (Fable 5), and Cursor (Composer 2.5) across the benchmark's task set.⁵ The composite result: GPT-5.5 led with a 24.0% full-pass rate, edging out Claude Fable 5's 22.0% — widely reported at the time as a surprise, since Claude had been favored going in.⁶

The catch, documented directly on ALE's own site, is that Claude Fable 5's score isn't a clean measurement of Fable 5 alone. Fable 5 is Anthropic's safety-guardrailed public release of its more capable "Mythos-class" model — Claude Mythos 5, the unrestricted version, was never independently testable and remains limited to vetted partners.⁸

Anthropic's classifiers screen every prompt reaching Fable 5, and on ALE specifically, roughly 35% of tasks got flagged as "cybersecurity or biology" and silently rerouted mid-task to the weaker Claude Opus 4.8 — almost entirely benign genomics, clinical, and physical-science tasks the filter caught alongside anything genuinely sensitive.⁵⁸ Every one of the benchmark's 19 life-sciences tasks triggered the fallback.⁵

Splitting the results tells a different story than the composite number. On the 101 tasks Fable 5 answered untouched, it scored 22.8% — essentially tied with GPT-5.5's 23.8% on the same split. On the 51 tasks where the safety system rerouted mid-task, the Fable-5-then-Opus-4.8 hybrid scored 17.6%, tracking Opus 4.8's own standalone 15.7% almost exactly.⁵

Chart: Agents' Last Exam full-pass rates for Claude Fable 5, Claude Opus 4.8, and GPT-5.5, split by whether Fable 5's safety fallback triggered mid-task. Source: Agents' Last Exam, "agent-showdown" blog (June 2026), agents-last-exam.org.

In other words: pure Fable 5 was close to GPT-5.5 all along — a distinction that got lost in most of the June coverage, which reported the blended 22.0%-vs-24.0% composite as a clean capability gap.⁹

One caveat is worth adding before calling this purely a safety-filter artifact: GPT-5.5, which has no fallback mechanism of its own, also drops to 17.6% on that same 51-task affected split — identical to the Fable-5-then-Opus-4.8 hybrid's score there. That suggests the affected tasks (overwhelmingly life-sciences and biology-adjacent work) may simply be harder for every system tested, not uniquely punishing for Fable 5.

So the safer conclusion is narrower than "no capability gap": the safety filter explains why Fable 5's composite score isn't a clean read on Fable 5 alone, but it doesn't establish that an unrestricted Fable 5 would have scored meaningfully higher than GPT-5.5 on those specific hard tasks — that's genuinely unknown, since Fable 5 never got to try them itself.

Anthropic's classifiers were still active when Fable 5 returned to general availability on July 1, 2026, "with new usage limits and safeguards," after a 19-day suspension triggered by a since-lifted U.S. export-control order — so there's no indication the same dynamic wouldn't still apply today.¹⁰

GPT-5.6 sets a new high — reported two different ways

OpenAI's GPT‑5.6 family — Sol (flagship), Terra (balanced), and Luna (fastest/cheapest) — went generally available on July 9, 2026, 13 days after the government-gated preview restricted access to a small group of partners under a cyber executive order.¹¹¹

Unlike that preview, this launch led with numbers, and ALE was the headline one: OpenAI's official announcement states that "GPT‑5.6 Sol sets a new high of 53.6, eclipsing Claude Fable 5 (adaptive reasoning) by 13.1 points," with the gap narrowing only slightly to 11.4 points at medium reasoning effort and a roughly four-times-lower estimated cost.¹

Here's the wrinkle worth knowing about before quoting that number elsewhere: OpenAI's own results table, on the very same page, lists Sol's ALE score as 52.7 — not 53.6.¹ The two figures aren't interchangeable.

Only the 53.6 figure reconciles with the stated "13.1 points" gap over Claude Fable 5's tabled score of 40.5 (53.6 − 40.5 = 13.1; 52.7 − 40.5 = 12.2, which doesn't match). Berkeley RDI's own July 15 research recap independently confirms the 53.6 figure, attributing it specifically to "Codex powered by GPT-5.6 Sol (XHigh reasoning)," and adds a second, differently-scaled metric: a 30.6% full-pass rate on the current 152-task public split (ALE-V1).²

That full-pass figure is the one that lines up on an apples-to-apples basis with the earlier 24.0%/22.0% numbers from June — and it shows genuine movement, from roughly 25% best-in-class before GPT-5.6 to 30.6% after.²⁴

Chart: Agents' Last Exam average scores (0–100 scale) by model. Source: OpenAI, "GPT-5.6" launch page, official comparison table (July 9, 2026), openai.com.

The two OpenAI numbers likely come down to configuration rather than an outright error — the table's plain "GPT-5.6 Sol" entry doesn't specify a harness or reasoning effort, while the prose's 53.6 is explicitly "Codex powered by GPT-5.6 Sol" at XHigh reasoning.

Worth being precise about the direction of ALE's own findings here, though: the benchmark's research found that swapping which model powers an agent moves scores roughly three times more than swapping the harness around it — model choice matters more than scaffolding, not the other way around.⁴

That actually argues against reading the 52.7-to-53.6 gap as mostly a harness effect; a 0.9-point difference is modest enough that a reasoning-effort difference, or simply a different reporting pipeline for the table versus the prose, is at least as plausible an explanation.

Either way, it's a useful reminder that even a company's own launch page can carry two different numbers for the same claimed result, and that "the AI agent benchmark score" is rarely just one number.

For scale, on the same official comparison table Claude Fable 5 actually posted a competitive 83.1% on Terminal-Bench 2.1, versus Sol's 88.8% (91.9% in its "Ultra" multi-agent mode) — underscoring how differently ALE ranks the same models compared with a benchmark built around shorter, more code-centric tasks.

It's the same lesson NerdLevelTech flagged when GPT-5.4 was reported beating humans at computer use: a single headline benchmark score rarely tells the whole story of what an agent can actually do.¹

What this means if you're building or buying

Treat any single ALE number — 53.6, 52.7, 30.6%, or otherwise — as a floor of plausibility, not a purchasing decision. A few practical takeaways follow from how this benchmark actually behaves:

• Check which metric you're reading. "Average score" (0-100, partial credit) and "full-pass rate" (% of tasks fully solved) are different scales measuring different things, and vendors don't always label which one they're quoting.
• Ask what harness produced the number. "GPT-5.6 Sol" and "Codex running GPT-5.6 Sol at XHigh reasoning" can post meaningfully different scores on the same benchmark. The scaffolding around a model is not a footnote.
• Read safety-driven fallbacks into any Claude Fable 5 comparison. A composite score that blends a model with its own weaker fallback isn't the same as a clean capability measurement — and this isn't disclosed by default in most secondhand benchmark roundups.
• Weight the Near-Term tier over the hardest tier for near-term planning. ALE's easier public tasks already show real, partial competence; its hardest tier is closer to a research milestone marker than a deployment signal.⁴
• Test on your own workflow before switching models. A benchmark built from 300 professionals' real projects is a far better proxy for "will this replace part of a job" than most agent benchmarks — but it's still not your job, your data, or your tools.

The Bottom Line

GPT-5.6 Sol's Agents' Last Exam score is a genuine record, and OpenAI is right that long-horizon task completion — not a static reasoning score — is where agent products actually succeed or fail in production.

But the record itself comes with an asterisk on OpenAI's own launch page (52.7 versus 53.6), and the model it's being compared against carries a bigger one: Claude Fable 5's showing on this specific benchmark is inseparable from Anthropic's own safety classifiers, which quietly handed off roughly a third of the workload to a weaker model.

None of that makes GPT-5.6 Sol's result less real. It does mean that one of the toughest AI agent benchmarks yet is still mostly telling us how far there is left to go — a 30.6% full-pass rate on real professional work, not a 30.6% chance the job is done.