GPT-5.6 Sets a New High on Agents' Last Exam (2026)

[Alex]: Welcome back to Nerd Level Tech AI Cast, where we take your curiosity and run it through a neural net—several times for good measure.

[Jamie]: I’m Jamie, your resident “wait, what does that mean?” enthusiast.

[Alex]: And I’m Alex, your friendly neighborhood explainer of all things AI. Today, we're diving into something that shook up the agent world—OpenAI’s GPT-5.6 just set a new high score on Agents' Last Exam. Yes, that’s a real thing, and no, it’s not the pop quiz you forgot to study for.

[Jamie]: Honestly, I still have nightmares about pop quizzes. But this sounds way scarier—“Agents’ Last Exam” has some final boss energy. What actually is it?

[Alex]: [CHUCKLES] It does sound ominous, doesn’t it? But Agents' Last Exam—ALE for short—is a benchmark cooked up by UC Berkeley’s Center for Responsible, Decentralized Intelligence, or RDI. Instead of simple tests, ALE pulls real, professional work from over 300 actual practitioners across places like MIT, Stanford, Goldman Sachs—you know, just your average Tuesday crowd.

[Jamie]: So, not your typical “can this agent solve a Sudoku?” More like, “can it do my taxes while debugging code and editing a video?”

[Alex]: Exactly! These are tasks that took humans days or even weeks. ALE covers 55 different industries—finance, law, computational math, media, life sciences. It’s not a toy benchmark; it’s the big leagues.

[Jamie]: Okay, but how do they actually grade these AI agents? It’s not like you can just eyeball a complicated financial report and go, “Eh, that looks about right.”

[Alex]: Great question. They use what’s called a “gate-and-score” system. For each task, the agent works in a locked-down virtual machine with all the real files and software. First, it has to pass a binary gate, like “Did the file process without errors?” If it fails, it’s game over—zero points. If it passes, then it gets scored, often with scripts that compare its output to the human expert’s, right down to the details.

[Jamie]: So, no mercy grading. No “A for effort” if it almost gets there.

[Alex]: Nope. And here’s the kicker—these aren’t short jobs. Some take hours, some take days. Most other benchmarks are like sprints; ALE’s a marathon.

[Jamie]: So what did GPT-5.6 actually do that’s so special?

[Alex]: On July 9th, OpenAI’s GPT-5.6, specifically the “Sol” variant with the XHigh-reasoning setting, scored 53.6 out of 100 on ALE—that’s a new high. And it managed a 30.6% “full-pass” rate across 152 public tasks. For context, before this, the best anyone did was about 25%.

[Jamie]: Wait, 30.6%? That’s...not even a D-minus in school. Should we be impressed?

[Alex]: [LAUGHS] I see what you’re saying. It’s not perfect, but when you realize how hard these tasks are—and that even state-of-the-art agents regularly get 80% on benchmarks like Terminal-Bench, but then drop to 20 or 30% on ALE—it’s actually a big leap. ALE is designed to expose what agents can’t do, not just what they can.

[Jamie]: So ALE’s like the “real world” test, and most agents are finding out life is a lot harder than the practice problems.

[Alex]: Nailed it. Most failures come not from broken code, but from agents not understanding the job or making weird decisions. Like, they’re handed professional software—SolidWorks, DaVinci Resolve—and then they just try to hack it with random terminal scripts. It’s like giving a chef a fancy kitchen and watching them try to microwave ramen.

[Jamie]: [LAUGHS] Been there, done that. So, was GPT-5.6 just miles ahead of the competition, or is there more to the story?

[Alex]: There’s always more. The main rival here was Claude Fable 5 from Anthropic. But Claude’s results are messy because of its safety systems. About a third of the tasks—especially in life sciences and biology—got rerouted to a weaker model, Claude Opus 4.8, whenever the system thought things might get dicey.

[Jamie]: Wait, so Fable 5 didn’t even try those harder tasks? Isn’t that like sending in your backup goalie for the penalty shootout?

[Alex]: That’s a pretty good analogy! For the 101 tasks where Fable 5 actually ran, it almost matched GPT-5.5’s earlier results. But on the rerouted 51 tasks, it performed like Opus 4.8, which is a clear step down. So, the headline numbers comparing GPT-5.6 and Claude Fable 5 aren’t exactly apples-to-apples.

[Jamie]: I also heard that OpenAI itself couldn’t make up its mind about the score. What’s up with that?

[Alex]: [CHUCKLES] Yes, that’s a fun detail. Their official launch page lists two numbers: 52.7 and 53.6. The higher number matches the “Codex powered by GPT-5.6 Sol” with XHigh reasoning—the configuration that set the record. The lower one is probably just a different setup. Even OpenAI can’t resist a little scoreboard confusion.

[Jamie]: Classic. It’s like when your phone says you have “1 hour left” of battery, but it dies in 10 minutes. Never trust the numbers at face value!

[Alex]: Exactly. And the takeaway here is that even with record scores, GPT-5.6 still fails 7 out of 10 real tasks on ALE. The hardest tasks? The average pass rate is just 2.6%. So, AI agents aren’t quite ready to take over your job—yet.

[Jamie]: Good news for my rent payments. But seriously, what does all this mean for deploying agents in the real world? Should companies be excited or cautious?

[Alex]: Cautiously optimistic. ALE shows that agents are getting better at complex, real-world jobs, but there’s still a long road ahead. Benchmarks like Terminal-Bench make agents look like geniuses, but ALE reminds us how much context, nuance, and domain knowledge matter. If your work is as complicated as these ALE tasks, don’t fire your team just yet.

[Jamie]: So, in summary: GPT-5.6 set a new high, but the real world is still winning. And as always, the leaderboard is just a snapshot—not the whole picture.

[Alex]: Well put. If you want the full story, check out the public ALE leaderboard and dive into the methodology. It’s a fascinating look at what AI can—and can’t—do when the training wheels come off.

[Jamie]: And if you’re building anything with AI agents, remember: test them on more than just the easy stuff. Or as my grandma says, “Don’t trust a chef who only knows how to boil water.”

[Alex]: Wise words, Jamie’s grandma. That’s all for today’s Nerd Level Tech AI Cast. Thanks for tuning in to geek out with us!

[Jamie]: Don’t forget to subscribe, share, and let us know which agent you’d trust with your next group project—if any.

[Alex]: Until next time, keep your prompts clever and your benchmarks honest. [OUTRO MUSIC FADES OUT]