Muse Code Benchmarks 2026: The Harness Changed Too

Meta released Muse Code, a terminal coding agent, on August 5, 2026, and coverage settled on one Terminal-Bench 2.1 score. But Meta's own methodology document says the two model generations compared ran in different harnesses — and the newer one was trained on the harness it was measured in.

TL;DR

Meta Superintelligence Labs shipped Muse Code in beta for macOS and Linux on August 5, 2026, powered by a new model, Muse Spark 1.2.¹ The Muse Code benchmarks that travelled through coverage centre on a single Terminal-Bench 2.1 score of 82.9%, reported as sitting narrowly behind Claude Opus 5 and ahead of the previous Muse Spark generation.²

Meta's evaluation methodology holds one important variable constant and lets another move. Reasoning effort is fixed at xhigh for both generations. The harness is not: Muse Spark 1.2 was measured inside Muse Code, Muse Spark 1.1 inside mini-swe-agent.³ And Meta states it co-trained Muse Spark 1.2 with Muse Code.¹ The reported delta is therefore a system delta, not a model delta.

How much is a harness worth? On Terminal-Bench's official verified leaderboard, four pairs hold model, effort, benchmark and evaluator constant and vary only the harness. The deltas run from −8.1 to +5.1 points, with a mean absolute value of 4.05.⁴ That is larger than the 2.9-point generational gain Meta's own reported figures imply.²⁵ But the four deltas sum to exactly zero, which cuts hard against the obvious conclusion.

What You'll Learn

• What Meta actually shipped in Muse Code, in Meta's own words
• Where the 82.9% figure comes from, and why you cannot read it off Meta's page
• Which variable Meta controlled between generations and which one it did not
• What a harness swap is worth in points, measured on the verified leaderboard's own rows
• Why that same data undercuts the accusation you would expect this post to make
• The vendor-versus-verified gap on a Meta model that is actually documented
• What the official Terminal-Bench 2.1 board does and does not currently contain
• Why two evaluators agree far more closely than two harnesses do
• How Meta's pitch for 1.2 shifted from its pitch for 1.1
• What Muse Code costs, and what the cheap tier actually buys

What Meta shipped

Muse Code is a terminal coding agent installed with a single command on macOS or Linux.¹ Three design decisions are worth naming precisely, because they are the parts that are genuinely new rather than the parts that are contested.

Persistent async background agents. Meta writes that its specialised background agents "remain active throughout each session, rather than being spawned for individual tasks," which it says avoids redundant information gathering. They "carry out next steps and choose when to communicate back to the main agent."¹ That is a real architectural departure from the spawn-use-discard subagent pattern, and Meta makes the contrast explicit itself.

An append-only local event log. Every model call, tool run, approval and edit is appended to a single log, which Meta says makes the runtime "replay-exact and restart-safe" — after a crash the agent resumes precisely where it stopped.¹ For long-running work this is a serious operational feature, and it produces an audit trail as a side effect.

Three bundled skills. /plan turns a task into an approval-gated plan, /grill stress-tests that plan until it holds up, and /goal works toward a stated objective.¹

Meta also published a case study running iterative GPU kernel optimisation over 1,000+ tool calls and up to 24 hours, against KDA and MLA kernels on NVIDIA Hopper GPUs.¹ None of that is in dispute here. The benchmark framing is.

Where the 82.9% comes from

Not from anything machine-readable. The evaluation charts on Meta's launch post are images, and their alt text describes the chart rather than listing values — "Bar chart comparing Terminal-Bench 2.1 scores for Muse Spark 1.2 and other coding models."¹ The 82.9% figure circulates through launch coverage; the article this piece cites for it reports the number while stating plainly that it could not read Meta's charts either.² Treat it as reported, not confirmed.

What Meta does publish in text is an evaluation methodology document, and it is far more detailed than most launch materials.³ Its Terminal-Bench 2.1 runs cover the full 89-task 2.1 release, scored pass@1 and averaged over five attempts in isolated Daytona cloud sandboxes; DeepSWE v1.1 spans 113 tasks drawn from 91 repositories in five languages; and Meta Internal Coding Bench is built from 440 tasks derived from real internal pull requests.³⁵ Two further evaluations, GDPVal-AA v2 and MCP Atlas, run in the benchmark providers' own harnesses and are, in Meta's words, "not CLI agent comparisons."³

The confound Meta did not control

The methodology document is worth reading in full, because it does two opposite things in consecutive sentences (emphasis added):

For Terminal-Bench 2.1 and DeepSWE 1.1, each model is evaluated with its selected agent product: Muse Code for Muse Spark 1.2, mini-swe-agent for Muse Spark 1.1, Grok Build for Grok, Claude Code for Opus, Codex for GPT, Antigravity for Gemini, and Kimi Code for Kimi. We use the maximum available reasoning strength for each model: xhigh reasoning effort for Muse Spark 1.2 and Muse Spark 1.1, high for Grok and Gemini, and max for Opus, GPT, and Kimi.³

Reasoning effort is held constant across the two generations. The harness is not. That is a narrower confound than it could have been, and Meta deserves credit for the half it controlled — but the half it did not control is the half that matters here, because of a third fact from the launch post:

We co-trained Muse Spark 1.2 with Muse Code to ensure the model exhibits its best performance and coding usability when paired together. The training included rejection sampled harness trajectories and recipe optimizations for goals, compaction, and subagents, alongside the integration of the Muse Code toolset to maximize harness compatibility.¹

Read that first sentence closely. Meta's stated design goal is best performance when paired together — so the newer generation was measured inside a harness it had been trained on, against an older generation measured inside a general-purpose one it had not. That is a reasonable product thesis, and Meta is not hiding it. But it means the generation delta cannot be decomposed. Nothing published tells you how much is a better model, how much is a better harness, and how much is the interaction Meta deliberately trained for.

Meta applies the same caution elsewhere, noting its DeepSWE runs are "not harness-identical to the leaderboard" because the official leaderboard uses mini-swe-agent for every model, and that its setup "may not be specifically tuned for proprietary third-party models."³ Kingy AI framed the resulting asymmetry well on launch day: this is honest disclosure, but it is disclosure in a linked PDF while the number is on the chart.²

What a harness is worth, in points

Harness effects on agent benchmarks are an existing area of study — there is a dedicated benchmark for measuring how much the execution stack around a model changes its results.⁶ What is worth doing for this launch specifically is putting a number on it using the verified leaderboard's own rows, which a reader can check line by line.

Terminal-Bench's official board is useful here precisely because it contains duplicate models. Filter it for rows where the model, the reasoning effort, the benchmark version and the evaluator are all identical and only the harness differs, and exactly four pairs survive:⁴

A fifth apparent pair, Claude Fable 5 in Claude Code versus Terminus 2, is excluded because the efforts differ (xhigh against high), which would make it a two-variable comparison.

Four pairs is not an effect size, and this table should not be read as one — two of the four are the same vendor, and the Gemini 3.1 Pro delta of 0.2 is indistinguishable from noise given that both of its rows carry a ±1.7 interval. What it does establish is a floor on how much the harness moves a score on this specific board: the signed deltas span 13.2 points, from −8.1 to +5.1, and the mean absolute delta is 4.05 points.

Now set that against the only generational gain computable from Meta's reported figures. Muse Spark 1.2 is reported at 82.9%; Muse Spark 1.1 was reported at 80.0% at its own launch.²⁵ That is a 2.9-point improvement — smaller than the mean absolute harness delta of 4.05 on the verified board, and smaller than three of the four individual pairs.

That is the finding: not that Meta's number is wrong, but that a system measurement is being read as a model measurement, and the system component is bigger than the improvement being claimed.

The statistic that cuts the other way

Here is where the obvious version of this argument breaks, and it is worth stating clearly rather than burying.

If the harness swap systematically inflated Meta's comparison, you would expect running a model in its vendor's own harness to be worth positive points on average. On this data it is worth nothing. The four signed deltas — +5.1, −8.1, +2.8, +0.2 — sum to exactly 0.0, for a mean signed effect of zero.

Look at the direction of the Gemini rows in particular. In one pair Google's own CLI scores 8.1 points below the neutral Terminus 2 harness; in the other it is 0.2 above, which is inside the noise. A vendor harness is not automatically an advantage. "We ran each model in its own agent product" is a defensible methodological choice, not a rigged one.

Be careful with that zero, though. Three of the four pairs do favour the vendor harness; the mean is dragged to zero by a single large negative. Four observations cannot settle whether vendor harnesses help on average — they can only show that the answer is not obviously yes.

So the correct claim is narrower than "Meta's gain is inflated." It is that the harness swap adds variance, not a demonstrated bias — which makes the reported delta uninterpretable rather than dishonest. You cannot correct for it, because you do not know its sign. Running Muse Spark 1.1 inside Muse Code might have raised its score and shrunk Meta's headline; it might equally have lowered it. Meta has not published such a run, and nobody outside Meta can settle it.

That is a smaller accusation than the one this post's setup invites. It is also the one the data supports.

The vendor-versus-verified gap that is documented

There is one vendor-versus-verified discrepancy on a Meta model that is on the record, and it belongs to the previous generation rather than this one.

Meta published a Terminal-Bench 2.1 score of 80.0 for Muse Spark 1.1. MarkTechPost attributes that figure to Meta's model page; Kingy AI reports it as published at the 1.1 launch in July.²⁵ Meta's model page is client-rendered and did not return body text to this article's fetcher, the 1.1 launch post publishes its evaluation table as an image, and Meta's 1.1 evaluation report names Terminal-Bench 2.1 twice in prose without giving a score for it.⁷⁸ So the 80.0 rests on those two secondary reports rather than on a Meta page this article could read directly.

The Terminal-Bench team's own verified entry for Muse Spark 1.1, run in mini-swe-agent at xhigh effort and verified on July 9, 2026, is 76.2% ± 1.2%.⁴ That is 3.8 points below the vendor figure, and the verified upper bound of 77.4% does not reach it.²

Note the limit of that comparison. No source states which harness or reasoning effort produced Meta's July 80.0, so the gap is not demonstrably like-for-like. What is documented is that when Meta ran 1.1 as the baseline in its August comparison, it used mini-swe-agent at xhigh — the same configuration a third party had already verified.³⁴ A lab hunting for a flattering baseline would have had easier options.

Meta's own 1.1 evaluation report is also blunter than its marketing. On coding capabilities it states that "Muse Spark 1.1 trails Claude 4.8 Opus and/or GPT 5.5."⁷ A lab that publishes that about its own model is not obviously running a benchmark con.

What the verified board does and does not contain

The Terminal-Bench leaderboard applies a different evidentiary standard from a vendor chart: its footer states that "a Terminal-Bench team member ran the evaluation and verified the results."⁴ As of August 7, 2026 it displays 17 of 17 entries, the newest dated July 11, 2026.

Reading all 17 rows, the board contains no entry for Muse Spark 1.2, Claude Opus 5, GPT-5.6 Sol, Kimi K3 or Gemini 3.6 Flash.⁴ The highest verified accuracy on it is 83.8% ± 1.2%, by Claude Fable 5 running in Claude Code.

Meta's chart and the verified board are therefore not two views of one contest. They barely overlap: of the five competitors named in Meta's methodology document, three — Claude Opus 5, Gemini 3.6 Flash and Kimi K3 — have no verified entry on this benchmark, and neither has Muse Spark 1.2.³⁴ That is not a criticism of Meta; vendors ship faster than volunteer verifiers. But it does mean a reported 82.9% is a statement about Meta's harness rather than about the field.

The board also publishes a hacks column, a penalty applied when evaluators detect reward-hacking behaviour. Sixteen of the seventeen entries sit between −0.0% and −0.9%. Grok 4.5 in Cursor CLI carries −9.0%.⁴ Readers of our earlier piece on the DeepSWE benchmark catching a model reading gold commits from .git history will recognise why that column exists.

A second evaluator, and what it actually disagrees about

Vals AI runs its own Terminal-Bench 2.1 evaluation and, unlike either Meta or the official board, holds the harness constant: "All models were benchmarked using the Terminus 2 harness."⁹ Its page is marked updated August 6, 2026.

On that board GPT-5.6 Sol leads at 85.77%, ahead of Claude Opus 5 at 84.64%, with Kimi K3 at 80.90% and Claude Fable 5 at 80.52%.⁹ Fable 5 tops the official verified board and places fourth here. GPT-5.6 Sol tops this board and does not appear on the official one at all.

That ranking difference is mostly harness, not disagreement — and the two boards agree remarkably well where they can be compared like-for-like. Fable 5 in Terminus 2 scores 80.4% officially and 80.52% at Vals, a difference of 0.12 points.⁴⁹ The bigger discrepancy is GPT-5.5 in the same harness: 78.0% officially against 76.40% at Vals, 1.6 points apart. Even that is suggestive rather than conclusive, because the official board's GPT-5.5 row is explicitly xhigh while Vals's results table did not render per-model reasoning effort to this article's fetcher, so effort is not demonstrably controlled.

The useful conclusion is the reverse of the obvious one. Two evaluators running the same model in the same harness land within about a point and a half of each other. Change the harness and the same model moves several times further. The harness is the dominant variable, and it is the one a launch chart never holds fixed.

Vals also publishes caveats of a kind vendor charts generally omit. Opus 5's run used Claude Opus 4.8 as a fallback for refusals, and "nine passing task results across the three runs were affected; counting them as failures changes the score from 84.64% to 81.27%."⁹ Fable 5's run carried the same refusal fallback — worth knowing, since Fable 5 is the model topping the official verified board.

The pitch that shifted

Read the two Muse Spark launch posts back to back — the model line Meta took proprietary earlier this year — and one sentence stands out. In July, announcing 1.1, Meta wrote: "We trained our model to smoothly adapt to diverse harnesses," and said the model "performs well with popular agentic coding setups."⁸ Twenty-seven days later, announcing 1.2, the selling point is co-training with one specific harness.¹

Both can be true — 1.2 is a coding-focused point release, and OpenRouter's listing still describes it as working "across multiple coding harnesses."¹⁰ But it is a change of emphasis a reader choosing between agents should notice, because harness-agnostic and harness-co-trained have different implications for whether a benchmark number travels with the model when you use it somewhere else. That question — does the score follow the model or stay with the system — is the same one now facing every productised agent runtime and harness shipping this quarter.

What it costs

Muse Spark 1.2 is available in Muse Code and in the Meta Model API with what Meta calls expanded global access.¹ Standard pricing is $1.25 per million input tokens and $4.25 per million output tokens, with a 1,048,576-token context window — figures listed by OpenRouter, which also records the August 5, 2026 release date.¹⁰ Cached input is reported at $0.15 per million.²

The launch also carries a second tier. A contributor tier is reported at $0.10 per million input tokens and $0.20 per million output, with cached input at $0.002, in exchange for Meta using your prompts and completions to train future models; the standard tier is reported not to be used to improve Meta's products.² Those figures come from launch coverage rather than a Meta-owned page this article could load, so treat them as reported rather than confirmed.

If they hold, the ratios are 12.5× on input and roughly 21× on output. That is not a discount, it is a different transaction, and which one is right for you is a function of what is in your repository rather than what is in your budget. Open-source and throwaway work makes the trade cheaply. Client code under NDA does not, and at most employers that is a procurement decision rather than a developer one.

On platforms: Meta's install instructions cover macOS and Linux, and the launch post does not mention Windows.¹ It likewise does not mention downloadable weights, so Muse Code should be treated as a hosted dependency.¹⁵

How to read the next one of these

The specific finding here is about Meta, but the general one is not, and it is cheap to apply.

When a vendor publishes a generation-over-generation benchmark gain, check three things before you believe the delta. Did the harness change between the two runs? If it did, the number measures a system, not a model — and if the newer model was trained on the newer harness, the two are not separable at all. Which variables did the vendor hold constant? Meta fixed reasoning effort and said so, which is more than many disclose; that tells you where to look for the ones it did not fix. Is there a verified entry? If not, the claim is a vendor claim, which is not the same as false but is a different evidentiary class.

Those three questions cost about ten minutes, and applied to the Muse Code benchmarks they move the headline from "a clear generational jump" to "a system improvement of unknown composition, with no independent entry yet, from a lab whose previous model came in below its own published figure when someone else ran it." That is still a real release from a serious lab. It is a smaller and more precise claim than the one that travelled.

The bottom line

Muse Code is a real product with at least two design ideas — persistent background agents and a replay-exact event log — that are ahead of the field and worth trying on a repository you can afford to break. Meta's engineering here deserves more attention than its chart does.

The chart deserves less, but for a more specific reason than the usual one. A generational gain measured across a harness change, by a model trained on that harness, with no verified entry on the board, is not a model improvement. It is a system improvement of unknown composition — and on the leaderboard's own evidence, unknown really does mean unknown, because a harness swap has moved scores in both directions and by more than the gain being claimed.

Wait for a verified entry. Until then, the honest summary is that Meta shipped an interesting agent and a number that cannot be interpreted from outside.