---
title: "claude opus 4.5 benchmarks: which numbers predict quality"
description: "opus 4.5's reported scores, read by someone who runs agent teams on these models: which benchmarks predict delegated work, and which ones are noise."
url: "https://aldena.ai/blog/claude-opus-4-5-benchmarks"
---

# claude opus 4.5 benchmarks: which numbers predict quality

Published 2026-08-24.

## five numbers on the opus 4.5 card predicted six months of delegated work

I am writing this in August 2026, so Claude Opus 4.5 is not the newest model anymore. It shipped in late November 2025, and the Claude 5 family has arrived since. I read [Sonnet 5's numbers separately](https://aldena.ai/blog/claude-sonnet-5-benchmarks). I keep coming back to the Opus 4.5 card anyway. It is the rare benchmark story I can now check against real production work.

I run a platform where [teams of agents](https://aldena.ai/learn/what-is-a-multi-agent-system) plan, code, and ship real work. Opus 4.5 filled many of those seats through the first half of 2026. A seat is one named role on an agent team. With that hindsight the card sorts cleanly. Five numbers predicted how the model behaved as a delegated worker: two coding rows, one novel-reasoning row, one long-horizon row, and one injection row. The rest of the card was honest losses that barely mattered for this job.

[Vellum's benchmark analysis](https://www.vellum.ai/blog/claude-opus-4-5-benchmarks) is the best single collection of the reported figures. It lines Opus 4.5 up against GPT-5.1 and Gemini 3 Pro, and I lean on it throughout. The numbers under it come from [Anthropic's launch materials and model card](https://www.anthropic.com/news/claude-opus-4-5), plus third-party runs like Andon Labs' Vending-Bench 2 and the Gray Swan arena. I ran none of these evaluations myself. What I add is the reading: which scores predict the quality of work you can hand off and then walk away from.

## the reported scores, side by side

Every figure below is as reported at launch in November 2025 and collected in Vellum's roundup at the time. I have not re-run any of them.

| benchmark                                       | claude opus 4.5 | gpt-5.1    | gemini 3 pro |
| ----------------------------------------------- | --------------- | ---------- | ------------ |
| SWE-bench Verified (coding)                     | 80.9%           | 76.3%      | 76.2%        |
| Terminal-Bench (terminal operation)             | 59.3%           | 47.6%      | 54.2%        |
| ARC-AGI-2 (novel reasoning)                     | 37.6%           | 17.6%      | 31.1%        |
| GPQA Diamond (science knowledge)                | 87.0%           | 88.1%      | 91.9%        |
| MMMLU (multilingual)                            | 90.8%           | 91.0%      | 91.8%        |
| MMMU (visual reasoning)                         | 80.7%           | 85.4%      | 81.0%        |
| Vending-Bench 2 (long-horizon, net worth)       | $4,967.06       | not listed | $5,478.16    |
| Gray Swan injection attack success (lower wins) | 4.7%            | 21.9%      | 12.5%        |

Opus 4.5 won everywhere the benchmark looks like a job: fixing bugs in real repositories, driving a terminal, solving puzzles it has never seen, resisting injected instructions. It lost everywhere the benchmark looks like an exam: science quizzes, multilingual multiple choice, visual questions. That split is the argument of this post.

## coding: the pair of numbers that maps to pull requests

SWE-bench Verified is the closest public benchmark to the daily work of a [coding agent](https://aldena.ai/learn/what-are-the-ai-coding-agents). SWE-bench is short for software engineering benchmark. The Verified set is real GitHub issues in real repositories, graded by whether the tests pass. Opus 4.5's SWE-bench Verified score at launch was 80.9%, against 76.3% for GPT-5.1 and 76.2% for Gemini 3 Pro. Scores on that benchmark were already bunched near the top, so a four-point lead is wide. Anthropic presented it as the first model to clear 80% there.

The row I watched more closely is Terminal-Bench. The launch numbers were 59.3% for Opus 4.5, 54.2% for Gemini 3 Pro, and 47.6% for GPT-5.1. Most of a working agent's time goes to operating a machine. That means cloning a repository, installing dependencies, running the test suite, reading an error trace, and deciding whether to retry or change course. Writing the code is the smaller part of the run.

A room here is one chat with a team of agents, with one server behind it. In my rooms the failure mode that wastes the most credits is rarely "wrote the wrong function". It is "got lost operating the computer". A model that leads on both halves of the pair is one you can leave alone for longer. Opus 4.5 led on both. Either number alone would have misled me. A high SWE-bench score with weak terminal skills describes a brilliant engineer who cannot work the tools.

![Anthropic's launch comparison table for Claude Opus 4.5, Sonnet 4.5, Opus 4.1, Gemini 3 Pro, and GPT-5.1 across nine benchmarks, with the Opus 4.5 column highlighted](https://cdn.aldena.ai/images/blog/claude-opus-4-5-benchmarks-2.png)

## reasoning: arc-agi-2 told me more than gpqa did

ARC-AGI-2 is a set of abstract puzzles built to resist memorization. It produced the widest gap on the whole card. Opus 4.5 scored 37.6% at launch, more than double GPT-5.1's 17.6% and clearly ahead of Gemini 3 Pro's 31.1%. GPQA Diamond, a graduate-level science exam, went the other way. Opus 4.5 scored 87.0% there, behind GPT-5.1's 88.1% and well behind Gemini 3 Pro's 91.9%. Vellum also reports Humanity's Last Exam, another exam-style benchmark built to be very hard. Opus 4.5 sat at around 43.2% there with search. Gemini 3 Pro was roughly 7 points ahead of that without search.

For delegated work I weight those two rows very differently. An agent on the job has tools. It can search, read files, and check documentation, so those tools supply most stored exam knowledge for free.

No tool supplies the answer when the plan meets a surprise: a flaky test, an unclear requirement, a dependency that will not build on the room's server. Novel-problem reasoning is the closest public proxy for that moment. It is also the moment that decides whether an agent finishes the job or keeps repeating the same failed steps. ARC-AGI-2 predicted how well I could delegate. GPQA predicted quiz scores.

## multilingual and visual: real losses, mostly noise for this seat

Opus 4.5 lost some rows, and those deserve the same attention. On MMMLU, the multilingual multiple-choice exam, it scored 90.8% at launch. That was the lowest of the three frontier models, behind GPT-5.1's 91.0% and Gemini 3 Pro's 91.8%, though ahead of Sonnet 4.5's 89.1%. On MMMU, the visual reasoning exam, it came last again with 80.7%, against Gemini 3 Pro's 81.0% and GPT-5.1's 85.4%. The gap to GPT-5.1 there is not a rounding error.

Whether those losses matter depends entirely on the seat. An agent answering customers in Portuguese and Japanese will feel the multilingual gap. An agent judging screenshots of a user interface has a good reason to run a different model from the one writing the backend. A backend developer agent touches neither number in a normal week. The exam rows are real, but they matter little for the seats most people were hiring Opus 4.5 into. This is the strongest argument I know for picking models per role instead of per company. It is also why my [three-way comparison of Llama, GPT, and Claude](https://aldena.ai/blog/llama-vs-gpt-vs-claude) refuses to name [a single winner](https://aldena.ai/learn/which-is-the-best-ai-agent).

## long-horizon planning: the closest thing to a job interview

Vending-Bench 2 is run by Andon Labs. It hands the model a simulated vending machine business to run over a long stretch. The model manages pricing, ordering, and cash, and the score is the closing net worth. In the launch results Opus 4.5 finished at $4,967.06. That is a clear step up from Sonnet 4.5's $3,849.74, and still short of Gemini 3 Pro's $5,478.16. Another honest loss.

I still rank this benchmark near the top of the card. It is the only widely reported number that measures how errors compound across hundreds of decisions in a row. That is the same pressure a manager agent carries when it holds a backlog through a week of [handoffs](https://aldena.ai/learn/what-is-agentic-orchestration). No single decision is hard, and every small drift adds up.

I do not over-read it. It is one scenario, and scores quoted to the cent make small differences look more precise than they are. So I read it as a ranking rather than a measurement. As a ranking it said Opus 4.5 could hold a plan far longer than Sonnet 4.5 did, while Gemini 3 Pro held it longer still.

## agentic safety: the least glamorous number predicts the most

The Gray Swan arena attacks a model doing [agentic work](https://aldena.ai/learn/what-is-agentic-ai) with prompt injections. A prompt injection is a hidden instruction planted in text the model reads. The arena reports how often the attack works. Vellum's roundup of the launch numbers gives 4.7% for Opus 4.5, 12.5% for Gemini 3 Pro, and 21.9% for GPT-5.1. For a chat model that row is a curiosity. For a model holding a shell, a repository, and an inbox, it is the number that matters most. Read the percentages as frequencies: roughly one crafted attack in five landed against GPT-5.1, one in eight against Gemini 3 Pro, and one in twenty-one against Opus 4.5.

One in twenty-one is a large relative win, and it is still not zero. No number on a model card makes untrusted input safe by itself. Injection resistance lowers how often an attack succeeds. What you build around the model decides how much damage a successful attack can do. A benchmark cannot show that second half, and that is where the platform does the work.

## how model choice maps to agent roles in aldena

In aldena, model choice is per agent, not per account. A room [hires named roles](https://aldena.ai/learn/what-are-ai-employees): analyst, architect, developers, a reviewer, a bug fixer. Each seat carries its own model and its own reasoning effort, from low to max. The [roster](https://aldena.ai/features/agents) shows the roles, and the [models page](https://aldena.ai/models) carries every model's current rate.

Opus 4.5 shipped with an effort parameter, which trades thinking time against cost. Anthropic also cut the price at launch, to $5 per million input tokens and $25 per million output, about a third of Opus 4.1's rate. Those are Anthropic's November 2025 figures. That pricing per million tokens is what made a frontier model in an always-working seat worth it at all. The card above translated into seating decisions almost row by row:

- Builder seats, meaning the developers and the bug fixer, are chosen on the coding pair. The 80.9% and the 59.3% made Opus 4.5 my default in those seats through spring 2026.
- The reviewer and the architect are chosen on the ARC-AGI-2-shaped rows, at high effort. The reviewer never edits code, so there I pay for judgment alone, never for speed.
- Manager seats care most about the long-horizon row, because the manager is what holds the plan across handoffs.
- The front-door seats, the assistant and the project manager, get something cheap and fast. Exam gaps do not matter at that desk.
- Every seat runs behind the [approval gate](https://aldena.ai/features/human-in-the-loop), whatever its Gray Swan number. Each tool an agent can reach carries an allow, ask, or deny policy. A paused run waits durably for an answer, and no agent merges its own pull request. The benchmark tells you how often an attack lands. The gate limits what a landed attack can touch.

![aldena agent settings with a model selected and reasoning effort set for one agent role](https://cdn.aldena.ai/images/blog/claude-opus-4-5-benchmarks-1.png)

The Claude 5 family has since taken several of those seats. I judged each swap by the same rows, not by the exam ones.

## the card-reading checklist I kept

New model cards keep arriving, and this one taught me a reading order for anyone putting models into agent seats.

First, read the coding pair together: a repository benchmark and a terminal benchmark. Either one alone is misleading. Second, read one long-horizon number, and read it as a ranking. Third, read injection attack success as a frequency. Then add your own approval gates instead of trusting the number on its own. Treat small exam gaps as noise, unless the seat reads screenshots or answers in fourteen languages.

Read that way, Opus 4.5's card said the model was the strongest delegated worker of its generation and the weakest exam-taker. Six months of pull requests later, that is exactly the model it turned out to be.