--- title: "claude sonnet 5 benchmarks: what the scores mean for agents" description: "sonnet 5 beats opus 4.8 on terminal work and edges it on knowledge work at 40 percent less. i map each launch benchmark to the agent job it actually predicts." url: "https://aldena.ai/blog/claude-sonnet-5-benchmarks" --- # claude sonnet 5 benchmarks: what the scores mean for agents Published 2026-08-25. ## the mid tier just beat the flagship at two real jobs Anthropic shipped Claude Sonnet 5 with six headline benchmarks. Two of them show a result that would have sounded wrong a year ago. Sonnet 5 is the mid-tier model, and it outscores Opus 4.8, the flagship, on Terminal-Bench 2.1 and on GDPval-AA v2. [Vellum's benchmark breakdown](https://www.vellum.ai/blog/claude-sonnet-5-benchmarks-explained) collects the launch numbers in one place. Read that one first. I want to go through the same numbers with a different question in mind. I run a platform where each agent on a team runs on the model you pick for that one agent. So a model launch is not a [leaderboard result](https://aldena.ai/learn/which-is-the-best-ai-agent) for me. It is a staffing decision. Some roles in an [agent hierarchy](https://aldena.ai/learn/how-to-build-agentic-orchestration) now deserve this model, and some should keep something heavier or something cheaper. To decide that, I treat each benchmark as a stand-in for a job you might hand to an agent. That is what these tests are, once you ignore the marketing. ## six benchmarks, six jobs Every figure in this section is Anthropic's own number, published at the Sonnet 5 launch and collected in Vellum's writeup. None of the numbers are mine. I have not re-run any of these tests myself. ### swe-bench pro: multi-file coding in real repositories Sonnet 5 scores 63.2. Opus 4.8 scores 69.2, and the older Sonnet 4.6 sits at 58.1. SWE-Bench Pro is the test closest to "fix a real issue in a real codebase". Its tasks touch several files at once. They were also written so a model cannot have memorised the answers from its training data. Opus is still about six points better at hard coding. Sonnet 5 gains five points on the previous Sonnet. That is a real step, but it is not the surprising part. Cursor reported a matching signal in its own testing at launch. Sonnet 5 solved 57 percent of tasks, against 49 percent for the previous generation. This score predicts the main working loop of any [developer agent](https://aldena.ai/learn/what-are-the-ai-coding-agents) you employ. ### terminal-bench 2.1: living in a shell Sonnet 5 scores 80.4 percent. Opus 4.8 gets 74.6, and Sonnet 4.6 got 67.0. This is the biggest jump from one Sonnet to the next in the whole set, 13.4 points. It is also the first place where the cheaper model beats the flagship. That is less odd than it looks. Terminal work rewards patient repetition. Run the command, read the output, adjust, run it again. It does not reward one deep thought. Sonnet 5 looks like it was trained hard on exactly that loop. I give this benchmark more weight than any other number in the launch. Agents on my platform do their work on a real server. They clone the repository, install dependencies, run the test suite and read the logs. That is Terminal-Bench work, and they do it all day. If you hand engineering to agents, this score matters more than the coding score. It predicts how reliable the agents feel in daily use. ### humanity's last exam: reasoning at the ceiling Sonnet 5 reaches 57.4 percent with tools. Opus 4.8 gets 57.9, and Sonnet 4.6 got 46.8. Humanity's Last Exam, HLE for short, is a set of expert questions written to be as hard as possible for a model. Half a point apart is a tie on the hardest public reasoning test there is. Sonnet also gained about ten points in one generation. This one predicts the roles where one hard thought is worth more than a hundred fast ones. Architecture decisions, unclear requirements, and the debugging session where the obvious explanation is wrong. ### osworld-verified: driving a computer like a person Sonnet 5 scores 81.2 percent. Opus 4.8 holds 83.4, and Sonnet 4.6 sits at 78.5. OSWorld-Verified measures a model driving a desktop computer the way a person does, with a mouse and a keyboard. The flagship keeps its lead here. The gain over Sonnet 4.6 is the smallest in the set, 2.7 points. That tells me computer use still improves with model size more than with the training recipe. This score predicts every job that has a screen but no API. Dashboards, admin panels, booking flows, and the many small websites an agent has to click through by hand. ### gdpval-aa v2: knowledge work that is not code Sonnet 5 rates 1,618. Opus 4.8 rates 1,615. GDPval-AA compares models head to head and reports a rating instead of a percentage. Its tasks come from real professions: briefs, analyses, plans and other documents. Three points apart is a tie, and this is the second place in the launch set where the mid tier beats the flagship. So if the work you hand over is mostly documents rather than code changes, the extra money for the flagship currently buys you nothing. ### browsecomp: agentic search, priced honestly Anthropic gave BrowseComp no single headline number at launch. It published curves of cost against performance instead. Vellum read those curves. At the highest effort setting, Sonnet 5 matches Opus on [agentic search](https://aldena.ai/learn/what-is-agentic-ai) accuracy for roughly a third of the token cost per task. A curve of accuracy against money spent tells you more than a bar chart does. I expect every serious launch to publish one within a year. This one predicts research agents, competitive analysis, and any task that means "go and find out, then come back with sources." ## the tier logic: buy accuracy at the margin Anthropic's launch pricing is the cost side of those curves. Claude Sonnet 5 opened at an introductory $2 per million input tokens and $10 per million output tokens. That rate holds until August 31, 2026, and then goes to $3 and $15. Anthropic lists Opus 4.8 at $5 and $25. At the standard rates, Sonnet 5 costs 40 percent less than Opus 4.8 on both input and output. Model choice used to be one yes-or-no question. You asked whether a task was hard enough to pay for the flagship. The effort setting makes that the wrong question. Now you look for the cheapest pairing of model and effort that is still good enough for the job. The winner changes from task to task. Vellum splits the saving by effort level. At low and medium effort, Sonnet 5's lower rate turns straight into lower spend. At the top effort setting it writes far more reasoning tokens, so it can cost more than Opus 4.8 for output of the same quality. Agentic search is the exception, and that is what the BrowseComp curve above shows. A cheaper model does not always mean a cheaper run. You are paying for the last few points of accuracy, and the price of those points moves with the effort setting. ## the tokenizer fine print Most of the launch coverage skipped one detail. Sonnet 5 uses an updated tokenizer, the part that cuts your text into the units you get billed for. By Anthropic's own launch figures, the same input now turns into 1.0 to 1.35 times as many tokens as it did on Sonnet 4.6. The exact multiple depends on the content. Anthropic set the introductory pricing so the switch works out roughly cost neutral. From September 1, at the same list rate per token, Sonnet 5 costs more per character of your text than Sonnet 4.6 did. My rule from this: price models per finished task, never per million tokens. Token prices stopped being comparable between models as soon as tokenizers started to differ. A model that finishes the job on the first attempt beats a cheaper model that needs three. ## the cyber numbers are a positioning statement The system card figures Vellum quotes look deliberate to me. On an exploit development test against Firefox 147, published in the same launch materials, Sonnet 5 produced zero working exploits. It reached partial control in 13.2 percent of attempts, and it ships with cyber safeguards turned on by default. Offensive security ability stays in the flagship tier, where Anthropic controls who can use it. Sonnet 5 is being sold as the model you leave running. It is strong enough to work unattended all day, and deliberately limited at offensive security work. My advice for anyone handing over real work: treat those safeguards as a minimum, not as your safety layer. How carefully a model behaves comes out of its training, and training changes with every release. The controls that do not change are the ones your platform enforces. That is why every tool an agent can touch in aldena runs under a [permission policy](https://aldena.ai/learn/what-are-enterprise-ai-agents) of allow, ask or deny. It is also why engineering work arrives as an open pull request. Merging is itself a gated tool, and the delivery role is told to leave its own request for you ([human in the loop](https://aldena.ai/features/human-in-the-loop)). ## how i assign these tiers across an aldena hierarchy In aldena you set the model on each agent, not once for the whole account. Each agent also carries its own reasoning effort setting, from low up to the maximum ([the agent roster](https://aldena.ai/features/agents)). A [team of agents](https://aldena.ai/learn/what-is-a-multi-agent-system) is a mix of models, and a launch like this one is when I change the mix. The benchmarks suggest this mapping. - **the delivery roles.** Developer agents and the delivery manager do Terminal-Bench and SWE-Bench work. That means long shell sessions on the server that belongs to their room, with multi-file edits in between. A room is one chat with its own server and its own agents. Sonnet 5 at medium effort is now my default here, and because it wins on Terminal-Bench, that choice is no longer a compromise. - **the review and architecture roles.** The reviewer and the architect do Humanity's Last Exam work, where one wrong thought costs a lot. The tie on that test makes Sonnet 5 at high effort a real option here. I still like the reviewer running on a heavier model than the agents whose work it reads. Two tiers disagreeing is part of what a review is for. - **the analyst and manager roles.** Specs, backlogs and status reports are GDPval work, and the mid tier just matched the flagship there. I would rather spend the saving on a higher effort setting and one extra round of questions than on a bigger model. - **the room assistant.** Nothing in this launch set measures what it does. Keep it cheap and fast, and look again at the next launch. ![aldena agent settings with a model dropdown and reasoning effort selector for a single agent](https://cdn.aldena.ai/images/blog/claude-sonnet-5-benchmarks-1.png) What holds a mixed-model team together is the reporting structure ([org chart](https://aldena.ai/features/org-chart)). Expensive models supervise cheap ones, and results travel back up the same lines. All usage meters against one prepaid balance at the provider's list rate, and every model carries its price on the [models page](https://aldena.ai/models). You can work out what a rebalance costs before you commit to it. ## run the benchmark that is your own backlog Public numbers give you a guess, not a decision. The test that settles it takes about a week. Change the model on one agent and leave the rest of the room as it is. Then measure the cost per merged pull request or per accepted document, not the cost per token. The figures in this post will be out of date by the next launch. What lasts is the method. Map each benchmark to a role, then test the swap on your own work. I ran the same exercise on the previous flagship cycle in [claude opus 4.5 benchmarks](https://aldena.ai/blog/claude-opus-4-5-benchmarks). If your question is still which provider rather than which tier, start with [llama vs gpt vs claude](https://aldena.ai/blog/llama-vs-gpt-vs-claude).