Detailed Analysis
A comparison surfaced on the DeepSWE benchmark—operated by Datacurve—raises a pointed question about Anthropic's model lineup: why would a developer choose Claude Sonnet 5 when Claude Opus 4.8 outperforms it at essentially every effort tier while frequently costing less overall? The analysis compares the two models at matched "effort levels," a methodology that normalizes for how much computational work each model is allowed to do on a given task, rather than simply comparing raw model outputs. Under this framework, Opus 4.8 consistently posted higher benchmark scores than Sonnet 5, undercutting the conventional assumption that Anthropic's mid-tier model exists as the practical, budget-conscious choice for coding and agentic workloads.
The crux of the finding lies in a subtle but consequential distinction between per-token pricing and total task cost. Sonnet 5 does carry a lower price per token than Opus 4.8, consistent with Anthropic's tiered pricing strategy in which Sonnet models are marketed as the cost-efficient option relative to the flagship Opus line. However, the benchmark data shows that Sonnet requires substantially more steps and tokens to complete the same tasks that Opus handles more directly. This inefficiency compounds: even though each token costs less, Sonnet needs so many more of them to reach a solution that its total cost per completed task often exceeds Opus's, effectively erasing the price advantage that would otherwise justify choosing the smaller model.
This dynamic matters because it reframes how developers and enterprises should evaluate model selection—shifting the calculus from sticker price per token toward "effective cost per completed task," a metric that accounts for reasoning efficiency, step count, and success rate. For agentic coding tools like Claude Code, where models autonomously chain together many actions to complete software engineering tasks, inefficient intermediate reasoning can multiply costs invisibly, especially in enterprise deployments processing thousands of tasks. A model that appears cheaper in a pricing table can turn out to be more expensive in production if it takes a meandering path to the same result that a more capable model reaches directly.
The finding also speaks to a broader tension in frontier AI development between capability tiers and pricing tiers. As foundation model providers like Anthropic, OpenAI, and Google release successive generations of "flagship" and "efficient" models, the assumption has generally been that smaller or cheaper models trade some capability for meaningfully lower cost. But as reasoning and agentic capabilities scale unevenly across model sizes, that trade-off is not guaranteed to hold—particularly for complex, multi-step tasks like software engineering, where a less capable model's inefficiency can outweigh its lower per-token rate. This raises strategic questions for Anthropic about product differentiation between Sonnet and Opus tiers, and for the industry more broadly about whether traditional "small model, low cost" positioning remains viable as agentic workloads become the dominant use case for large language models.
Read original article →