Detailed Analysis
A user report circulating on Reddit and echoed across developer forums raises pointed questions about whether Anthropic's Opus 5 represents genuine progress over its predecessor, Opus 4.8, for a specific but important category of work: long-form, context-heavy tasks with many interdependent constraints. The author, a native Japanese speaker working on a novel with complex worldbuilding, character knowledge states, and foreshadowing, describes a consistent pattern in which Opus 5 forms an early interpretation of a revision plan and then continues executing against that interpretation even after being explicitly corrected—sometimes acknowledging a correction in one part of a response while silently reverting to the rejected premise elsewhere. Critically, the author is careful to separate this from comparisons with Anthropic's higher-end "Fable 5" model (likely a codename or competitor reference), arguing that Fable 5 outperforming Opus 5 is unsurprising and not itself evidence of a problem. The concerning claim is narrower and more specific: that Opus 5 appears to be a regression from Opus 4.8, the model it's meant to succeed, on exactly the kind of complex, multi-turn, high-context work Opus-tier models are supposed to excel at.
The significance of this report lies in what it suggests about a potential gap between benchmark performance and real-world reliability in extended, stateful interactions. Modern LLM evaluations tend to emphasize single-shot task completion, coding benchmarks, or short-context reasoning puzzles—domains where a model can look highly capable while still struggling to track evolving constraints across dozens of turns. The author's account describes a failure mode that is subtle and easy to miss in typical eval suites: not incoherent output, but confidently wrong output built on a stale premise, sometimes contradicting itself within the same response. This is a different and arguably more insidious problem than obvious hallucination, because it requires careful human review to detect, and because correcting it doesn't reliably fix downstream reasoning—the author reports needing multiple correction turns to get results Opus 4.8 could produce in one pass. That has direct implications for the "usage efficiency" narrative Anthropic and others often use to market newer models; if a model consumes fewer tokens per turn but requires more turns to reach a correct answer, its net efficiency may be worse rather than better.
The report also connects to a broader and increasingly discussed phenomenon in AI development: the tension between models optimized for bounded, well-specified tasks (like greenfield coding or short creative prompts) versus models expected to serve as reliable collaborators across large, evolving bodies of context—big codebases, long manuscripts, extended planning documents. Corroborating reports cited in the thread from software developers working on large "brownfield" codebases describe structurally similar issues: misunderstood architecture, ignored constraints, scope creep, and confident continuation from incorrect initial assumptions. That convergence across completely different domains (fiction writing in Japanese versus software engineering in presumably English) is notable, because it undercuts the alternative explanation that this is a language-specific or genre-specific quirk. Instead, it points to something more fundamental about how the model handles context revision and premise-tracking as task complexity and instruction history accumulate.
This kind of grassroots, comparative bug report is characteristic of how the AI user community has increasingly become an informal QA layer for frontier model releases, surfacing regressions that internal benchmarks may not catch because they don't stress-test long-horizon consistency in the same way. For Anthropic, whose positioning of Opus as the flagship "most capable" tier depends on trust for exactly this kind of sustained, high-stakes work, reports like this are the kind of signal that typically prompts either clarifying guidance on appropriate use cases (e.g., recommending Opus 5 for bounded tasks and something else for large-context revision work) or a targeted fix in a subsequent point release. It also underscores a persistent open problem in LLM design more broadly: scaling raw capability and reasoning effort (as seen in "xhigh" effort settings mentioned in the report, which reportedly made the model more entrenched in its wrong interpretation rather than less) does not automatically translate into better context-tracking or premise revision, and may in some cases amplify commitment to an early, incorrect framing rather than encouraging reconsideration.
Read original article →