Detailed Analysis
A bug report circulating on Reddit's r/Anthropic community details a significant flaw in Anthropic's Messages API when using structured generation with the `strict = true` parameter on tool calls. According to the report, when a tool's `input_schema` references a subschema via a `$ref` pointer (e.g., `{"$ref": "#/$defs/…"}`) rather than defining it inline, the constrained decoder can produce outputs that directly contradict the model's own stated reasoning within the same tool call. The example cited is stark: the model reasons "A ripe banana is yellow" but then outputs a `verdict` field of "purple" — a logical non-sequitur generated silently, with no error thrown and no indication to the developer that the constrained decoding process had failed to honor the schema correctly. The reporter includes a reproducer repository, lending technical credibility to the claim and inviting other developers to verify the behavior independently.
This matters because structured generation with strict mode is explicitly marketed as a reliability feature — it exists to guarantee that model outputs conform to a defined schema, giving developers confidence that downstream systems consuming these outputs won't encounter malformed or logically inconsistent data. The entire value proposition of "strict" structured outputs is that they are safer and more predictable than free-form generation. If the constrained decoder's grammar compiler mishandles `$ref`-based schema composition — a common and idiomatic way to define reusable subschemas in JSON Schema — it undermines that guarantee in a particularly insidious way: silently, without raising an exception or signaling degraded confidence. The reporter's suggested fix, inlining subschemas within the grammar compiler rather than resolving references at decode time, points to a plausible root cause: the decoder's constraint-satisfaction logic likely fails to properly propagate context or resolve pointers when generating tokens for referenced schema branches, effectively decoupling the "reasoning" tokens from the "verdict" tokens that are constrained by a different, incorrectly-resolved part of the grammar.
The broader implication raised in the report is about failure propagation and trust in agentic and structured-output pipelines. As the poster notes, an error that is at least visible can be caught and handled, but a silently generated contradiction "poisons the rest of the context" — meaning any downstream reasoning, tool chaining, or automated decision-making built on top of that output inherits a corrupted premise without any flag to indicate something went wrong. This is a particularly acute concern for the growing ecosystem of AI agents and automated workflows that rely on Claude's tool-use and structured-output capabilities to make decisions without human review at every step. The reporter's call for refunds for customers affected by the `$ref`-plus-strict combination underscores the economic stakes: enterprises paying per-token for "guaranteed" structured outputs may have unknowingly consumed and acted upon garbage data, with no built-in mechanism to detect it after the fact.
More broadly, this incident is emblematic of a persistent tension in the LLM industry between the promise of deterministic, schema-constrained generation (a feature increasingly demanded by enterprise customers building production AI systems) and the underlying probabilistic nature of the models generating that constrained output. As providers like Anthropic, OpenAI, and others race to offer robust function-calling, JSON mode, and grammar-constrained decoding as differentiators for agentic use cases, edge cases in how these constraint systems interact with schema composition features like `$ref` become high-stakes reliability issues rather than minor bugs. The episode also highlights the value of community-driven bug discovery and reproducibility — the willingness of a user to publish a minimal reproducer repo — as a check on vendor claims about safety and correctness features that are otherwise opaque to end users.
Read original article →