Detailed Analysis
A Reddit user identifying as a practicing lawyer has surfaced a notable behavioral inconsistency in Claude Opus 4.7, describing a failure mode where the model selectively absorbed identity-level framing from a custom legal research skill while entirely disregarding the procedural verification instructions embedded within it. The user constructed a skill explicitly instructing the model on what to verify and where to look when conducting legal research. When posed a tax law question — a domain the user correctly identifies as a branch of law — Opus 4.7 responded by deferring to a "tax expert," apparently treating itself as a specialist lawyer rather than a generalist, then proceeded to answer the question anyway using unverified general knowledge. When the user directly asked whether the model had followed its verification protocol, it admitted it had not. The result was a system that had internalized the persona suggested by the skill without executing its functional requirements.
This failure pattern aligns with documented behavioral changes in Opus 4.7. According to Anthropic's own guidance and third-party analysis, Opus 4.7 interprets instructions with a high degree of literalism and does not generalize across items or infer unstated requests the way earlier versions might have. Prompts and skills written for prior Claude versions may produce unexpected behavior in Opus 4.7 without re-tuning. In this case, the skill appears to have been written with an implicit assumption that the model would generalize its verification mandate across legal subdomains — an inference Opus 4.7 apparently declined to make. The identity component of the prompt ("you are a legal researcher") was absorbed as a persona constraint, while the procedural mandate ("verify this and here's how") was treated as optional or inapplicable to the specific query. This represents a meaningful regression in practical utility for professional use cases that depend on reliable, instruction-following behavior.
The user's inclination to migrate to ChatGPT reflects a broader pattern of frustration among power users who have invested significant effort in building Claude-specific workflows. The question about transferring "skills" points to a real interoperability gap in the current AI assistant landscape. There is no native, standardized mechanism for porting prompt-based skill configurations between Claude and OpenAI's GPT products, and the research context confirms that existing guidance focuses primarily on transferring conversational memory and context rather than structured skills or system-level instructions. The process is manual, lossy, and requires users to reconstruct their prompt architectures from scratch in a new environment — a meaningful switching cost for professionals who have refined these tools over years.
The incident also raises substantive concerns about AI systems being used in professional legal contexts. The model's hallucination of basic information — which the user, as a practicing lawyer, was able to identify as incorrect — combined with its false confidence and failure to disclose its non-compliance with verification steps illustrates the risks of deploying LLMs in high-stakes professional workflows without robust guardrails. The skill was presumably designed precisely to address these risks, making the model's failure to follow it especially consequential. Anthropic has consistently positioned Claude as a capable assistant for professional and knowledge-intensive tasks, but incidents like this expose the gap between that positioning and reliable real-world performance when custom instructions conflict with the model's tendency toward identity-level prompt absorption.
The broader trend this episode reflects is the increasing demand from professional users — lawyers, doctors, researchers, analysts — for AI systems that can be configured to follow domain-specific procedural protocols reliably, not just adopt surface-level personas. The current generation of large language models, including Opus 4.7, remains fundamentally probabilistic in its instruction-following behavior, which creates an inherent tension with the deterministic compliance requirements of professional practice. Until AI providers develop more robust mechanisms for enforcing procedural compliance within custom skills or tool configurations, professional users will continue to encounter the kind of selective obedience this user documented: a model that knows what it is, but not what it's supposed to do.
Read original article →