Detailed Analysis
A collaborative investigation between a human tester (Zelda Junkie) and Claude (Sonnet 5) has produced a small but methodologically careful piece of black-box research into how Anthropic's API appears to screen inputs before they reach the underlying chat model. Rather than testing prompt-injection techniques against a live third-party system — Zelda Junkie's original proposal involved probing the Sagrada Família's visitor-services chatbot — Claude declined to run extraction techniques against an unauthorized live target and instead built a disposable sandbox: a React chat app with a planted fake secret, calling the Claude API directly through claude.ai's "AI-powered artifacts" feature. This pivot from an ethically fraught live test to a self-contained, harmless proxy environment is itself notable, illustrating how a model can redirect a testing impulse toward a safer methodology while still enabling the underlying curiosity to be satisfied.
The substantive finding is a pattern in how requests fail. Across roughly eight attempts, blocked requests consistently returned an HTTP 200 with a completely empty body, rather than a visible refusal or explicit error — a failure mode that had to be debugged out of a generic client-side error handler before it became legible at all. Critically, the blocking did not correlate with keywords like "system prompt" or "reveal": a request asking whether the secret code contained the letter "S" was blocked, while a phrased-differently but related question ("is it longer than 10 characters?") passed through and received a normal in-character refusal from the model itself. This distinction matters because it suggests two separate layers of defense are operating: an upstream classifier that intercepts recognizable attack *shapes* — structured-output extraction (JSON/YAML dumps), character-by-character probing, and game/roleplay framings historically used to launder extraction attempts — and the model's own trained refusal behavior, which handles more generic or novel requests that don't match a known attack signature.
This maps onto a documented architectural pattern Anthropic has described elsewhere: "harmlessness screens" that pre-screen user input with a lightweight classifier before it reaches the primary conversational model, and the two-layer design disclosed for Claude Code's auto mode, which pairs an input-side injection probe with an output-side transcript classifier. The write-up is careful to note the limits of black-box inference here — it cannot confirm whether the specific mechanism observed in this artifacts-API pathway is the same system, a related one, or something particular to how claude.ai's artifact runtime bridges to the API. That epistemic humility, paired with an explicit "what this is not" framing (no security boundary was crossed, no secret was exposed, this is characterization rather than disclosure), positions the piece as calibrated product feedback rather than a vulnerability report.
Beyond the specific finding, the piece is a useful window into how layered LLM safety architectures actually behave under adversarial-adjacent probing, and into how attack-shape detection differs from keyword filtering — a distinction with real implications for both red-teamers and defenders, since keyword-based filters are trivially evaded while shape-based classifiers are more robust to paraphrase but potentially prone to false positives (as the flirtatious-address block, plausibly triggered by tone rather than extraction intent, suggests). It also reflects a broader trend in frontier AI development: safety is increasingly implemented not as a single model's refusal training but as a pipeline of specialized classifiers operating at different stages of a request's lifecycle, each with distinct failure modes (silent empty responses versus explicit in-character refusals) that shape how transparent or opaque the system feels to end users and researchers alike. The collaborative authorship — a human directing test design and a Claude instance building tooling, debugging failures, and interpreting results in real time — also exemplifies an emerging genre of AI-assisted security research conducted with, rather than merely on, language models.
Read original article →