Detailed Analysis
A Reddit post from the r/ClaudeAI community details an independent developer's implementation of Andrej Karpathy's "LLM Council" concept, rebuilt as a public-facing web app called Ask the Council. Karpathy, the prominent AI researcher and former OpenAI/Tesla executive, originally published the LLM Council pattern as a GitHub repository—a multi-agent deliberation system where multiple language model calls act as independent "advisors," critique each other's answers, and converge on a synthesized verdict. The developer behind this project took that command-line-oriented concept and translated it into an accessible hosted application, explicitly motivated by wanting non-technical users, including friends and a partner, to be able to use the tool without touching a repo or terminal.
The technical architecture is notable for how it operationalizes deliberative AI reasoning at scale. Five parallel Claude Sonnet calls generate advisor responses from distinct persona prompts, which are then shuffled and anonymized as "Response A–E" before being fed back to all five advisors for cross-review—identifying the strongest answer, the biggest blind spot, and critically, what all five responses collectively missed. A final synthesis call, upgraded to Claude Opus, produces the verdict. This totals eleven model calls per query, a meaningful compute cost that explains why the developer abandoned an earlier attempt to build this inside Anthropic's Cowork environment, where spawning eleven parallel agents proved too API-intensive, in favor of a purpose-built web app with more controlled infrastructure.
The persona design choices reveal an interesting use of Claude itself as a creative collaborator in the build process. Rather than generic labels like "Skeptic" or "Optimist," the developer asked Claude to cast historical figures whose reputations map onto specific cognitive styles: Diogenes as the Contrarian, Aristotle for First Principles thinking, Napoleon as the Expansionist, Frida Kahlo as the Outsider perspective, Julius Caesar as the Executor, and Marcus Aurelius as the Chairman who authors the final verdict. This anthropomorphized framing serves a practical purpose beyond novelty—it gives each parallel Claude instance a coherent, differentiated voice, which likely improves the diversity of reasoning paths compared to simply varying temperature or prompt phrasing alone. The developer's stated insight that "persona variety comes from prompting, not model size" reflects a broader and increasingly common observation in applied LLM work: prompt engineering and role conditioning can produce meaningfully divergent outputs from the same underlying model, reducing the need for expensive model diversity to achieve deliberative variety.
Two operational findings stand out as generalizable lessons for anyone building similar multi-agent critique systems. First, anonymizing responses before cross-review measurably increased the honesty and rigor of the models' critiques—an echo of blind peer-review dynamics in human institutions, suggesting LLMs exhibit some analog of social conformity or attribution bias that anonymization can mitigate. Second, the meta-question "what did ALL five miss?" emerged as the single highest-value prompt in the chain, functioning as a check against groupthink or shared blind spots across differently-prompted instances of the same base model. This matters because it points to a real limitation of ensemble approaches using a single model family: even with persona diversity, all instances share the same training data and underlying biases, so explicitly prompting for collective blind spots becomes a necessary countermeasure rather than a nice-to-have.
More broadly, this project sits within a growing wave of "AI council" or "mixture of agents" architectures that treat deliberation, critique, and synthesis as a way to squeeze more reliable reasoning out of existing models without waiting for next-generation capability jumps. Karpathy's original concept and this Claude-based reimplementation both reflect a maturing intuition in the AI community: that structured multi-agent debate, even among instances of the same underlying model, can outperform single-shot prompting for complex, high-stakes questions. The choice to mix model tiers—Sonnet for volume, Opus for final judgment—also illustrates a cost-conscious pattern likely to become standard practice as developers build consumer-facing products atop frontier models, using cheaper models for breadth and reserving premium models for the moments where quality most directly reaches the end user.
Read original article →