Detailed Analysis
A hobbyist developer has built and open-sourced a tool called "claude-fuse" that runs Claude and GPT-5.5 in parallel on identical prompts, then invokes a third, separate Claude instance to synthesize the two outputs into a single merged response. The project is notable for its cost architecture: rather than routing requests through metered API keys, it leverages existing consumer subscriptions — Claude's interface for the synthesis step and OpenAI's Codex CLI (operating in a headless, read-only mode) for GPT-5.5 — effectively making the marginal cost of multi-model querying zero for users who already subscribe to both services. The entire workflow is exposed as a single `/fuse` command within Claude Code, lowering the barrier to adoption considerably. The repository is MIT-licensed and includes an installation script, with the full benchmark methodology and results documented in the README.
To validate the tool beyond subjective impression, the developer conducted a small blind benchmark: five prompts were each answered by Claude alone, GPT-5.5 alone, and the fusion system, producing 15 anonymized responses that were evaluated by three independent judges without knowledge of which model produced which output. The fusion system won three of the five tasks and claimed nine of the fifteen first-place individual judge rankings. Its strongest performance came on the two coding tasks, where judges awarded unanimous first place — a result the developer attributes to the synthesizer's ability to preserve correct logic from whichever model had the more accurate implementation while incorporating superior documentation and edge-case handling from the other. The loss, notably, was on the most tightly constrained task: a product description with a hard 55-to-65 word ceiling. Merging two well-formed but independently unconstrained answers predictably produced an output that exceeded the word limit, and all three judges penalized it, placing the fusion last.
The benchmark's methodological limitations are worth acknowledging and, to the developer's credit, are explicitly disclosed. The sample size of five prompts is far too small for statistical conclusions, and the use of Claude-family models as evaluators introduces a potential bias toward outputs that resemble Claude's style — a form of in-group preference that could inflate fusion scores given Claude serves as the synthesis engine. The responses were also not length-normalized before judging, which may have advantaged longer, more thorough answers on open-ended tasks while compounding the penalty on the constrained writing prompt. The developer frames these results as "directional, not a published benchmark," which is an appropriate characterization.
The project reflects a broader DIY trend in the AI enthusiast community of building model-ensemble systems without institutional research infrastructure. The core intuition — that different large language models have complementary strengths, and that a meta-model can identify and combine the best elements of each — has academic precedent in mixture-of-experts architectures and LLM-as-judge research, but this implementation brings the concept to consumer-grade tooling. The routing of GPT-5.5 through Codex CLI rather than the standard API is a particularly inventive workaround, repurposing a developer tool designed for code assistance as a general inference endpoint to bypass per-token billing. This kind of subscription arbitrage is likely to become more common as AI companies bundle increasingly capable models into flat-rate tiers.
The clearest practical signal from the experiment is the task-type dependency of fusion utility. Open-ended tasks with objective correctability — debugging, code generation, technical analysis — appear to benefit from multi-model synthesis because errors in one model's output can be caught and corrected by referencing the other's. Tightly format-constrained tasks, by contrast, are actively harmed by synthesis, because combining two valid but independently constructed outputs almost always violates the constraint. This maps loosely onto a distinction between tasks where quality is additive and tasks where quality is conditional on a hard boundary. If the finding holds at larger scales, it suggests that effective use of tools like claude-fuse requires users to first classify their prompt type before deciding whether to invoke the fusion layer at all — a meta-decision that may itself be automatable.
Read original article →