Detailed Analysis
A Reddit-published experiment by a developer running Claude Code against their own code-index MCP server offers a data point that cuts against a common assumption in agentic tool design: that keeping full tool schemas in the context window drives more tool usage. The developer ran two configurations across 32 sessions and three repositories. In the first, from before July 19, the code-index server was registered directly with Claude Code, exposing 15 tools with full JSON Schema definitions injected into the prompt on every turn — roughly 7.4 kB of token overhead. In the second, starting July 19, the same functionality was moved behind a CLI wrapper, collapsing the prompt footprint to a single 150-byte line. The behavioral result was nearly identical in both conditions: a roughly 1:13–1:14 ratio of index-server calls to grep/ripgrep calls, meaning the agent reached for grep about 13 times more often than the purpose-built index tool regardless of whether it had constant, verbose visibility into the tool's capabilities or a minimal pointer.
The finding matters because it challenges a load-bearing assumption in how developers build and reason about tool ecosystems for LLM agents: that schema visibility is the primary lever controlling tool selection. The intuitive design pattern — surface rich, detailed tool definitions so the model "sees" what's available and chooses appropriately — turns out, in this single-user case, to produce no measurable uplift in adoption. Removing over 95% of the prompt overhead associated with the tool didn't hurt usage, and a leaner index also didn't help usage, suggesting the bottleneck isn't context-window competition for the model's attention but something more structural in the agent's default behavior loop. The author's secondary analysis reinforces this: even in sessions where the model had already successfully invoked the index tool, grep was used in 8 out of 9 subsequent search-like actions, with a median gap of 27 calls between an index invocation and the next grep. This implies that once a session's context has "moved on" from the tool, the model reverts to its default, generalist search behavior (Bash-invoked grep or the native Grep tool) rather than re-reaching for a specialized instrument — a kind of habitual stickiness to broadly-applicable tools over narrower ones, independent of how much schema information is on offer.
This connects to a broader and increasingly discussed problem in agentic AI system design: the diminishing returns and possible counterproductivity of large, static tool-definition payloads in system prompts. As MCP (Model Context Protocol) adoption has grown through 2025, many teams have registered dozens of tools with verbose schemas, operating on the assumption that more available capability, clearly described, yields better task performance. Anthropic and others have increasingly emphasized techniques like tool-use filtering, dynamic tool loading, and "progressive disclosure" of capabilities specifically because large schema payloads consume context budget that could otherwise go toward reasoning traces, file contents, or conversation history. This experiment offers empirical, if narrow, support for that shift: it suggests that for narrow-utility tools competing against ingrained, general-purpose alternatives like grep, the cost of keeping schemas resident in context may be closer to pure overhead than to a genuine usage driver.
The author is appropriately cautious about generalizing from the results, noting confounds: a single user, a single working style, sequential rather than randomized time periods, and different projects across the two windows, plus the obvious conflict of interest in evaluating one's own tool. Still, the methodology — counting actual tool invocations against grep invocations across real coding sessions rather than relying on self-report or synthetic benchmarks — reflects a maturing, more empirical strain of community-driven research into how Claude Code and similar agents actually behave in practice, as opposed to how their designers assume they behave. As agentic coding tools proliferate and developers build increasingly elaborate MCP server ecosystems around Claude Code, this kind of small-scale, transparent instrumentation may become a valuable complement to official benchmarks, surfacing gaps between intended tool-selection design and the model's emergent, and sometimes stubbornly default, habits.
Read original article →