T-03Tool Search & Deferred Loading

/tools/tool-search

Keep a large tool catalog out of the window: the model starts with a search tool, and only the definitions it finds are loaded.

Step-by-step walkthrough7 steps

Five servers, fifty-eight tools. Every definition costs tokens whether or not it gets used.
Load them all and ~55K tokens are gone before the task starts — and past a few dozen tools, selection accuracy starts to drop.
Instead the window starts with a search tool and the few tools used on every task. The rest are deferred.
When the model needs a capability, it searches the catalog for it by name and description.
The search returns a handful of references — not the catalog.
Only those definitions are expanded into context: full schemas, for three tools.
The window holds what this task needs, and because the prefix never changed, the prompt cache still hits.

Key insight

Tool definitions are context like anything else, and a five-server setup can spend ~55K tokens on them before the task begins. Deferring them keeps the window small and selection accurate — and because discovered definitions are appended to the conversation rather than the tool list, the cached prefix survives.

Failure mode

Tools the search cannot find. A terse name and a one-line description are invisible to a keyword search: the capability exists, and the agent concludes it doesn’t.

T-03Tool Search & Deferred Loading

Elsewhere in the atlas