Google has added Tool Retrieval to the Gemini API, a preview feature that keeps an agent’s tool schemas out of the prompt until the model asks for one. Google says it’s designed to cut input tokens and keep tool selection accurate as an agent connects to dozens or hundreds of tools.
Lucia Loher, a product manager for the Gemini API, and Nora Kassner, a research scientist at Google DeepMind, wrote the announcement, which went up on 19 November. Google for Developers pointed to it on X, saying developers can scale to hundreds of tools without context bloat.
Introducing Tool Retrieval in the Gemini API ✨
— Google for Developers (@googledevs) November 19, 2026
With Tool Retrieval, you can scale to hundreds of tools without context bloat. Load parameter schemas on demand to preserve selection accuracy, cut tokens, and keep your prefix cache intact.
Read the blog: https://t.co/Pd2SvVosuL pic.twitter.com/Dths6RxWaD
How it works
Every function you give a model comes with a schema, the JSON description of its parameters and which ones are required. Normally all of them are sent on every request. Google says that inflates input token counts, adds latency and lowers tool-selection accuracy as the list grows.
To turn Tool Retrieval on, you add a tool_search tool to the request and set defer_loading: true on the tools you want held back. The model still sees the name and description of each deferred tool, so it knows what exists, but the parameter schemas stay out of context. When the model needs one, it asks for it by name, the API loads the schema and the model emits the function call, all inside the same request. Google’s example defers an issue_refund function and asks Gemini 4 Hydrogen to refund US$40 on a damaged order.
tools = [
{"type": "tool_search"},
{
"type": "function",
"name": "issue_refund",
"description": "Refund an order to the original payment method.",
"defer_loading": True,
"parameters": {...},
},
# ... 50+ other deferred functions
]

Google’s docs say the search and schema steps appear in the response as tool_search_call and tool_search_result, and that you don’t act on them. Your own functions still run in your code, and you return a function_result as usual. Google’s advice is to leave the handful of tools an agent uses most undeferred and defer the long tail. A deferred function can also carry a short_description, which the model reads in place of the full description; in early testing each deferred tool added about 26 tokens, the docs say.

Server-side or client-side search
Gemini runs the search by default, so there’s no index to build. If you keep your own tool registry, embedding index or per-user permission filter, you can declare the search tool with "execution": "client" instead. Your app then runs the lookup and returns complete function declarations in the function_result, which lets you add tools mid-conversation, such as when a user connects a new integration. The two modes can’t be combined in one request.
| Mode | Who finds the schemas | Extra round trips | Google’s pick for |
|---|---|---|---|
| Static loading | Nobody; all sent every request | None | Fewer than about 20 stable tools |
| Server-side retrieval | Gemini | None | Large but fixed tool libraries |
| Client-side retrieval | Your app | One per search | Proprietary registries, namespaces, per-user tool sets |
Remote MCP servers work too. Setting defer_loading: true on an mcp_server tool defers every tool that server exposes, so you can connect a large server without listing its tools one by one.
What it costs
Tool Retrieval has no separate charge. Tokens are billed at the standard rates for the model you call. Turning on the search tool adds a small fixed block of input tokens, and each deferred tool adds one line to the prompt.
Each search adds cost. Google’s docs say each server-side search runs another model call in the same request, carrying the full context plus the newly loaded schemas, and that call is billed at the input rate and reported in total_tool_use_tokens. A schema the model loads stays in context and is billed on every later call. The docs work two examples with round numbers, which they describe as illustrative rather than measured, with output tokens and caching left out. With 200 tools at 400 tokens each, a two-request conversation costs 164,000 input tokens with static loading and about 23,650 with Tool Retrieval. With 10 tools and a 30,000-token context, it’s 68,000 static and about 92,830 with Tool Retrieval.

Google also pitches prefix caching. Loaded schemas are added where the model loads them, not at the start of the prompt, so the system instructions, tool list and earlier turns stay an unchanged prefix. The docs add one side effect: deferring tools shrinks the prompt, and a prompt that only cleared the minimum cacheable size because of its full schemas can fall below it.
Limits to know about
Tool Retrieval is in preview, needs the v1beta API version and works only in the Interactions API. The older generateContent method doesn’t support it, and Google says the API surface may change before general availability. The docs list gemini-4-hydrogen, gemini-4-argon, gemini-3.8-flash and gemini-3.6-flash as supported, and the announcement says it works with Gemini 3.6 and later, including all Gemini 4 models. The docs’ list doesn’t name Gemini 4 Neon or Carbon. Setting defer_loading on an unsupported model returns a 400 error.
A few rules decide whether a request works. Retrieved tools stay in context for the rest of the conversation, with no way to unload them. The API doesn’t store tool declarations between requests, so you resend the full tools array every time, even when you chain turns with previous_interaction_id. If you manage history yourself, you must send the tool_search_call and tool_search_result steps back unchanged, signature included, or the API returns a 400 error. Tools you name in your system instructions shouldn’t be deferred, because the model can call one without searching and the next request fails. A query that matches no deferred tool produces a plain text response rather than a structured error.
Google’s announcement gives no measured token saving or accuracy figure, and the docs tell developers to test on their own traffic. Google hasn’t given a date for general availability.
Discover more from Sulat AI & Tech News
Subscribe to get the latest posts sent to your email.




