Gemini Tool Retrieval and Mac Computer Use: Google Ships the Plumbing
Google spent Thursday on agent plumbing rather than a new model, and both pieces are worth knowing. At 18:00 UTC it put Tool Retrieval into preview in the Gemini Interactions API. At 17:00 UTC it gave the Gemini app for Mac computer use. One is for builders whose agents have too many tools, the other for users who want the agent to click through their desktop. Posts: https://blog.google/innovation-and-ai/technology/developers-tools/gemini-api-tool-retrieval/ and https://blog.google/innovation-and-ai/products/gemini-app/computer-use-mac/
Tool Retrieval is simple to describe. Add a tool_search tool, set defer_loading to true on the tools you want held back, and the model sees only each tool's name and description. When it needs one, it asks by name, the API loads the full schema, and the model makes the call, all inside the same request, with every step visible in interaction.steps. No index to host; search runs server side. Deferred schemas stay out of the start of the prompt, so the cached prefix stays stable turn to turn. And it works with remote MCP servers: flag an mcp_server tool as deferred and every tool on that server loads on demand. If you keep your own registry or per-user permission filter, there is a client-side mode that costs one extra round trip per search.
What the post does not have: any number. No token savings, no accuracy change at 50 or 500 tools. Google says the feature is designed to reduce input tokens and keep selection accurate, which is the right goal, but builders should measure before trusting it. It is also not a new idea; deferred tool loading already exists in other agent platforms, including Claude Code's tool search. What matters is that a second big platform now treats bloated tool lists as a platform problem, not a prompt engineering problem.
Computer use on Mac runs in an isolated background window with a live preview in the chat, so your own screen and cursor stay free. You choose which apps Gemini can access, watch each step, pause anytime, and approve key actions before they happen. Requirements are narrow: Apple Silicon, macOS Sequoia 15.0 or later, personal accounts 18 and over, Google AI Plus, Pro and Ultra, and TechCrunch reports US English only. Google is late here compared with desktop agents from OpenAI and Anthropic, and the post gives no reliability numbers.
The one independent read on Gemini 4 Neon so far is a modeled projection from micjm's Agent Duel leaderboard (HN 249), putting Neon at Elo 1,066 against 1,078 for Opus 5.5 and clocking 127 tokens per second on OpenRouter. It is a projection, not a measurement. Taken together, Google's week looks like a lab building the whole stack around an orchestrator model: cheaper context, an app allowlist, an approval gate. The model is the headline; the plumbing is what decides whether agents built on it hold up.
← Back to all articles
Tool Retrieval is simple to describe. Add a tool_search tool, set defer_loading to true on the tools you want held back, and the model sees only each tool's name and description. When it needs one, it asks by name, the API loads the full schema, and the model makes the call, all inside the same request, with every step visible in interaction.steps. No index to host; search runs server side. Deferred schemas stay out of the start of the prompt, so the cached prefix stays stable turn to turn. And it works with remote MCP servers: flag an mcp_server tool as deferred and every tool on that server loads on demand. If you keep your own registry or per-user permission filter, there is a client-side mode that costs one extra round trip per search.
What the post does not have: any number. No token savings, no accuracy change at 50 or 500 tools. Google says the feature is designed to reduce input tokens and keep selection accurate, which is the right goal, but builders should measure before trusting it. It is also not a new idea; deferred tool loading already exists in other agent platforms, including Claude Code's tool search. What matters is that a second big platform now treats bloated tool lists as a platform problem, not a prompt engineering problem.
Computer use on Mac runs in an isolated background window with a live preview in the chat, so your own screen and cursor stay free. You choose which apps Gemini can access, watch each step, pause anytime, and approve key actions before they happen. Requirements are narrow: Apple Silicon, macOS Sequoia 15.0 or later, personal accounts 18 and over, Google AI Plus, Pro and Ultra, and TechCrunch reports US English only. Google is late here compared with desktop agents from OpenAI and Anthropic, and the post gives no reliability numbers.
The one independent read on Gemini 4 Neon so far is a modeled projection from micjm's Agent Duel leaderboard (HN 249), putting Neon at Elo 1,066 against 1,078 for Opus 5.5 and clocking 127 tokens per second on OpenRouter. It is a projection, not a measurement. Taken together, Google's week looks like a lab building the whole stack around an orchestrator model: cheaper context, an app allowlist, an approval gate. The model is the headline; the plumbing is what decides whether agents built on it hold up.
Comments