Amir Pournasserian

Search

Should you put an AI agent behind an MCP tool?

· Part 33 of 35

Should you put an AI agent behind an MCP tool?

Sooner or later someone suggests it: wrap your retrieval-augmented generation (RAG) pipeline, research agent or “smart search” behind one Model Context Protocol (MCP) tool called search or ask. The calling model sends a question, and the inner agent plans, retrieves, reasons and answers. It sounds helpful. My answer is more cautious. Here’s why, when it’s the right call, and how to build one well.

This is part 33 of my series on what an MCP server does under the 2026-07-28 specification, after the design patterns and anti-patterns of part 32. The facts are as I read them in October 2026.

In brief

  1. My answer: an agent behind a tool is a legitimate pattern, but it shouldn’t be your default, and for search it’s usually the wrong first choice.
  2. The calling model is already an agent. A second one behind search doubles the reasoning cost, hands back a summary in place of evidence, and adds opacity, timeout risk and a second injection surface.
  3. An inner agent earns its place when the reasoning needs knowledge or access the calling model can’t have, when the work is long research, or when you need consistent answers across hosts or control over model choice, cost and guardrails.
  4. If you build one, offer it as a second tier beside a fast search and fetch, return its sources in structuredContent, keep it read-only, and have it call your backends only with the user’s authority.
  5. My rule of thumb: expose your data as tools and your expertise as prompts or skills.

Two shapes for the same question

In shape A, your server exposes retrieval tools (the search plus fetch pattern of part 32), and the outer model plans, reformulates, reads and cites. In shape B, the outer model hands the question to an ask or research tool. Behind it, an inner agent with its own large language model (LLM), loop and prompts works against the index and your internal systems.

A: Retrieval tools: Outer model plans, reformulates, reads, cites, search tool hybrid search + rerank, fetch tool full documents, Index; B: Agent behind a tool: Outer model delegates the question, ask / research tool, Inner agent own LLM, own loop, own prompts, Index + internal systems; Outer model plans, reformulates, reads, cites → search tool hybrid search + rerank; search tool hybrid search + rerank → Index; Outer model plans, reformulates, reads, cites → fetch tool full documents; fetch tool full documents → Index; Outer model delegates the question → ask / research tool; ask / research tool → Inner agent own LLM, own loop, own prompts; Inner agent own LLM, own loop, own prompts → Index + internal systemsA: Retrieval toolsB: Agent behind a toolOuter modelplans, reformulates,reads, citessearch toolhybrid search+ rerankfetch toolfull documentsIndexOuter modeldelegatesthe questionask / researchtoolInner agentown LLM, own loop,own promptsIndex + internalsystems

Shape B shouldn’t be your default, though it has its place in the cases below. That’s my opinion, not the specification’s.

Why it’s often a mistake

The outer model is already an agent: it can plan, reformulate queries, read results and synthesize. Putting a second agent behind search costs you five things:

  • Double reasoning cost. Two models think about the same question, and you pay for it in latency and in tokens.
  • Lost evidence. The outer model receives a summary instead of evidence, so it can’t check sources, judge relevance or cite precisely. The inner agent’s mistakes arrive already presented as confirmed answers.
  • Opacity. Neither the user nor the outer model can see why a result came back, and debugging means tracing two reasoning loops.
  • Timeouts. A multi-step inner loop easily runs past host timeouts, and on Streamable HTTP a dropped stream cancels the work, as part 16 explains.
  • A doubled injection surface. The inner agent reads untrusted content and passes its interpretation upward, which opens a second path for the prompt injection through outputs that part 21’s threat model lists.

When an inner agent is the right call

None of that makes the pattern wrong. I’d reach for it in these situations:

SituationWhy an inner agent helps
Retrieval needs proprietary query planning, such as joining several internal systems with domain rulesThe outer model can’t see your schema or business logic
Results are too large for the caller’s contextThe inner agent can compress many sources into a cited brief
You need consistent answers across ChatGPT, Claude, Copilot and othersAnswer quality stops depending on which outer model is calling
The work is long-running research, minutes rather than secondsIt fits the Tasks extension
You need control on the server side over model choice, cost, guardrails and evaluationYou own the prompt, the model and the evals

Put as a decision, it comes down to two questions about the reasoning and one about time:

New 'smart' capability → Can the calling model do the reasoning if given good data?; Can the calling model do the reasoning if given good data? → Expose data as tools search + fetch, structured results: yes; Can the calling model do the reasoning if given good data? → Does reasoning need knowledge or access the caller cannot have?: no; Does reasoning need knowledge or access the caller cannot have? → Expose data as tools search + fetch, structured results: no; Does reasoning need knowledge or access the caller cannot have? → Will it run longer than about a minute?: yes; Will it run longer than about a minute? → Agentic tool, synchronous answer + citations in structuredContent: no; Agentic tool, synchronous answer + citations in structuredContent → Offer both tiers: fast search and optional research; Will it run longer than about a minute? → Agentic tool returning a Task: yes; Agentic tool returning a Task → Offer both tiers: fast search and optional researchNew 'smart' capabilityCan the calling modeldo the reasoningif given good data?Expose data as toolssearch + fetch,structured resultsDoes reasoning needknowledge or accessthe caller cannot have?Will it run longerthan about a minute?Agentic tool, synchronousanswer + citationsin structuredContentAgentic tool returning a TaskOffer both tiers:fast search andoptional researchyesnonoyesnoyes

My rule of thumb: expose your data as tools and your expertise as prompts or skills, and put an agent behind a tool only when the reasoning itself depends on knowledge or access the calling model can’t have. Part 12 walks through choosing among the primitives.

The minute in the last question is my rule of thumb too, not the specification’s. The Tasks rule below narrows it to 30 to 60 seconds and says why.

If you do build one: the design rules

Here are my eleven design rules, in three groups:

The shape of the tool

  • Offer two tiers. A fast, deterministic search (and fetch), plus an optional agentic research or ask. Let the caller choose.
  • Return evidence, not just prose. Put the synthesized answer in content and the sources, quotes and confidence in structuredContent, the two halves of a tool result from part 8. Never strip citations.
  • Say it in the description. For example: “Returns a synthesized answer with citations; typically 5 to 15 s.” Then the outer model can choose sensibly between search and research.
  • Consider agent-to-agent protocols when the inner agent is really an autonomous collaborator with its own multi-turn conversation. MCP tools suit bounded, mostly single-shot functions.

Running it within bounds

  • Call your model directly. Sampling is deprecated (see part 22), so the server calls its own LLM provider and pays for it. Budget tokens per call, and cap loop iterations and wall-clock time.
  • Don’t guess when the question is unclear. Ask a clarifying question with elicitation through Multi Round-Trip Requests (MRTR, part 13) instead of letting the inner agent assume.
  • Use Tasks for long runs. My rule of thumb: anything that might take longer than about 30 to 60 seconds runs as a task. No specification sets that number. The 60 matches the default request timeout of the official TypeScript software development kit (SDK), and the Tasks overview sets the bar lower: it warns that many clients and transport intermediaries impose timeouts that make blocking impractical beyond a few seconds.
  • Trace end to end. Continue the caller’s traceparent into the inner agent’s spans (part 20), so one trace runs from the outer model through MCP and the inner LLM to the backends.

Keeping it safe

  • Keep the inner agent read-only. No writes, sends or deletes. Side effects belong in explicit tools that the host can gate behind user approval.
  • Use the user’s authority, not yours. Downstream calls use on-behalf-of tokens scoped to the caller (part 19), never a broad service principal. Otherwise you’ve built a confused deputy.
  • Treat retrieved content as data. The inner prompt should say so explicitly, and outputs should quote sources rather than relay instructions found in documents.

Method and caveats

  • Built from the “big question” section of my guide’s design-patterns chapter, written against the 2026-07-28 specification.
  • The verdict, the decision flow, the design rules and the rule of thumb are my opinion and practice, not requirements of the specification.
  • Where the Tasks overview and the official TypeScript SDK are more precise than my notes (when work belongs in a task), this article follows them, as checked on 8 October 2026, and gives 30 to 60 seconds as my rule of thumb.