MCP vs. OpenAPI Tool Calling: Benchmarking Latency, Token Cost, and Reliability in Production AI Agents

Why the Protocol Choice Is Now a P99 Decision

Every agent platform converges on the same inflection point: the tool catalog outgrows the prompt. What worked with eight tools — inlining a JSON Schema for each into every model turn — collapses at eighty. Prefill latency climbs, prompt-cache hit rates fall, and the model starts selecting the wrong operationId because twelve of them read like near-duplicates.

Two integration models dominate the answer. OpenAPI tool calling compiles an HTTP contract into a function definition and lets the model emit arguments. MCP (Model Context Protocol) declares tools over a JSON-RPC 2.0 session, so the agent discovers capabilities at runtime rather than shipping them in the prompt.

The debate is usually framed as ideology. It isn't. It's a set of measurable engineering trade-offs across three axes: invocation latency, token cost per session, and failure-mode surface. Below is a reproducible way to measure all three, plus the hybrid architecture most production teams converge on.

A note on numbers: every figure here comes from a synthetic harness you can run yourself. Treat them as shape, not gospel — absolute values shift with model, region, TLS session reuse, and catalog shape. The ratios hold.

The Architectural Difference That Drives Every Benchmark Number

OpenAPI tool calling: schema-as-prompt

An OpenAPI 3.1 document is an HTTP contract. Wired into an LLM, a gateway transforms each operation into a tool definition and inlines it into the request. The model emits arguments, your runtime deserializes them, validates against the derived JSON Schema, and issues an HTTP call.

The critical property: the catalog lives in the context window. Every turn re-sends it unless the provider's prompt cache serves it.

MCP: protocol-as-runtime

Model Context Protocol is a JSON-RPC 2.0 protocol with a lifecycle: initialize → capability negotiation → tools/list → tools/call. Servers can be local processes over stdio or remote services over Streamable HTTP, with session state carried in the Mcp-Session-Id header.

The critical property: the catalog lives behind an interface. The client can page tools/list, filter by capability, or defer discovery entirely.

Benchmark Methodology You Can Reproduce

Three catalogs — 5, 50, and 200 tools — modeled on CRM, payments, and observability domains. Each tool averaged ~180 tokens of schema. Each run executed 20-turn sessions with a fixed synthetic task script so model-side variance was bounded.

Measured per invocation:

• Resolution latency — time to produce a validated argument object.
• Invoke latency — wire round trip for the underlying operation.
• Prompt tokens — billed input tokens including schema overhead.
• Outcome — success, validation error, transport error, model hallucination.

// bench/harness.ts — minimal, dependency-light measurement loop
import { performance } from 'node:perf_hooks';
import type { Client } from '@modelcontextprotocol/sdk/client/index.js';

export type Variant = 'openapi-http' | 'mcp-stdio' | 'mcp-http';

export interface ToolCall {
  name: string;
  args: Record<string, unknown>;
}

export interface Sample {
  variant: Variant;
  resolveMs: number;
  invokeMs: number;
  promptTokens: number;
  completionTokens: number;
  ok: boolean;
  errorCode?: string;
}

export async function timeInvocation(
  variant: Variant,
  client: Client,
  call: ToolCall,
  promptTokens: number,
): Promise<Sample> {
  const started = performance.now();
  let ok = false;
  let errorCode: string | undefined;

  try {
    const result = await client.callTool(
      { name: call.name, arguments: call.args },
      undefined,
      { timeout: 15_000 }, // fail fast; never let a hung tool block the turn
    );
    if ('isError' in result && result.isError) {
      errorCode = 'TOOL_ERROR';
    } else {
      ok = true;
    }
  } catch (err) {
    errorCode = classify(err); // normalize SDK + transport errors
  }

  const elapsed = performance.now() - started;
  return {
    variant,
    resolveMs: Number.NaN, // filled by the caller's trace span
    invokeMs: elapsed,
    promptTokens,
    completionTokens: 0,
    ok,
    errorCode,
  };
}

function classify(err: unknown): string {
  const msg = err instanceof Error ? err.message.toLowerCase() : String(err);
  if (msg.includes('timeout')) return 'TIMEOUT';
  if (msg.includes('401') || msg.includes('unauthorized')) return 'AUTH';
  if (msg.includes('session')) return 'SESSION_EXPIRED';
  if (msg.includes('schema') || msg.includes('validation')) return 'VALIDATION';
  return 'TRANSPORT';
}

Run it under autocannon or k6 to get tail percentiles; single-threaded for loops systematically under-report p99.

Latency: Where the Milliseconds Actually Go

The transport hop is not the interesting number. Median wire overhead differences are single-digit milliseconds. The interesting number is what the extra catalog does to the prefill phase.

Transport-level overhead (per invocation, TLS session reused)

stdio MCP wins because there is no network. Remote MCP is roughly equivalent to a direct HTTPS call — one round trip either way. Nobody chooses a protocol on these numbers.

Prefill is where the catalog size taxes you

A 200-tool catalog at 180 tokens per schema is ~36,000 input tokens before the conversation starts. Uncached, at a typical 3,000–8,000 tokens/second prefill rate for a hosted model, that is 5–12 seconds of added time-to-first-token. With prompt caching hit, it collapses to near zero — but cache TTLs expire, cache keys change when you add a tool, and cache writes cost more than reads.

The honest latency model is:

TTFT ≈ prefill(catalog + history) + queue
E2E  ≈ TTFT + n_turns × (decode + tool_latency + resolve_latency)

MCP's structural advantage is that tools/list supports pagination and filtering — you can put 30 relevant tools in the prompt instead of 200. That is a catalog-shaping win, and it's an order of magnitude larger than any transport delta.

Token Cost: The Schema Tax and How to Abolish It

Schemas are billed input tokens on every turn that re-reads them. Model it explicitly:

# cost/catalog_budget.py
from dataclasses import dataclass

@dataclass(frozen=True)
class CostModel:
    input_usd_per_mtok: float = 3.00
    cached_input_discount: float = 0.10  # cached reads ~10% of base price


def session_cost(
    tools: int,
    tokens_per_schema: int,
    turns: int,
    tokens_per_turn: int,
    cache_hit_rate: float,
    m: CostModel = CostModel(),
) -> dict[str, float]:
    catalog = tools * tokens_per_schema
    base_input = turns * tokens_per_turn

    uncached_turns = turns * (1 - cache_hit_rate)
    cached_turns = turns * cache_hit_rate

    billed = (
        uncached_turns * (catalog + tokens_per_turn)
        + cached_turns * (tokens_per_turn + catalog * m.cached_input_discount)
    )
    return {
        'catalog_tokens_per_turn': float(catalog),
        'session_input_tokens': float(billed),
        'session_cost_usd': billed / 1_000_000 * m.input_usd_per_mtok,
        'baseline_no_tools_usd': base_input / 1_000_000 * m.input_usd_per_mtok,
    }

print(session_cost(tools=200, tokens_per_schema=180, turns=20,
                   tokens_per_turn=1_500, cache_hit_rate=0.0))
# {'catalog_tokens_per_turn': 36000.0, 'session_input_tokens': 750000.0,
#  'session_cost_usd': 2.25, 'baseline_no_tools_usd': 0.09}

At a 0% cache hit rate, the catalog is 96% of your input bill and buys you nothing on turns where the model didn't need those tools. Even at a 90% cache hit rate, you're paying for 36k tokens of context that dilutes attention and increases mis-selection.

Three techniques that actually reduce the tax

1. Progressive disclosure. Expose a small set of meta-tools (search_tools, describe_tool) and load full schemas only for the shortlist. MCP makes this natural because discovery is a runtime call; with OpenAPI you're usually regenerating prompts.
2. Catalog partitioning. One MCP server per domain (billing, observability, CRM), mounted per-agent. A billing agent never sees the observability schemas.
3. Code execution over tool chaining. Anthropic's code-execution-with-MCP pattern lets the model write a short program that calls MCP tools inside a sandbox. Intermediate results stay in the sandbox instead of round-tripping through the context window — frequently a 90%+ token reduction on multi-step workflows.

Reliability: Failure Modes and Their Real Taxonomies

Success rate is the metric that quietly decides your on-call rotation.

The nasty one is the last row. An OpenAPI call that returns 200 OK with an empty list is indistinguishable from success unless you validate the response body against the schema. Add that validation at the client boundary, not in the model.

// lib/resilient-call.ts — bounded retry, jitter, idempotency, circuit breaker
export interface CallPolicy {
  maxAttempts: number;
  baseDelayMs: number;
  timeoutMs: number;
  breakerThreshold: number;
}

const DEFAULT: CallPolicy = { maxAttempts: 3, baseDelayMs: 120, timeoutMs: 10_000, breakerThreshold: 5 };

export class CircuitOpenError extends Error {
  constructor(tool: string) { super(`Circuit open for tool: ${tool}`); this.name = 'CircuitOpenError'; }
}

const failures = new Map<string, number>();

export async function callWithPolicy<T>(
  tool: string,
  fn: (signal: AbortSignal) => Promise<T>,
  policy: CallPolicy = DEFAULT,
  isRetryable: (e: unknown) => boolean = () => true,
): Promise<T> {
  if ((failures.get(tool) ?? 0) >= policy.breakerThreshold) throw new CircuitOpenError(tool);

  for (let attempt = 1; attempt <= policy.maxAttempts; attempt++) {
    const ac = new AbortController();
    const timer = setTimeout(() => ac.abort(), policy.timeoutMs);
    try {
      const result = await fn(ac.signal);
      failures.set(tool, 0);
      return result;
    } catch (err) {
      if (attempt === policy.maxAttempts || !isRetryable(err)) {
        failures.set(tool, (failures.get(tool) ?? 0) + 1);
        throw err;
      }
      // full jitter — prevents thundering herds across concurrent agents
      const delay = Math.random() * policy.baseDelayMs * 2 ** (attempt - 1);
      await new Promise((r) => setTimeout(r, delay));
    } finally {
      clearTimeout(timer);
    }
  }
  throw new Error('unreachable');
}

Idempotency is non-negotiable with retries

Retrying a createOrder call without an idempotency key turns a transient 504 into duplicate revenue. Emit a deterministic key derived from the agent's turn ID plus the tool name, and require the downstream API to honor it (Idempotency-Key header, per the IETF draft).

The Hybrid Architecture That Wins in Production

The benchmark conclusion is not "MCP beats OpenAPI." It's that they fail at different things.

Ship MCP when: tools are internal, evolve weekly, carry session state, or number in the hundreds. The runtime-discovery model lets you add a tool without a prompt redeploy.

Ship OpenAPI when: the API is external and stable, has a published spec you don't control, or is consumed by non-agent clients too. You get no benefit from wrapping a mature public API in a protocol hop.

The winning pattern is a single tool registry in front of both, normalizing into one canonical schema:

Agent runtime
   └─ Tool Registry (canonical JSON Schema, scoped per agent)
        ├─ MCP client ──► billing-mcp (stdio)   ──► internal services
        ├─ MCP client ──► observability-mcp (HTTP) ──► Prometheus/Grafana
        └─ OpenAPI adapter ──► Stripe / GitHub / Twilio (public contracts)

You can bridge the other direction too — expose mature OpenAPI operations through an MCP server so the agent sees one interface:

// bridge/openapi-to-mcp.ts — one MCP tool per OpenAPI operation
import { Server } from '@modelcontextprotocol/sdk/server/index.js';
import { ListToolsRequestSchema, CallToolRequestSchema } from '@modelcontextprotocol/sdk/types.js';
import type { OpenAPIV3_1 } from 'openapi-types';

interface Operation {
  operationId: string;
  description: string;
  inputSchema: Record<string, unknown>;
  method: string;
  path: string;
}

function compileOperations(doc: OpenAPIV3_1.Document): Operation[] {
  const ops: Operation[] = [];
  for (const [path, item] of Object.entries(doc.paths ?? {})) {
    for (const method of ['get', 'post', 'put', 'patch', 'delete'] as const) {
      const op = item?.[method];
      if (!op?.operationId) continue; // skip unnamed ops: the model can't call what it can't name
      ops.push({
        operationId: op.operationId,
        description: op.summary ?? op.description ?? op.operationId,
        inputSchema: deriveSchema(op, doc),
        method: method.toUpperCase(),
        path,
      });
    }
  }
  return ops;
}

export function createBridge(opts: {
  doc: OpenAPIV3_1.Document;
  baseUrl: string;
  authHeader: () => Promise<string>;
}): Server {
  const ops = compileOperations(opts.doc);
  const byId = new Map(ops.map((o) => [o.operationId, o]));
  const server = new Server({ name: 'openapi-bridge', version: '1.0.0' }, { capabilities: { tools: {} } });

  server.setRequestHandler(ListToolsRequestSchema, async () => ({
    tools: ops.map((o) => ({ name: o.operationId, description: o.description, inputSchema: o.inputSchema })),
  }));

  server.setRequestHandler(CallToolRequestSchema, async (req) => {
    const op = byId.get(req.params.name);
    if (!op) {
      return { isError: true, content: [{ type: 'text', text: `Unknown tool: ${req.params.name}` }] };
    }
    try {
      const res = await fetch(new URL(op.path, opts.baseUrl), {
        method: op.method,
        headers: { 'content-type': 'application/json', authorization: await opts.authHeader() },
        body: op.method === 'GET' ? undefined : JSON.stringify(req.params.arguments ?? {}),
        signal: AbortSignal.timeout(15_000),
      });
      const body = await res.text();
      // Treat non-2xx as a tool error, not a model error — let the agent self-correct.
      return { isError: !res.ok, content: [{ type: 'text', text: body.slice(0, 8_000) }] };
    } catch (err) {
      return { isError: true, content: [{ type: 'text', text: `Transport error: ${(err as Error).message}` }] };
    }
  });

  return server;
}

Note the body.slice(0, 8_000) — unbounded tool responses are the single most common cause of context blowout in production agents. Truncate at the boundary and let the model ask for a narrower query.

Production Readiness Checklist

• Bound every schema. Enforce a token budget per tool (e.g., ≤ 250 tokens) in CI; fail the build if the catalog exceeds it.
• Scope per agent. Mount only the MCP servers a given agent needs; never share a global catalog.
• Validate responses, not just status codes. JSON Schema check the tool output before it enters context.
• Cap tool output size. Truncate with a continuation hint; never inject a 500 KB JSON blob.
• Timeout every call. 10–15 s default, tighter for read-only lookups.
• Idempotency keys on all mutating tools. Retries without them are data corruption.
• Circuit breakers per tool, not per agent — one flaky endpoint shouldn't kill the session.
• Structured tracing. Span per tools/call and per HTTP request, with session_id and turn_id as attributes.
• Cache-aware prompt design. Put the stable catalog prefix first so provider caching hits.
• Version your MCP servers. Breaking a tools/list schema silently is worse than a 500.

Pitfalls to Avoid

• Benchmarking transport instead of prefill. A 5 ms protocol delta is noise next to 36k tokens of schema. Measure TTFT with realistic catalogs.
• Naming tools like your API. repos.listForOrg is worse than list_repositories. Verb-first, unambiguous names measurably reduce mis-selection.
• Trusting `tools/list` ordering. Paginate and deduplicate; some servers return unstable ordering across sessions.
• Ignoring MCP session expiry. A Mcp-Session-Id that dies mid-workflow will look like a model failure. Reconnect and replay.
• Treating `isError` as fatal. Returning structured errors as tool content ({"error":"sku_not_found"}) lets the model retry intelligently; a thrown exception just aborts the turn.
• Wrapping a public API in MCP for no reason. You add a hop, an auth layer, and a session model to something that already worked.

Sources & Technical References

• Model Context Protocol Specification (2025-06-18) — lifecycle, tools, transports: <https://modelcontextprotocol.io/specification/2025-06-18>
• MCP Architecture & Transports (stdio, Streamable HTTP) — <https://modelcontextprotocol.io/docs/learn/architecture>
• JSON-RPC 2.0 Specification — the wire protocol underneath MCP: <https://www.jsonrpc.org/specification>
• OpenAPI Specification 3.1.0 — <https://spec.openapis.org/oas/v3.1.0.html>
• Anthropic: Introducing the Model Context Protocol — <https://www.anthropic.com/news/model-context-protocol>
• Anthropic Engineering: Code Execution with MCP — token-reduction pattern for tool chaining: <https://www.anthropic.com/engineering/code-execution-with-mcp>
• Anthropic Engineering: Writing Effective Tools for Agents — tool naming and description guidance: <https://www.anthropic.com/engineering/writing-tools-for-agents>
• OpenAI Function Calling Guide — tool schema semantics and parallel calls: <https://platform.openai.com/docs/guides/function-calling>
• Anthropic Tool Use Documentation — tool choice, tool_result handling: <https://docs.anthropic.com/en/docs/build-with-claude/tool-use>
• RFC 9110 — HTTP Semantics (idempotency, status code semantics): <https://www.rfc-editor.org/rfc/rfc9110.html>
• IETF draft: The Idempotency-Key HTTP Header Field — <https://datatracker.ietf.org/doc/draft-ietf-httpapi-idempotency-key-header/>
• RFC 8446 — TLS 1.3 (session resumption and its effect on RTT measurement): <https://www.rfc-editor.org/rfc/rfc8446.html>
• MDN: Server-Sent Events — the framing used by MCP Streamable HTTP: <https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events>
• OAuth 2.1 Authorization Framework (draft) — MCP's authorization baseline: <https://datatracker.ietf.org/doc/draft-ietf-oauth-v2-1/>