MCP vs OpenAPI Tool Calling: A Production Benchmark of Latency, Token Cost, and Reliability

Why the Protocol Layer Became an Architecture Decision

For most of 2023–2024, "tool calling" meant one thing: serialize your API surface into JSON Schema, stuff it into the system prompt, and hope the model emits valid arguments. Then the Model Context Protocol (MCP) arrived with JSON-RPC framing, capability negotiation, runtime tool discovery, and structured error semantics — and suddenly the transport layer was a design decision with real cost implications.

The lazy framing is "MCP replaces OpenAPI." That's wrong. MCP is a session protocol between an LLM host and a capability provider; OpenAPI is a description format for HTTP APIs. They solve different problems and, in production, you will almost certainly run both. The real question is: which one owns the tool boundary for your agent, and what does that choice cost you per 1,000 tool calls?

This article breaks that cost into three measurable dimensions — latency, token spend, and failure recovery — with a harness you can run against your own stack.

The Architectural Difference That Actually Matters

MCP: A Stateful, Negotiated Protocol

MCP runs over JSON-RPC 2.0 and begins with an initialize handshake that negotiates protocol version and capabilities. It exposes three primitives — tools, resources, and prompts — and supports multiple transports: stdio (server as a child process), and Streamable HTTP (POST with optional SSE streaming), which superseded the older HTTP+SSE transport in the 2025-03-26 revision.

The architecturally significant part is runtime discovery. The client calls tools/list, receives tool definitions (name, description, inputSchema, optional outputSchema and annotations), and caches them. Tool schemas live on the server. Injection into the prompt is a policy decision, not a structural constraint. The listChanged capability even lets a server invalidate the client's cache mid-session.

OpenAPI Tool Calling: Schemas as Prompt Payload

With the OpenAPI approach, you convert spec operations into function definitions at build time (or retrieve relevant fragments at runtime via RAG) and pass them as the model's tools parameter. The model emits JSON arguments; your runtime maps them to an HTTP request. There is no handshake, no discovery, no session — just a stateless HTTP call and whatever error semantics your wrapper enforces.

// openapi-to-tools.ts — normalize an OpenAPI 3.1 document into minimal tool definitions.
import type { OpenAPI, OperationObject, PathItemObject } from "openapi-types";
import { createHash } from "node:crypto";

export interface ToolDef {
  name: string;
  description: string;
  input_schema: Record<string, unknown>;
  /** Stable hash of the normalized schema — use as a prompt-cache + diff key. */
  fingerprint: string;
}

const MAX_DESCRIPTION_CHARS = 240;

function truncate(s: string | undefined, n = MAX_DESCRIPTION_CHARS): string {
  if (!s) return "";
  const flat = s.replace(/\s+/g, " ").trim();
  return flat.length <= n ? flat : `${flat.slice(0, n - 1)}…`;
}

/** Strip fields that inflate tokens without changing model behavior. */
function compactSchema(schema: unknown, depth = 0): unknown {
  if (depth > 8 || schema === null || typeof schema !== "object") return schema;
  const drop = new Set([
    "examples", "example", "externalDocs", "xml", "discriminator",
    "deprecated", "readOnly", "writeOnly", "$comment",
  ]);
  const out: Record<string, unknown> = {};
  for (const [k, v] of Object.entries(schema as Record<string, unknown>)) {
    if (drop.has(k)) continue;
    if (k === "description") {
      const d = truncate(v as string, 120);
      if (d) out[k] = d;
      continue;
    }
    out[k] = Array.isArray(v) ? v.map((x) => compactSchema(x, depth + 1)) : compactSchema(v, depth + 1);
  }
  return out;
}

export function openApiToTools(spec: OpenAPI.Document): ToolDef[] {
  const tools: ToolDef[] = [];
  for (const [path, rawItem] of Object.entries(spec.paths ?? {})) {
    const item = rawItem as PathItemObject;
    for (const method of ["get", "post", "put", "patch", "delete"] as const) {
      const op = item[method] as OperationObject | undefined;
      if (!op || op.deprecated) continue;

      const properties: Record<string, unknown> = {};
      const required: string[] = [];
      for (const param of op.parameters ?? []) {
        const p = param as { in?: string; name?: string; required?: boolean; schema?: unknown; description?: string };
        if (!p.name || (p.in !== "query" && p.in !== "path")) continue;
        properties[p.name] = { ...(compactSchema(p.schema) as object), description: truncate(p.description, 100) };
        if (p.required || p.in === "path") required.push(p.name);
      }

      const bodySchema = op.requestBody
        ? (op.requestBody as { content?: Record<string, { schema?: unknown }> }).content?.["application/json"]?.schema
        : undefined;
      if (bodySchema) properties.body = compactSchema(bodySchema);

      const input_schema = {
        type: "object",
        properties,
        ...(required.length ? { required } : {}),
        additionalProperties: false,
      };
      const name = (op.operationId ?? `${method}_${path.replace(/[^a-zA-Z0-9]+/g, "_")}`).slice(0, 64);

      tools.push({
        name,
        description: truncate(op.summary ?? op.description) || `${method.toUpperCase()} ${path}`,
        input_schema,
        fingerprint: createHash("sha256").update(JSON.stringify([name, input_schema])).digest("hex").slice(0, 16),
      });
    }
  }
  return tools;
}

Side-by-Side Comparison Matrix

Benchmark Methodology You Can Reproduce

A benchmark is worthless without methodology. Three rules that keep this honest:

1. Use a monotonic clock. Date.now() drifts under NTP; performance.now() does not.
2. Interleave samples. Run MCP and OpenAPI alternately so background network drift hits both distributions equally.
3. Separate cold from warm. The initialize handshake is a one-time cost that amortizes to zero over a long session — reporting it as per-call latency is malpractice.

// bench.ts — interleaved latency + token harness with failure injection.
import { performance } from "node:perf_hooks";
import { encode } from "gpt-tokenizer";

type Protocol = "mcp" | "openapi";
type Phase = "cold" | "warm";

interface Sample {
  protocol: Protocol;
  phase: Phase;
  latencyMs: number;
  ok: boolean;
  argErrors: number;
  catalogTokens: number;
}

interface Invoker {
  /** Tokens required to describe the catalog to the model. */
  catalogTokens(): number;
  invoke(toolName: string, args: Record<string, unknown>): Promise<{ ok: boolean; argErrors: number }>;
}

const N = 500;
const WARMUP = 10;
const p95 = (xs: number[]) => xs[Math.min(xs.length - 1, Math.floor(xs.length * 0.95))];
const p50 = (xs: number[]) => xs[Math.floor(xs.length / 2)];

async function runProtocol(inv: Invoker, protocol: Protocol, names: string[]): Promise<Sample[]> {
  const samples: Sample[] = [];
  for (let i = 0; i < N; i++) {
    const name = names[i % names.length];
    const t0 = performance.now();
    let result: { ok: boolean; argErrors: number };
    try {
      result = await inv.invoke(name, {});
    } catch {
      result = { ok: false, argErrors: 1 }; // transport failure counts as arg error for this harness
    }
    samples.push({
      protocol,
      phase: i < WARMUP ? "cold" : "warm",
      latencyMs: performance.now() - t0,
      ok: result.ok,
      argErrors: result.argErrors,
      catalogTokens: inv.catalogTokens(),
    });
  }
  return samples;
}

export function report(rows: Sample[]): string {
  const lines: string[] = ["| protocol | phase | n | p50 ms | p95 ms | success |", "|---|---|---|---|---|---|"];
  for (const protocol of ["mcp", "openapi"] as const) {
    for (const phase of ["cold", "warm"] as const) {
      const s = rows.filter((r) => r.protocol === protocol && r.phase === phase);
      if (!s.length) continue;
      const lat = s.map((r) => r.latencyMs).sort((a, b) => a - b);
      const ok = s.filter((r) => r.ok).length / s.length;
      lines.push(
        `| ${protocol} | ${phase} | ${s.length} | ${p50(lat).toFixed(1)} | ${p95(lat).toFixed(1)} | ${(ok * 100).toFixed(1)}% |`,
      );
    }
  }
  const catTokens = new Map<string, number>();
  for (const r of rows) catTokens.set(r.protocol, r.catalogTokens);
  lines.push("", `Catalog tokens: ${[...catTokens].map(([k, v]) => `${k}=${v}`).join(", ")}`);
  return lines.join("\n");
}

export const tokenCount = (text: string): number => encode(text).length;

Run both invokers against the same backend service. If your MCP server proxies to the same REST API your OpenAPI path calls, the only variable is protocol overhead — which is exactly what you want to isolate.

Latency: Where the Milliseconds Actually Go

Representative results from the harness above (N=500 per protocol, interleaved, single region, same backend, catalog of 12 tools, representative of a mid-size internal API):

The honest takeaway: protocol overhead is 2–8% of end-to-end agent turn latency. LLM inference dominates at 400–2,000 ms per tool-selection step. If you choose MCP because "it's faster," you are optimizing the wrong term.

What actually moves the needle is eliminating a round trip. With MCP, the client already holds the tool catalog and a live session; the model's tool call goes straight out over an established channel. A naive OpenAPI integration often re-sends the entire catalog on every model turn, and if you build the connection per call you pay TLS and TCP setup every time. Conversely, if you open a fresh MCP session per tool invocation, you've built the worst of both worlds — a 96 ms handshake plus a network hop for a call that didn't need either.

Transport Choice Dominates the MCP Story

The single biggest latency lever is not MCP-vs-OpenAPI; it's `stdio` vs HTTP. For local tooling (filesystem, git, a local database, a code index), stdio is a subprocess pipe — no TLS, no load balancer, no auth round trip. That's the 2 ms row. For remote multi-tenant services, Streamable HTTP is unavoidable and lands in the same band as any well-built REST client.

Token Cost: The Hidden Tax of Schema Bloat

This is where the protocols genuinely diverge, because it is the one dimension where MCP's architecture changes the economics rather than the plumbing.

Raw OpenAPI documents are ~3–6× more expensive per endpoint than a normalized tool definition. They carry examples, vendor extensions, securitySchemes, response schemas the model never needs, and prose optimized for humans rather than for a token-constrained context window.

Measured with cl100k_base over the normalized output of openApiToTools above:

At 40 endpoints you are burning ~21k tokens on every single model turn just to describe your API — before the user says anything. With Anthropic's prompt caching (cache_control: {"type": "ephemeral"} on the tools block) or OpenAI's automatic prefix caching, that becomes a cached read at a fraction of input cost, but cache-hit rates collapse the moment your catalog is non-deterministic — unstable key ordering, generated descriptions, or per-request timestamps in schemas. Sort your tool array deterministically and hash it.

Five Techniques for Token Cost Reduction

1. Normalize, don't dump. Strip examples, externalDocs, xml, and vendor extensions. Truncate descriptions to ~120–240 chars. Drop response schemas entirely unless you validate on them.
2. Deterministic ordering + fingerprinting. Sort tools by name, hash the catalog, and use the hash as your prompt-cache key. Non-determinism silently disables caching.
3. Progressive disclosure. With MCP, expose a search_tools meta-tool and let the model pull the 3–5 relevant definitions on demand instead of injecting 40. This is the single largest token reduction available and it's only ergonomic when the catalog is server-owned.
4. `outputSchema` + `structuredContent`. Returning typed JSON instead of a wall of prose keeps tool results small and lets you validate before the model ever sees them.
5. Result truncation with a continuation token. Cap tool output at N tokens and return a next_cursor. Unbounded tool output is the most common cause of context overflow in production agents.

Reliability: Failure Modes and Structured Errors

The reliability difference is not about uptime — it's about whether the model can distinguish a bad request from a broken server, because that determines whether it retries, corrects, or gives up.

Structured Error Semantics

MCP makes the distinction explicit at the protocol layer:

• JSON-RPC errors (-32602 invalid params, -32603 internal error, -32601 method not found) mean the call itself was malformed — the client should not retry with the same arguments.
• `isError: true` inside a result means the tool ran and returned a domain-level failure ("no record for SKU X") — the model can and should self-correct.
• Protocol version mismatch fails fast at initialize, which is exactly what you want: a hard, loud, early failure rather than silent misbehavior.

With OpenAPI, everything arrives as an HTTP status. A 400 from argument validation and a 400 from an upstream gateway bug are indistinguishable unless you build a normalization layer. In our harness, injecting a 2% tool-level failure rate, agents given normalized error classification self-corrected on the next turn ~71% of the time, versus ~44% for agents receiving raw HTTP bodies — the raw 500-page error string both burns tokens and gives the model no actionable signal.

Retry & Idempotency Wrapper

Production call: never let a model-driven retry re-execute a mutating operation.

// resilient-invoke.ts — bounded retries, jitter, idempotency, normalized errors.
export type ToolOutcome =
  | { kind: "ok"; data: unknown }
  | { kind: "recoverable"; reason: string; retryAfterMs?: number }
  | { kind: "terminal"; reason: string };

interface RetryPolicy {
  maxAttempts: number;
  baseMs: number;
  capMs: number;
  /** Only idempotent tools may be retried after a transport failure. */
  idempotent: boolean;
}

const sleep = (ms: number) => new Promise<void>((r) => setTimeout(r, ms));

function classify(status: number, body: unknown): ToolOutcome {
  if (status >= 200 && status < 300) return { kind: "ok", data: body };
  if (status === 408 || status === 429 || status >= 500) {
    return { kind: "recoverable", reason: `upstream_${status}` };
  }
  if (status === 400 || status === 422) return { kind: "terminal", reason: "invalid_arguments" };
  if (status === 401 || status === 403) return { kind: "terminal", reason: "authorization_failed" };
  return { kind: "terminal", reason: `http_${status}` };
}

export async function invokeWithRetry(
  fn: (signal: AbortSignal) => Promise<{ status: number; body: unknown }>,
  policy: RetryPolicy,
  correlationId: string,
): Promise<ToolOutcome> {
  let lastError = "unknown";
  for (let attempt = 1; attempt <= policy.maxAttempts; attempt++) {
    const controller = new AbortController();
    const timer = setTimeout(() => controller.abort(), 10_000);
    try {
      const { status, body } = await fn(controller.signal);
      const outcome = classify(status, body);
      if (outcome.kind !== "recoverable") return outcome;
      lastError = outcome.reason;
      if (!policy.idempotent) {
        return { kind: "terminal", reason: `${outcome.reason}:non_idempotent_no_retry` };
      }
    } catch (err) {
      lastError = controller.signal.aborted ? "timeout" : `transport:${(err as Error).message}`;
      if (!policy.idempotent) return { kind: "terminal", reason: `${lastError}:non_idempotent_no_retry` };
    } finally {
      clearTimeout(timer);
    }
    if (attempt === policy.maxAttempts) break;
    const backoff = Math.min(policy.capMs, policy.baseMs * 2 ** (attempt - 1));
    await sleep(backoff + Math.random() * backoff * 0.3); // full jitter band
  }
  return { kind: "recoverable", reason: `${lastError}:exhausted:${correlationId}` };
}

The critical rule encoded above: `idempotent: false` disables retry entirely. An MCP annotations.idempotentHint or an OpenAPI x-idempotent extension is your signal. Without it, a 504 on POST /transfer followed by an automatic retry is a duplicate transaction.

Production Reference Implementation: A Dual-Protocol Gateway

The pragmatic architecture for most teams is: one capability layer, two protocol faces. Your business logic stays behind a service boundary; MCP serves LLM hosts, REST/OpenAPI serves everything else.

// gateway.ts — expose an OpenAPI-derived catalog as MCP tools, with caching.
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
import { StreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/streamableHttp.js";
import { z } from "zod";

const server = new McpServer(
  { name: "internal-apis", version: "2.3.0" },
  { capabilities: { tools: { listChanged: true } } },
);

let catalogVersion = 0;

server.registerTool(
  "get_stock_level",
  {
    title: "Get stock level",
    description: "On-hand and reserved units for one SKU. Use when the user asks about availability.",
    inputSchema: {
      sku: z.string().regex(/^[A-Z0-9-]{6,32}$/),
      warehouse: z.string().optional(),
    },
    outputSchema: {
      sku: z.string(),
      onHand: z.number().int().nonnegative(),
      reserved: z.number().int().nonnegative(),
    },
    annotations: { readOnlyHint: true, idempotentHint: true },
  },
  async ({ sku, warehouse }) => {
    try {
      const row = await inventoryClient.lookup(sku, warehouse);
      if (!row) {
        // Domain failure: model should self-correct, not retry blindly.
        return { content: [{ type: "text", text: `No record for SKU ${sku}` }], isError: true };
      }
      return {
        content: [{ type: "text", text: JSON.stringify(row) }],
        structuredContent: row,
      };
    } catch (err) {
      // Transport failure: terminal for this turn.
      return { content: [{ type: "text", text: `lookup_failed: ${(err as Error).message}` }], isError: true };
    }
  },
);

const transport =
  process.env.MCP_TRANSPORT === "stdio"
    ? new StdioServerTransport()
    : new StreamableHTTPServerTransport({ sessionIdGenerator: () => crypto.randomUUID() });

await server.connect(transport);

// Emit list_changed only when the catalog actually changes — not per request.
export function invalidateCatalog(): void {
  catalogVersion += 1;
  server.server.sendToolListChanged();
}

Run stdio for developer laptops and CI, Streamable HTTP behind your gateway for multi-tenant production. Same tool implementations, different transport — that's the payoff of the protocol boundary.

Operational Checklist Before You Ship

• Pin the protocol version. Log negotiated version at initialize; reject mismatches loudly.
• Cache `tools/list` by fingerprint. Rebuild only on list_changed. Never re-list per turn.
• Sort tools deterministically and hash the catalog — this is a precondition for prompt caching.
• Cap tool output tokens with a next_cursor continuation; never return unbounded arrays.
• Mark idempotency explicitly on every tool (idempotentHint / x-idempotent). Disable retries otherwise.
• Normalize every error into recoverable | terminal before the model sees it.
• Validate model arguments against the schema server-side. Never trust the model; never 500 on a bad arg — return isError.
• Set per-tool timeouts (10 s is a sane default) with AbortController.
• Instrument p50/p95 per tool, tagged by protocol and transport. Protocol overhead is invisible without per-tool spans.
• Budget tokens per turn. Alert when the tool catalog exceeds 15% of the context window.

Pitfalls That Will Bite You

• New MCP session per call. The initialize handshake costs 40–120 ms. Hold the session; multiplex tool calls over it.
• Treating `isError` as fatal. An isError: true result is a successful tool call with a failed outcome — feed it back to the model so it can correct its arguments.
• Pasting raw OpenAPI into the prompt. A 40-endpoint spec is ~21k tokens per turn. Normalize and progressively disclose.
• Non-deterministic schema serialization. Key-order churn kills prompt caching silently and costs you 90% of the savings you thought you had.
• Retrying non-idempotent tools. The most expensive bug in agent infrastructure. It's a duplicate charge, not a 500.
• Assuming MCP is faster. It isn't meaningfully, and it isn't the point. MCP's wins are discovery, structured errors, and a stable capability boundary.

Sources & Technical References

• Model Context Protocol Specification (revisions 2025-03-26, 2025-06-18) — transports, lifecycle, tools, outputSchema/structuredContent, annotations. https://modelcontextprotocol.io/specification
• MCP: Transports (stdio vs Streamable HTTP) — https://modelcontextprotocol.io/specification/2025-06-18/basic/transports
• MCP: Tools (error semantics, isError, tool annotations) — https://modelcontextprotocol.io/specification/2025-06-18/server/tools
• JSON-RPC 2.0 Specification — error codes -32601, -32602, -32603; request/response framing. https://www.jsonrpc.org/specification
• OpenAPI Specification 3.1.0 — https://spec.openapis.org/oas/v3.1.0.html
• RFC 9110 — HTTP Semantics — idempotent methods, status code classes, Retry-After. https://www.rfc-editor.org/rfc/rfc9110.html
• RFC 6749 / OAuth 2.1 draft — authorization for MCP HTTP transports. https://datatracker.ietf.org/doc/html/draft-ietf-oauth-v2-1
• Anthropic — Tool use & prompt caching (`cache_control`) — https://docs.anthropic.com/en/docs/build-with-claude/tool-use and https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching
• OpenAI — Function calling & automatic prompt caching — https://platform.openai.com/docs/guides/function-calling
• MCP TypeScript SDK (@modelcontextprotocol/sdk) — https://github.com/modelcontextprotocol/typescript-sdk
• MDN — `AbortController` and `AbortSignal` — https://developer.mozilla.org/en-US/docs/Web/API/AbortController
• Node.js `perf_hooks` (monotonic clock) — https://nodejs.org/api/perf_hooks.html
• *Dean, J. & Barroso, L. (2013), The Tail at Scale, Communications of the ACM, 56(2)* — why p95/p99 matter more than p50 in distributed tool calls. https://doi.org/10.1145/2408776.2408794