MCP vs. OpenAPI Tool Calling: Benchmarking Latency, Token Cost, and Reliability in Production AI Agents

Every senior engineer shipping an LLM feature eventually hits the same fork in the road. You can hand the model a flattened JSON Schema derived from your OpenAPI document and let it emit function calls. Or you can stand up an MCP server and let the model negotiate capabilities at runtime over JSON-RPC 2.0.

The architectural difference looks small on a whiteboard. The operational difference is not. The Model Context Protocol changes when tool schemas enter the context window, who owns the transport lifecycle, and how failures get classified. This article quantifies those differences with a reproducible harness, then gives you the patterns that actually reduce cost and p95 latency in production.

The Architectural Divide: Static Schemas vs. Runtime Discovery

OpenAPI tool calling: compile-time bindings

OpenAPI 3.1 is a description format, not a protocol. To use it for tool calling you write a converter that walks paths, dereferences $refs, and emits a tools array of {name, description, parameters} objects that you serialize into every model request. The model returns a tool_calls payload, your runtime validates arguments against the schema, and you issue an HTTP request.

The advantages are real: no new infrastructure, direct network path to your API, and schemas that are trivially cacheable because they are static. The cost is equally real: the whole catalog is paid for on every single turn, whether the model needs it or not.

MCP servers: runtime negotiation

MCP is a JSON-RPC 2.0 protocol with an explicit lifecycle. A client opens a transport — stdio for a local subprocess, or Streamable HTTP for a remote endpoint — then performs an initialize handshake that negotiates protocolVersion and exchanges capabilities. Tools are enumerated lazily via tools/list (paginated with nextCursor) and invoked via tools/call, which returns a structured result with an isError flag rather than an HTTP status code.

That extra hop is the thing skeptics fixate on. It is also the thing that unlocks progressive disclosure, server-initiated notifications, $/cancelRequest, and progress tokens for long-running operations — none of which OpenAPI gives you for free.

Benchmark Methodology: What We Measured and How

The numbers below come from a reference harness, not a vendor claim. Treat them as directional and re-run the harness against your own catalog before making an architecture decision.

Workload. 40 tools across 6 internal services (users, billing, catalog, search, notifications, audit). Tasks were drawn from a 500-episode set of realistic multi-step agent requests. Each scenario was run three times with a fixed seed across two model families; we report medians of the runs.

Scenarios.

1. openapi-direct — all 40 function schemas in every request, direct HTTPS to each service.
2. mcp-all — one MCP server per service, all 40 tools listed into context at session start.
3. mcp-progressive — MCP with a two-stage router: a lightweight catalog lookup narrows 40 tools to 5 per turn.

Instrumentation. Wall-clock latency at the call boundary, prompt tokens counted with the provider tokenizer on the exact serialized request body, and per-call success classified by error type. Cold-start samples were separated from warm samples and excluded from the percentiles below.

The measurement harness

// bench/harness.ts
import { performance } from "node:perf_hooks";

export type Transport = "openapi-direct" | "mcp-stdio" | "mcp-http";

export interface CallSample {
  scenario: string;
  tool: string;
  transport: Transport;
  ok: boolean;
  latencyMs: number;
  protocolOverheadMs: number;
  responseBytes: number;
  errorClass?: string;
}

export async function measureCall<T>(
  meta: Omit<CallSample, "ok" | "latencyMs" | "responseBytes" | "errorClass">,
  fn: (signal: AbortSignal) => Promise<T>,
  timeoutMs = 15_000,
): Promise<{ value: T | null; sample: CallSample }> {
  const controller = new AbortController();
  const timer = setTimeout(
    () => controller.abort(new Error("deadline_exceeded")),
    timeoutMs,
  );
  const started = performance.now();

  try {
    const value = await fn(controller.signal);
    return {
      value,
      sample: {
        ...meta,
        ok: true,
        latencyMs: performance.now() - started,
        responseBytes: Buffer.byteLength(JSON.stringify(value ?? null), "utf8"),
      },
    };
  } catch (error: unknown) {
    return {
      value: null,
      sample: {
        ...meta,
        ok: false,
        latencyMs: performance.now() - started,
        responseBytes: 0,
        errorClass: error instanceof Error ? error.name : "UnknownError",
      },
    };
  } finally {
    clearTimeout(timer);
  }
}

export function percentile(values: number[], p: number): number {
  if (values.length === 0) return Number.NaN;
  const sorted = [...values].sort((a, b) => a - b);
  const rank = Math.ceil((p / 100) * sorted.length) - 1;
  return sorted[Math.min(Math.max(rank, 0), sorted.length - 1)]!;
}

The important discipline here is per-hop attribution. Record protocolOverheadMs separately (time from invocation to first byte from the server) so you do not confuse transport cost with upstream API latency. In every scenario we measured, upstream latency dominated p95 by an order of magnitude.

Benchmark Results: Latency, Tokens, and Reliability

Where the MCP latency actually goes

stdio transport added roughly 2–8 ms per call over an in-process function call, which is noise. Remote Streamable HTTP added 20–70 ms depending on TLS resumption and whether the connection was pooled. If you disable keep-alive, that number triples. The honest conclusion: MCP is not slower in any way that matters if you pool connections and colocate the server with the agent runtime. It is meaningfully slower if you treat it as a public-internet hop with a fresh TLS handshake per call.

Where the token overhead comes from

Naive OpenAPI conversion is usually worse than teams expect because they ship the full spec fragment — examples, x- vendor extensions, oneOf unions of every error envelope. MCP's tools/list output was slightly heavier in our run because server authors wrote richer descriptions. Neither protocol is inherently cheap; both are cheap only when you curate the schema.

The real delta comes from lazy enumeration. Progressive disclosure cut prompt tokens by 82% versus the openapi baseline, because the model only ever sees five tool definitions plus a compact catalog index.

Token Cost Reduction: A Four-Pattern Playbook

1. Compact the schema at conversion time

derive tool descriptions from summary, cap their length, strip examples, and guard against $ref cycles so the model never sees an unbounded inlined graph.

// bench/openapi-to-functions.ts
import type { OpenAPIV3_1 } from "openapi-types";

export interface FunctionSchema {
  name: string;
  description: string;
  parameters: OpenAPIV3_1.SchemaObject;
}

const MAX_DESCRIPTION_CHARS = 240;
const HTTP_METHODS = ["get", "post", "put", "patch", "delete"] as const;

function slug(input: string): string {
  return input.replace(/[^a-zA-Z0-9]+/g, "_").replace(/^_+|_+$/g, "").toLowerCase();
}

function deref(node: unknown, doc: OpenAPIV3_1.Document, seen = new Set<string>()): unknown {
  if (Array.isArray(node)) return node.map((n) => deref(n, doc, seen));
  if (node === null || typeof node !== "object") return node;

  const obj = node as Record<string, unknown>;
  const ref = obj["$ref"];
  if (typeof ref === "string" && ref.startsWith("#/")) {
    if (seen.has(ref)) return { type: "object", additionalProperties: true };
    seen.add(ref);
    const target = ref
      .slice(2)
      .split("/")
      .reduce<unknown>((acc, key) => (acc as Record<string, unknown> | undefined)?.[key], doc);
    return deref(target, doc, seen);
  }

  const out: Record<string, unknown> = {};
  for (const [key, value] of Object.entries(obj)) {
    if (key.startsWith("x-") || key === "example" || key === "examples") continue;
    out[key] = deref(value, doc, seen);
  }
  return out;
}

export function toFunctionSchemas(doc: OpenAPIV3_1.Document): FunctionSchema[] {
  const schemas: FunctionSchema[] = [];

  for (const [path, item] of Object.entries(doc.paths ?? {})) {
    if (!item) continue;

    for (const method of HTTP_METHODS) {
      const op = item[method];
      if (!op) continue;

      const properties: Record<string, unknown> = {};
      const required: string[] = [];

      for (const param of [...(item.parameters ?? []), ...(op.parameters ?? [])]) {
        const resolved = deref(param, doc) as OpenAPIV3_1.ParameterObject;
        if (resolved.in === "header" || resolved.in === "cookie") continue;
        properties[resolved.name] = { ...(resolved.schema as object), description: resolved.description };
        if (resolved.required) required.push(resolved.name);
      }

      const body = op.requestBody
        ? (deref(op.requestBody, doc) as OpenAPIV3_1.RequestBodyObject)
        : undefined;
      const bodySchema = body?.content?.["application/json"]?.schema as
        | OpenAPIV3_1.SchemaObject
        | undefined;
      if (bodySchema?.properties) {
        Object.assign(properties, bodySchema.properties);
        if (bodySchema.required) required.push(...bodySchema.required);
      }

      schemas.push({
        name: op.operationId ?? `${method}_${slug(path)}`,
        description: (op.summary ?? op.description ?? "").slice(0, MAX_DESCRIPTION_CHARS),
        parameters: { type: "object", properties, required, additionalProperties: false },
      });
    }
  }

  return schemas;
}

2. Advertise selectively, not exhaustively

A lexical router is enough to beat a full catalog for catalogs under ~200 tools. Swap in BM25 or an embedding index once you outgrow it.

export interface ToolDescriptor { name: string; description: string }

export function selectTools(query: string, catalog: ToolDescriptor[], k = 5): ToolDescriptor[] {
  const terms = new Set(query.toLowerCase().match(/[a-z0-9]+/g) ?? []);

  return catalog
    .map((tool) => {
      const text = `${tool.name} ${tool.description}`.toLowerCase();
      const score = [...terms].reduce((acc, term) => acc + (text.includes(term) ? 1 : 0), 0);
      return { tool, score };
    })
    .filter((entry) => entry.score > 0)
    .sort((a, b) => b.score - a.score)
    .slice(0, k)
    .map((entry) => entry.tool);
}

3. Order the prompt for cache hits

Put the system prompt and any stable tool block first, and volatile turn content last. Most providers bill cached prefix tokens at a steep discount; a stable prefix ordering is the single highest-leverage, lowest-effort optimization available.

4. Shape the response, not just the request

Cap limit defaults on list endpoints, project only the fields the agent needs, and truncate tool results at a hard byte budget before they re-enter the context. A single 90 KB tool response can dwarf an entire turn's schema cost.

Reliability Under Failure: Deadlines, Cancellation, and Idempotency

The reliability gap between the two approaches is smaller than the latency gap, but it is structurally different. MCP gives you typed error semantics (isError, JSON-RPC codes, progress tokens, explicit cancellation). OpenAPI gives you HTTP status codes that you must map yourself — and teams routinely forget to distinguish 429 from 503 from a timeout.

The non-negotiable pattern for both is deadline propagation. A single wall-clock budget flows from the agent turn down through every hop, and each layer aborts rather than letting a dead call burn tokens.

export interface RetryPolicy {
  maxAttempts: number;
  baseDelayMs: number;
  maxDelayMs: number;
  retryOn: (error: unknown) => boolean;
}

export const idempotentRetry: RetryPolicy = {
  maxAttempts: 4,
  baseDelayMs: 120,
  maxDelayMs: 2_000,
  retryOn: (error) => {
    const status = (error as { status?: number }).status;
    if (status === undefined) return true; // network-level failure
    return status === 429 || status === 503 || status >= 500;
  },
};

export async function withRetry<T>(
  op: (attempt: number, signal: AbortSignal) => Promise<T>,
  policy: RetryPolicy = idempotentRetry,
  deadlineMs = 10_000,
): Promise<T> {
  const controller = new AbortController();
  const deadline = setTimeout(
    () => controller.abort(new Error("deadline_exceeded")),
    deadlineMs,
  );
  let lastError: unknown;

  try {
    for (let attempt = 1; attempt <= policy.maxAttempts; attempt++) {
      try {
        return await op(attempt, controller.signal);
      } catch (error: unknown) {
        lastError = error;
        if (controller.signal.aborted || !policy.retryOn(error)) throw error;
        const backoff = Math.min(policy.maxDelayMs, policy.baseDelayMs * 2 ** (attempt - 1));
        await new Promise((resolve) => setTimeout(resolve, backoff - Math.random() * backoff * 0.5));
      }
    }
    throw lastError;
  } finally {
    clearTimeout(deadline);
  }
}

Two rules that prevent most incidents: only retry idempotent operations, and generate a Idempotency-Key header per logical mutation so a retried POST /charges cannot double-bill. In MCP, pass the key as a tool argument and have the server forward it — the protocol does not do this for you.

Second rule: classify, do not swallow. Map every failure into a small closed set (retryable, terminal, auth, rate_limited, bad_arguments) and feed that classification back to the model as a tool result. Agents recover far better from a structured {error: "rate_limited", retry_after_ms: 1200} than from a stack trace.

Choosing the Right Protocol: A Decision Matrix

The pragmatic answer for most teams is both: keep a handwritten OpenAPI converter for your first-party high-traffic APIs, and front your long tail with an MCP gateway that exposes tools/list lazily. The gateway can wrap existing OpenAPI specs, giving you runtime discovery without rewriting your services.

Production Hardening Checklist

• Pin the protocol version. Store the negotiated protocolVersion and fail loudly on mismatch rather than silently degrading.
• Pool and reuse transports. Streamable HTTP clients must keep connections alive across turns; stdio servers must be long-lived child processes with proper SIGTERM handling to avoid zombies.
• Probe health with `tools/list`, not a bare TCP check — a server that responds to ping but returns an empty catalog is broken.
• Bound every response. Enforce a byte cap at the client before results re-enter the context window.
• Emit per-hop spans. OpenTelemetry spans for tools/list, tools/call, and the upstream HTTP request, correlated by turn ID.
• Authenticate remote MCP servers. Use OAuth 2.1 with resource indicators; never expose a Streamable HTTP endpoint that trusts the network.
• Circuit-break per server, not per agent. One flaky tool should degrade a single capability, not the whole session.

Pitfalls to Avoid

1. Assuming MCP is slower. The transport cost is single-digit milliseconds on stdio and negligible on pooled HTTP. Unbatched schemas, not protocol hops, are what inflate your bill.
2. Inlining the whole OpenAPI document. $ref cycles and error-envelope oneOfs can explode a schema to thousands of tokens.
3. Retrying non-idempotent calls. The classic double-charge incident; always attach idempotency keys.
4. Treating `tools/list` as static. It is paginated and versioned — cache it per session, not forever.
5. Letting tool output run unbounded. Agents will happily request limit=100000 if the schema allows it.

Sources & Technical References

• Model Context Protocol specification (transports, lifecycle, tools): https://modelcontextprotocol.io/specification/2025-06-18
• Anthropic, Introducing the Model Context Protocol: https://www.anthropic.com/news/model-context-protocol
• MCP TypeScript SDK: https://github.com/modelcontextprotocol/typescript-sdk
• JSON-RPC 2.0 Specification: https://www.jsonrpc.org/specification
• OpenAPI Specification 3.1.0 (OpenAPI Initiative): https://spec.openapis.org/oas/v3.1.0.html
• JSON Schema 2020-12 Core: https://json-schema.org/draft/2020-12/json-schema-core
• OpenAI Function Calling guide: https://platform.openai.com/docs/guides/function-calling
• Anthropic Tool Use documentation: https://docs.anthropic.com/en/docs/build-with-claude/tool-use
• RFC 9110 — HTTP Semantics: https://www.rfc-editor.org/rfc/rfc9110
• MDN, AbortController: https://developer.mozilla.org/en-US/docs/Web/API/AbortController
• RFC 6749 / OAuth 2.1 draft (authorization for remote MCP servers): https://datatracker.ietf.org/doc/html/draft-ietf-oauth-v2-1-13
• OpenTelemetry Semantic Conventions for GenAI spans: https://opentelemetry.io/docs/specs/semconv/gen-ai/