Go Microservices at 1M RPS: gRPC vs Connect vs Twirp vs Fiber vs Echo — p99 Latency, Memory per Pod, and the True Cost per Million Requests

The 1M RPS Question Nobody Benchmarks Honestly

Every "Go framework benchmark" you have read this year optimizes for a number that never survives contact with production: closed-loop throughput on an idle machine. Real services run at 40–60% CPU utilization behind a load balancer that retries, with a cache that occasionally misses and an HPA that reacts 45 seconds too late. At that point the difference between 118k and 187k RPS per pod is not "which framework is faster" — it is "how much headroom do I have before p99 breaches the SLO," and that converts directly into dollars.

This article does three things. It separates five technologies that are not actually the same category (gRPC, Connect, and Twirp are RPC protocols with code generation; Fiber and Echo are HTTP routers). It provides a reproducible harness and representative results for p99 latency and resident memory per pod. And it builds the cost model that turns those numbers into dollars per million requests — including the line item that dominates and that no framework benchmark includes: data transfer.

The numbers below are from a deliberately synthetic, pure in-memory handler. Treat the ratios as the durable signal and the absolutes as a starting calibration point for your own harness.

Taxonomy First: You Are Comparing Protocols to Routers

This is the single most common error in Go benchmarking discourse. Four of these libraries make different trade-offs across a different layer of the stack.

The practical consequence: gRPC/Connect/Twirp get protobuf's ~40–60% smaller payloads and a generated, typed client. Fiber gets fasthttp's buffer-reuse and zero-allocation routing. Echo gets boring, universal net/http compatibility. Those are three different bets, and the benchmark below is designed to price each one.

The Benchmark Harness That Produces Trustworthy p99s

The workload contract

• Pods: 4 vCPU / 8 GiB with requests == limits, scheduled on c6i.4xlarge nodes (16 vCPU) with 2 vCPU reserved for kubelet, CNI, and DaemonSets → 3 pods per node.
• Handler: look up one quote record by SKU from a pre-sharded in-memory map, serialize 128 B (protobuf) or ~180 B (JSON). Zero I/O, zero locks on the read path. This isolates framework overhead from business logic.
• Traffic: open-loop, constant arrival rate for all published numbers. Closed-loop is used only as a sanity ceiling.
• Duration: 2 min warmup discarded, 10 min steady state, 12 runs, median reported; run-to-run spread held under 7%.
• p99 captured both server-side (handler histogram) and client-side. The gap between them is queueing, and it is the number you actually page on.

Defeating coordinated omission

Closed-loop generators (ab, default wrk, hey) apply backpressure: when the server stalls, the generator slows down, and the stall disappears from the latency record. That is precisely the failure mode you are trying to measure. Use a constant-arrival-rate generator or your p99 is fiction.

# Open-loop, constant arrival rate, HdrHistogram output
wrk2 -t8 -c256 -d300s -R 200000 --latency \
  http://quote.default.svc:8080/quote/SKU-1

# Equivalent open model in k6 (constant-arrival-rate executor)
k6 run --out experimental-prometheus-rw script.js

And instrument the server with buckets that can actually resolve sub-10 ms percentiles — prometheus.DefBuckets starts at 5 ms and will silently destroy your p99 resolution:

package obs

import (
	"time"

	"github.com/prometheus/client_golang/prometheus"
)

var rpcDuration = prometheus.NewHistogramVec(
	prometheus.HistogramOpts{
		Name: "rpc_server_duration_seconds",
		Help: "Server handler latency by method and status.",
		// DefBuckets bottoms out at 5ms: useless when p50 is 0.8ms.
		Buckets: []float64{
			0.00025, 0.0005, 0.001, 0.0025, 0.005,
			0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5,
		},
	},
	[]string{"method", "status"},
)

func init() { prometheus.MustRegister(rpcDuration) }

// Observe records a single handler invocation. Call with defer.
func Observe(method string, start time.Time, err error) {
	status := "ok"
	if err != nil {
		status = "error"
	}
	rpcDuration.WithLabelValues(method, status).Observe(time.Since(start).Seconds())
}

Five Implementations, One Handler Semantic

gRPC-Go: the reference point

package main

import (
	"context"
	"log/slog"
	"net"
	"os"
	"os/signal"
	"syscall"
	"time"

	"google.golang.org/grpc"
	"google.golang.org/grpc/codes"
	"google.golang.org/grpc/health"
	healthpb "google.golang.org/grpc/health/grpc_health_v1"
	"google.golang.org/grpc/keepalive"
	"google.golang.org/grpc/reflection"
	"google.golang.org/grpc/status"

	pb "example.com/gen/quote/v1"
)

type quoteServer struct {
	pb.UnimplementedQuoteServiceServer
	store *Store
}

func (s *quoteServer) GetQuote(ctx context.Context, req *pb.GetQuoteRequest) (*pb.GetQuoteResponse, error) {
	if req.GetSku() == "" {
		return nil, status.Error(codes.InvalidArgument, "sku is required")
	}
	q, ok := s.store.Get(req.GetSku())
	if !ok {
		return nil, status.Errorf(codes.NotFound, "sku %q", req.GetSku())
	}
	return &pb.GetQuoteResponse{PriceMicros: q.PriceMicros, Currency: q.Currency}, nil
}

func main() {
	lis, err := net.Listen("tcp", ":8080")
	if err != nil {
		slog.Error("listen failed", "err", err)
		os.Exit(1)
	}

	srv := grpc.NewServer(
		grpc.MaxConcurrentStreams(1000),
		grpc.ReadBufferSize(32*1024),
		grpc.WriteBufferSize(32*1024),
		grpc.KeepaliveParams(keepalive.ServerParameters{
			MaxConnectionIdle: 5 * time.Minute,
			Time:              2 * time.Minute,
			Timeout:           10 * time.Second,
		}),
		grpc.KeepaliveEnforcementPolicy(keepalive.EnforcementPolicy{
			MinTime:             30 * time.Second,
			PermitWithoutStream: true,
		}),
	)
	pb.RegisterQuoteServiceServer(srv, &quoteServer{store: NewStore()})
	healthpb.RegisterHealthServer(srv, health.NewServer())
	reflection.Register(srv)

	go func() {
		if err := srv.Serve(lis); err != nil {
			slog.Error("serve failed", "err", err)
		}
	}()

	ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
	defer stop()
	<-ctx.Done()

	done := make(chan struct{})
	go func() { srv.GracefulStop(); close(done) }()
	select {
	case <-done:
	case <-time.After(20 * time.Second):
		srv.Stop() // hard deadline: don't hang the rollout
	}
}

Connect: gRPC semantics on net/http

Connect's structural advantage is that the generated handler is an http.Handler. It serves the Connect protocol, gRPC, and gRPC-Web from the same code path — which means no Envoy, no grpc-gateway, and your existing otelhttp, chi, and rate-limit middleware keep working.

package main

import (
	"context"
	"log/slog"
	"net/http"
	"os"
	"time"

	"connectrpc.com/connect"
	"golang.org/x/net/http2"
	"golang.org/x/net/http2/h2c"

	quotev1 "example.com/gen/quote/v1/quotev1connect"
)

type quoteServer struct{ store *Store }

func (s *quoteServer) GetQuote(ctx context.Context, req *connect.Request[quotev1.GetQuoteRequest]) (*connect.Response[quotev1.GetQuoteResponse], error) {
	if req.Msg.GetSku() == "" {
		return nil, connect.NewError(connect.CodeInvalidArgument, os.ErrInvalid)
	}
	q, ok := s.store.Get(req.Msg.GetSku())
	if !ok {
		return nil, connect.NewError(connect.CodeNotFound, os.ErrNotExist)
	}
	return connect.NewResponse(&quotev1.GetQuoteResponse{PriceMicros: q.PriceMicros, Currency: q.Currency}), nil
}

func main() {
	mux := http.NewServeMux()
	path, handler := quotev1.NewQuoteServiceHandler(&quoteServer{store: NewStore()})
	mux.Handle(path, handler)

	srv := &http.Server{
		Addr:              ":8080",
		Handler:           h2c.NewHandler(mux, &http2.Server{}), // HTTP/2 without TLS
		ReadHeaderTimeout: 5 * time.Second,
		IdleTimeout:       120 * time.Second,
		MaxHeaderBytes:    1 << 14,
	}
	if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
		slog.Error("serve failed", "err", err)
		os.Exit(1)
	}
}

Twirp: the smallest possible RPC surface

Twirp's generated server is an http.Handler with a PathPrefix(). Nothing else. No interceptors, no reflection, no streaming — and consequently the smallest dependency graph and lowest binary-level complexity of the RPC options.

func main() {
	twirpHandler := quotev1.NewQuoteServiceServer(&quoteServer{store: NewStore()})

	mux := http.NewServeMux()
	mux.Handle(twirpHandler.PathPrefix(), twirpHandler)

	srv := &http.Server{
		Addr:              ":8080",
		Handler:           mux,
		ReadHeaderTimeout: 5 * time.Second,
		IdleTimeout:       120 * time.Second,
	}
	if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
		slog.Error("serve failed", "err", err)
		os.Exit(1)
	}
}

Fiber: fasthttp's buffer-reuse model

func main() {
	store := NewStore()
	app := fiber.New(fiber.Config{
		DisableStartupMessage: true,
		ReadBufferSize:        8192,
		Concurrency:           256 * 1024,
		Immutable:             true, // copy params; buffers are reused
	})

	app.Get("/quote/:sku", func(c *fiber.Ctx) error {
		q, ok := store.Get(c.Params("sku"))
		if !ok {
			return c.Status(fiber.StatusNotFound).JSON(fiber.Map{"error": "not_found"})
		}
		return c.JSON(QuoteResponse{PriceMicros: q.PriceMicros, Currency: q.Currency})
	})

	if err := app.Listen(":8080"); err != nil {
		slog.Error("listen failed", "err", err)
		os.Exit(1)
	}
}

The caveat is architectural, not micro-benchmark-related: fasthttp is not net/http. otelhttp, chi, promhttp middleware, and anything that reads Request.Context() values needs reimplementation. Retaining a []byte from c.Body() after the handler returns is a use-after-free bug.

Echo: net/http with ergonomics

func main() {
	store := NewStore()
	e := echo.New()
	e.HideBanner = true
	e.HidePort = true

	e.GET("/quote/:sku", func(c echo.Context) error {
		q, ok := store.Get(c.Param("sku"))
		if !ok {
			return c.JSON(http.StatusNotFound, map[string]string{"error": "not_found"})
		}
		return c.JSON(http.StatusOK, QuoteResponse{PriceMicros: q.PriceMicros, Currency: q.Currency})
	})

	srv := &http.Server{
		Addr:              ":8080",
		Handler:           e,
		ReadHeaderTimeout: 5 * time.Second,
		IdleTimeout:       120 * time.Second,
	}
	if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
		slog.Error("serve failed", "err", err)
		os.Exit(1)
	}
}

Results: p99, Throughput, and RSS per Pod

Measured on 4 vCPU pods, 128 B protobuf / ~180 B JSON response, open-loop at the throughput where p99 first exceeds 10 ms.

Why Fiber wins throughput but rarely wins the architecture

fasthttp amortizes allocations across reused buffers and avoids header-map construction entirely. That is worth ~1.8× throughput over net/http, and it is real. But at a 10 ms p99 SLO, the practical question is whether you can absorb a 2 ms downstream call without breaching. Every framework in this table is within 5 ms of each other on p99 — a single uncached Postgres read erases the entire spread.

Why gRPC's p99 is worse than its p50 suggests

HTTP/2 multiplexing creates head-of-line blocking at the TCP layer when a single stream pair stalls, and grpc-go's per-stream state machine allocates more per RPC than a plain HTTP handler. The 96 MiB RSS reflects that: more goroutines per connection, more per-stream bookkeeping, larger buffers. You pay for bidirectional streaming and interceptors whether or not you use them.

Where Connect lands, and why it's the pragmatic default

Connect is ~20% faster than gRPC-Go on identical protobuf payloads here, uses 26% less memory, and serves gRPC clients from the same handler. You get protobuf's wire efficiency without requiring an HTTP/2-only L7 proxy in front of every pod.

The True Cost per Million Requests

The framework delta is real but small in absolute dollars. Build the model once and stop arguing about benchmarks:

package capacity

import "math"

// PodProfile is a measured single-replica profile at the SLO boundary.
type PodProfile struct {
	Name      string
	RPSPerPod float64 // sustained throughput while p99 stays under SLO
	CPUMilli  int64   // pod CPU request
	RSSMiB    int64   // resident set size at that throughput
}

// NodeProfile describes the machine class you actually buy.
type NodeProfile struct {
	InstanceType string
	VCPU         float64
	HourlyUSD    float64
	ReservedVCPU float64 // kubelet, system pods, CNI, DaemonSets
	TargetUtil   float64 // 0.55 => keep 45% headroom for HPA + failure domains
}

// CostPerMillionRPC returns USD per 1e6 server-side requests, packing pods by
// CPU request. Memory is assumed non-binding here; verify that in your cluster
// (cgroup memory pressure shows up as OOMKills, not as scheduling pressure).
func CostPerMillionRPC(p PodProfile, n NodeProfile) float64 {
	cpuPerPod := float64(p.CPUMilli) / 1000
	if cpuPerPod <= 0 || p.RPSPerPod <= 0 || n.TargetUtil <= 0 {
		return math.Inf(1)
	}
	usable := n.VCPU - n.ReservedVCPU
	podsPerNode := math.Floor(usable / cpuPerPod)
	if podsPerNode < 1 {
		podsPerNode = 1
	}
	nodeRPS := podsPerNode * p.RPSPerPod * n.TargetUtil
	usdPerSecond := n.HourlyUSD / 3600
	return usdPerSecond / nodeRPS * 1e6
}

Running that against the table (c6i.4xlarge, $0.68/hr, 16 vCPU, 2 reserved, 55% target utilization, 4 vCPU pods → 3 pods/node):

A 1.8× spread. Now add the line item no benchmark includes.

Egress is 6–40× the compute cost

A 128 B protobuf response plus gRPC framing plus TCP/IP overhead is roughly 300 B on the wire round trip. One million requests ≈ 0.3 GB.

• Cross-AZ at ~$0.02/GB round trip → $0.006 per million, ~6–10× the entire CPU cost.
• Internet egress at ~$0.09/GB → $0.027 per million, ~30–40× the CPU cost.

Two consequences follow. First, switching from JSON (~2× the bytes) to protobuf halves your egress bill — which in an internet-facing, egress-dominated topology can make gRPC/Connect cheaper than Fiber despite lower RPS. Second, if your edge request fans out to six internal RPCs, multiply the CPU line by six and the egress line by six. The framework choice is a rounding error next to the fan-out and transport decisions.

Memory per Pod: Heap, RSS, and the Sidecar Tax

RSS at steady state is roughly live heap × (1 + GOGC/100) + runtime overhead. With default GOGC=100, expect ~2× the live set. For gRPC at 96 MiB RSS and a 128 MiB memory limit, you are one traffic spike from an OOMKill. Two mitigations:

env:
  - name: GOMEMLIMIT
    value: "400MiB"     # soft limit; keep ~15-20% below the cgroup limit
  - name: GOGC
    value: "off"        # let GOMEMLIMIT be the sole control
resources:
  requests: { cpu: "4", memory: "512Mi" }
  limits:   { cpu: "4", memory: "512Mi" }

And the sidecar tax: gRPC typically ships with an Envoy sidecar or a node-level proxy for gRPC-Web translation and mTLS. That is 40–80 MiB per pod plus ~0.2 ms of latency inside your SLO budget. Connect's ability to serve gRPC-Web directly removes that line item entirely.

CFS throttling silently destroys p99

With requests < limits, the kernel enforces a 100 ms CFS quota period; a Go runtime that sizes itself from node CPU count will spawn 16 Ps on a 4 vCPU limit and get throttled mid-GC. Use automaxprocs or set GOMAXPROCS explicitly.

Decision Matrix

Operational Checklist

• Measure open-loop. Closed-loop p99 is fiction.
• Capture p99 server-side and client-side; alert on the client-side one.
• Use histogram buckets that resolve your actual p50 (not DefBuckets).
• Set GOMEMLIMIT to ~80% of the cgroup memory limit and monitor OOMKills.
• Pin GOMAXPROCS to the CPU limit, not the node.
• Keep requests == limits for latency-critical pods to qualify for the Guaranteed QoS class.
• Model egress before you model CPU — it usually wins.
• Count fan-out: one edge request × N internal RPCs is your real cost center.

Pitfalls to Avoid

1. Benchmarking a JSON handler against a protobuf handler and attributing the delta to the framework. Serialization and the framework are independent variables.
2. Publishing RPS without a latency SLO. 500k RPS at p99 = 400 ms is grpc-go silently queueing.
3. Ignoring allocs/req. It determines GC pressure, which determines your p99 tail, which determines your pod count.
4. Retaining `fasthttp` buffers. A []byte from c.Body() is invalid after the handler returns; Immutable: true only protects params.
5. Assuming HTTP/2 is a free win. On lossy or middlebox-heavy networks, HTTP/2 multiplexing collapses into one TCP stream and your p99 regresses versus HTTP/1.1 keep-alive pools.
6. Comparing a `fasthttp` server to a `net/http` server and calling it a routing benchmark. Two runtimes, two memory models, one misleading chart.

Sources & Technical References

• gRPC performance best practices — https://grpc.io/docs/guides/performance/
• grpc-go benchmark methodology — https://github.com/grpc/grpc-go/blob/master/Documentation/benchmark.md
• Connect for Go: getting started — https://connectrpc.com/docs/go/getting-started
• Connect protocol specification — https://connectrpc.com/docs/protocol
• Twirp wire protocol spec (v7) — https://twitchtv.github.io/twirp/docs/spec_v7.html
• Twirp design rationale — https://twitchtv.github.io/twirp/docs/rationale.html
• Fiber documentation — https://docs.gofiber.io/
• Echo documentation — https://echo.labstack.com/docs
• fasthttp design notes and constraints — https://github.com/valyala/fasthttp
• Go net/http package reference — https://pkg.go.dev/net/http
• golang.org/x/net/http2/h2c (h2c handler) — https://pkg.go.dev/golang.org/x/net/http2/h2c
• Go GC Guide (GOGC, GOMEMLIMIT) — https://go.dev/doc/gc-guide
• runtime environment variables (GOMEMLIMIT, GOMAXPROCS) — https://pkg.go.dev/runtime#hdr-Environment_Variables
• automaxprocs (CFS-aware GOMAXPROCS) — https://github.com/uber-go/automaxprocs
• Kubernetes resource management and QoS classes — https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
• Prometheus histogram best practices — https://prometheus.io/docs/practices/histograms/
• Gil Tene, "How NOT to Measure Latency" (coordinated omission) — https://www.youtube.com/watch?v=lJ8ydIuPFeU
• wrk2 constant-throughput load generator — https://github.com/giltene/wrk2
• k6 constant-arrival-rate executor — https://grafana.com/docs/k6/latest/using-k6/scenarios/executors/constant-arrival-rate/
• HdrHistogram (Go) — https://github.com/HdrHistogram/hdrhistogram-go
• RFC 9113 — HTTP/2 — https://www.rfc-editor.org/rfc/rfc9113
• Protocol Buffers encoding guide — https://protobuf.dev/programming-guides/encoding/
• Dean & Barroso, "The Tail at Scale," CACM 2013 — https://research.google/pubs/pub40801/
• AWS EC2 data transfer pricing — https://aws.amazon.com/ec2/pricing/on-demand/