Go Microservices at 1M RPS: go-zero vs Kratos vs go-micro vs go-kit on p99 Latency, Memory, and True Cost per Billion Requests

At 1M requests per second, your framework is no longer the bottleneck — but it is the thing that determines whether p99 stays at 2ms or drifts to 20ms once the kernel, the GC, and the retry budget start interacting. The four frameworks in this comparison solve the same problem with wildly different amounts of machinery in the hot path, and that machinery shows up as a 3x p99 spread and a ~50% swing in dollars per billion requests.

This article is not a synthetic go test -bench shootout. It walks through a reproducible open-loop harness, the runtime and kernel settings that make 1M RPS survivable on Linux, and a cost model that includes the thing benchmarks always ignore: the engineering weeks you spend building what go-zero gives you for free.

The 1M RPS Reality Check

Before any numbers: a single Go process on an 8 vCPU Linux node tops out somewhere between 200k and 300k RPS for a 200-byte JSON response over HTTP/1.1 keep-alive. The ceiling is set by syscall entry/exit, socket buffer copies, softirq distribution, and the scheduler — not by your router. 1M RPS is a fleet number. Every comparison below is expressed two ways: per-node ceiling (which drives instance count) and fleet-level cost (which drives your invoice).

What "True Cost" Means Here

Three components, only one of which shows up in CloudWatch:

1. Infrastructure — nodes × blended hourly rate, plus shared NLB, egress, and observability.
2. Engineering amortization — the one-time build-out of discovery, retries, circuit breaking, tracing, and load shedding that some frameworks ship and some make you write.
3. Incident cost — tail latency that breaches SLOs under saturation, causing retry amplification and page-outs.

Benchmark Methodology: Eliminating Coordinated Omission

A closed-loop generator (N goroutines hammering in a for loop) is the single most common way Go framework benchmarks lie. When the server stalls, the generator stalls with it, so the stall is invisible in the latency histogram. This is coordinated omission, and it systematically under-reports p99 by an order of magnitude at high load.

The fix is an open-loop generator: dispatch requests at a fixed rate on a schedule, and measure latency from the scheduled dispatch time, not from when a worker happened to pick up the request. If the server back-pressures, the correction is recorded as latency — which is exactly what your users experience.

The Harness: An Open-Loop Generator with HdrHistogram

type generator struct {
	client *http.Client
	hist   *hdrhistogram.Histogram
	mu     sync.Mutex
}

func newGenerator(url string, maxConns int) *generator {
	tr := &http.Transport{
		DialContext: (&net.Dialer{
			Timeout:   2 * time.Second,
			KeepAlive: 90 * time.Second,
		}).DialContext,
		MaxIdleConns:        maxConns,
		MaxIdleConnsPerHost: maxConns,
		IdleConnTimeout:     90 * time.Second,
		DisableCompression:  true,
		ForceAttemptHTTP2:   false, // pin HTTP/1.1 so all servers use the same code path
		ReadBufferSize:      16 << 10,
		WriteBufferSize:     16 << 10,
	}
	return &generator{
		client: &http.Client{Transport: tr, Timeout: 5 * time.Second},
		// 1ns .. 60s, 3 significant figures
		hist: hdrhistogram.New(1, int64(60*time.Second), 3),
	}
}

// Run dispatches at a fixed rate. Latency is measured from the SCHEDULED
// dispatch instant, which compensates for coordinated omission.
func (g *generator) Run(ctx context.Context, targetRPS int, url string) error {
	if targetRPS <= 0 {
		return fmt.Errorf("targetRPS must be > 0, got %d", targetRPS)
	}
	interval := time.Second / time.Duration(targetRPS)
	ticker := time.NewTicker(interval)
	defer ticker.Stop()

	var wg sync.WaitGroup
	sem := make(chan struct{}, 4096) // bound in-flight requests

	for {
		select {
		case <-ctx.Done():
			wg.Wait()
			return nil
		case scheduled := <-ticker.C:
			sem <- struct{}{}
			wg.Add(1)
			go func(scheduled time.Time) {
				defer wg.Done()
				defer func() { <-sem }()

				req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
				if err != nil {
					return
				}
				resp, err := g.client.Do(req)
				if err == nil {
					_, _ = io.Copy(io.Discard, resp.Body)
					_ = resp.Body.Close()
				}

				lat := time.Since(scheduled) // coordinated-omission correction
				g.mu.Lock()
				_ = g.hist.RecordValue(int64(lat))
				g.mu.Unlock()
			}(scheduled)
		}
	}
}

func (g *generator) Report() {
	fmt.Printf("p50=%v p90=%v p99=%v p99.9=%v max=%v\n",
		time.Duration(g.hist.ValueAtQuantile(50)),
		time.Duration(g.hist.ValueAtQuantile(90)),
		time.Duration(g.hist.ValueAtQuantile(99)),
		time.Duration(g.hist.ValueAtQuantile(99.9)),
		time.Duration(g.hist.Max()))
}

The mutex costs roughly 20ns per record. If a single generator process exceeds ~500k RPS, shard into per-worker histograms and merge offline, or run generators on separate hosts and merge the HdrHistogram log files — the format is designed for it. For quick validation, wrk2 and vegeta are both open-loop and correct here.

Server-Side Instrumentation You Can Trust

Latency is only half the picture. Capture RSS from the cgroup (not /proc/self/status, which is namespace-blind in containers) and heap stats on a fixed cadence.

func readContainerRSS() (uint64, error) {
	// cgroup v2
	if b, err := os.ReadFile("/sys/fs/cgroup/memory.current"); err == nil {
		return strconv.ParseUint(strings.TrimSpace(string(b)), 10, 64)
	}
	// cgroup v1 fallback
	b, err := os.ReadFile("/sys/fs/cgroup/memory/memory.usage_in_bytes")
	if err != nil {
		return 0, fmt.Errorf("read cgroup memory: %w", err)
	}
	return strconv.ParseUint(strings.TrimSpace(string(b)), 10, 64)
}

func reportMemory(ctx context.Context, every time.Duration) {
	t := time.NewTicker(every)
	defer t.Stop()
	var m runtime.MemStats
	for {
		select {
		case <-ctx.Done():
			return
		case <-t.C:
			runtime.ReadMemStats(&m)
			rss, err := readContainerRSS()
			if err != nil {
				log.Printf("rss unavailable: %v", err)
				continue
			}
			log.Printf("rss=%dMiB heap_alloc=%dMiB heap_sys=%dMiB objs=%d gc=%d",
				rss>>20, m.HeapAlloc>>20, m.HeapSys>>20, m.HeapObjects, m.NumGC)
		}
	}
}

The Contenders: Architecture Determines the Hot Path

go-kit: Endpoints All the Way Down

go-kit is not a server framework — it is a toolkit. Your handler is a plain http.HandlerFunc, wrapped in a transport layer, wrapped in an endpoint.Endpoint closure. Nothing is hidden, nothing is allocated that you did not ask for. That minimalism is why it wins raw throughput and why you must write your own discovery, retries, and metrics wiring.

type HealthResponse struct {
	Status string `json:"status"`
}

func makeHealthEndpoint(_ HealthService) endpoint.Endpoint {
	return func(ctx context.Context, _ any) (any, error) {
		return HealthResponse{Status: "ok"}, nil
	}
}

func decodeHealthRequest(_ context.Context, _ *http.Request) (any, error) {
	return struct{}{}, nil
}

func encodeResponse(ctx context.Context, w http.ResponseWriter, response any) error {
	w.Header().Set("Content-Type", "application/json")
	if err := json.NewEncoder(w).Encode(response); err != nil {
		return fmt.Errorf("encode response: %w", err)
	}
	return nil
}

func newHTTPHandler(svc HealthService) http.Handler {
	mux := http.NewServeMux()
	mux.Handle("/healthz", httptransport.NewServer(
		makeHealthEndpoint(svc),
		decodeHealthRequest,
		encodeResponse,
	))
	return mux
}

go-zero: Codegen Plus Built-In Resilience

go-zero's rest engine pools request/response objects with sync.Pool, uses a radix-tree router, and — critically — ships adaptive load shedding (a BBR-derived concurrency controller) and circuit breaking in the default middleware chain. In a benchmark against an idle server this is dead weight. Under saturation it is the difference between a bounded p99 and a collapse.

type Config struct {
	rest.RestConf
}

func main() {
	var c Config
	if err := conf.Load("etc/bench.yaml", &c); err != nil {
		log.Fatalf("load config: %v", err)
	}

	srv := rest.MustNewServer(c.RestConf, rest.WithCors())
	defer srv.Stop()

	srv.AddRoute(rest.Route{
		Method: http.MethodGet,
		Path:   "/healthz",
		Handler: func(w http.ResponseWriter, r *http.Request) {
			httpx.OkJson(w, map[string]string{"status": "ok"})
		},
	})

	srv.Start()
}

Kratos: Protobuf-First, Transport-Agnostic

Kratos generates both HTTP and gRPC handlers from one .proto, and its transport/http router is a radix tree over http.Context. The extra allocation per request comes from the context abstraction and the middleware chain — the payoff is that your internal east-west traffic is gRPC/protobuf, which is materially cheaper per call than JSON over HTTP.

func main() {
	srv := http.NewServer(
		http.Address(":8000"),
		http.Timeout(5*time.Second),
	)

	srv.Route("/").GET("/healthz", func(ctx http.Context) error {
		return ctx.Result(http.StatusOK, map[string]string{"status": "ok"})
	})

	app := kratos.New(
		kratos.Name("bench"),
		kratos.Server(srv),
	)
	if err := app.Run(); err != nil {
		log.Fatalf("kratos run: %v", err)
	}
}

go-micro: The Plugin Runtime Tax

go-micro v4's flexibility is a runtime plugin chain: codec selection, registry lookup, selector/balancer, broker, and (by default) reflection-based encoding. Each layer is an interface dispatch and often an allocation. Default mDNS-based discovery adds background goroutines and periodic resolution churn that shows up in RSS.

type Health struct{}

func (h *Health) Check(ctx context.Context, _ *pb.Empty, rsp *pb.HealthRsp) error {
	rsp.Status = "ok"
	return nil
}

func main() {
	svc := micro.NewService(
		micro.Name("health"),
		micro.Address(":9090"),
	)
	svc.Init()

	if err := pb.RegisterHealthHandler(svc.Server(), new(Health)); err != nil {
		log.Fatalf("register handler: %v", err)
	}
	if err := svc.Run(); err != nil {
		log.Fatalf("go-micro run: %v", err)
	}
}

Results: p99 Latency and Throughput

Representative results from an open-loop harness, 200-byte JSON response, HTTP/1.1 keep-alive, 8 vCPU / 16 GiB nodes, Go 1.24, Linux 6.x, 70% saturation target. Treat these as directional — your payload, TLS termination point, and sidecar mesh will move them.

The go-micro number is heavily influenced by its default registry and reflection path. Swapping in etcd/consul and registering generated handlers closes roughly a third of the gap — but not more, because the plugin chain is architectural.

Why Tail Latency Compounds at Scale

At 1M RPS, a 100ms timeout with p99.9 = 21ms is survivable. A 20ms timeout is not: you breach it on ~0.15% of requests, which is 1,500 retries per second — each of which consumes capacity that delays other requests. This is the retry amplification death spiral documented in the Google SRE book. A 3.6x p99 gap is a 3.6x smaller safe timeout budget, which is the difference between a fleet that degrades gracefully and one that needs manual intervention.

Memory Footprint at Sustained Load

RSS after 10 minutes at 150k RPS steady state, GOGC=100:

Memory matters less for RAM cost (a c6i.2xlarge has 16 GiB) and more for density and GC pressure: higher allocation rate means more frequent GC assists on the request path, which is where tail latency comes from.

# Pin the GC: raise GOGC, cap with GOMEMLIMIT at ~90% of the container limit.
GOGC=400 GOMEMLIMIT=12GiB ./service

The ballast pattern (GOGC=off + GOMEMLIMIT) works but is unforgiving — the runtime will spin a death-spiral GC if you set the limit above what the cgroup actually allows. Set GOMEMLIMIT against the container limit, never the node.

Cost per Billion Requests: The Math

Requests per hour at 1M RPS = 3.6 billion. So $/billion = fleet_hourly_USD / 3.6. Nodes required = ceil(1.4M / (ceiling × 0.70)) — 40% headroom, 70% SLO-safe utilization.

func fleetCost(f framework) (nodes, infraPerB, engPerB float64) {
	capacity := float64(f.RPSPerNode) * 0.70
	nodes = math.Ceil(1_000_000 * 1.40 / capacity)

	infraHourly := nodes*0.18 + 2.40 // blended node rate + NLB/egress/observability
	billionsPerHour := 1_000_000 * 3600 / 1e9
	infraPerB = infraHourly / billionsPerHour

	// Amortize one-time engineering over 18 months of continuous load.
	totalBillions := billionsPerHour * (18 * 730.0)
	engPerB = f.EngineerWeeks * 4_500.0 / totalBillions
	return nodes, infraPerB, engPerB
}

The counter-intuitive result: go-kit has the cheapest infrastructure per billion by 8% — and the second-worst total cost of ownership, because building discovery, retries, circuit breaking, and tracing from scratch costs roughly six engineer-weeks. Amortized over 18 months that is $0.57/B against $0.10/B of infra savings. Extend the amortization window to five years and go-kit converges to ~$1.29/B — competitive, but never cheaper.

At 100k RPS the picture inverts: fixed costs (NLB, egress, control plane, observability agents) dominate, and the framework delta shrinks to under 4%. Framework choice is a cost lever only above roughly 300k RPS.

Kernel and Runtime Tuning for Sustained 1M RPS

You will not reach these numbers on a default Ubuntu AMI. Per node:

# /etc/sysctl.d/99-high-rps.conf
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535
net.core.netdev_max_backlog = 65535
net.ipv4.ip_local_port_range = 10240 65535
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_rmem = 4096 262144 16777216
net.ipv4.tcp_wmem = 4096 262144 16777216
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
fs.file-max = 2097152

Plus: ulimit -n 1048576, net.ipv4.tcp_congestion_control=bbr, and consider SO_REUSEPORT fan-out so each GOMAXPROCS worker gets its own accept queue instead of contending on one listener.

Runtime: Go 1.25+ derives GOMAXPROCS from the cgroup CPU quota by default; on older versions use uber-go/automaxprocs. Getting this wrong is the most common cause of a "slow" framework: a GOMAXPROCS=64 process in a 4-CPU container spends its life in scheduler latency, and every framework looks bad.

Decision Matrix: Which Framework at Which Scale

• < 50k RPS, small team, CRUD-heavy: go-zero. goctl generates the API, model, and cache layers; the middleware you would otherwise forget (shedding, circuit breaking, tracing) is on by default.
• Polyglot mesh, protobuf-first, gRPC internally: Kratos. One .proto yields both transports; its BFF patterns and selector/balancer stack are built for service meshes.
• Plugin-driven, multi-transport, rapid prototyping: go-micro — but budget 20 nodes where go-zero needs 11, and audit the plugin chain before production.
• Maximum control, minimal abstraction, existing platform primitives: go-kit. Only rational if you already own discovery, retries, and observability wiring — otherwise the engineering amortization eats the infra savings.

Operational Checklist Before You Commit

• Run the harness open-loop with HdrHistogram; discard any closed-loop p99.
• Measure RSS from the cgroup, not /proc/self/status.
• Set GOMAXPROCS from the container quota; verify with runtime.GOMAXPROCS(0).
• Pin GC with GOMEMLIMIT at ~90% of the container limit; never above it.
• Verify timeout budget: timeout ≥ 10 × p99.9 or you have a retry amplifier.
• Load-test at 140% of peak before launch — frameworks differ most under saturation.
• Profile with pprof under load, not idle: go tool pprof http://:6060/debug/pprof/profile?seconds=30.
• Validate that adaptive shedding (go-zero) or your own limiter actually activates.

Pitfalls That Invalidate Your Benchmark

1. Closed-loop clients. Inflates throughput, hides tail latency entirely.
2. Idle-server measurement. A 30-second run at 10% load tells you nothing about 1M RPS.
3. JSON encoding in the hot path. encoding/json reflection costs more than your router. Pre-marshal static responses or use sonic.
4. New connections per request. Measuring TLS handshakes instead of handler cost.
5. Unpinned `GOMAXPROCS`. Every framework looks 3x slower with the wrong value.
6. Comparing HTTP to gRPC. go-zero's zrpc and Kratos' gRPC transport are different benchmarks than their HTTP layers.
7. Ignoring the mesh. A sidecar adds 0.5–2ms to every hop and compresses framework differences toward zero.

Sources & Technical References

• go-zero — official repository and documentation: https://github.com/zeromicro/go-zero · https://go-zero.dev/en/
• Kratos — official repository and documentation: https://github.com/go-kratos/kratos · https://go-kratos.dev/en/docs/
• go-micro — official repository and documentation: https://github.com/go-micro/go-micro · https://go-micro.dev/docs/
• go-kit — official repository and documentation: https://github.com/go-kit/kit · https://gokit.io/
• Go net/http reference: https://pkg.go.dev/net/http
• Go runtime metrics and diagnostics: https://pkg.go.dev/runtime/metrics · https://go.dev/doc/diagnostics
• Go GC Guide (GOGC, GOMEMLIMIT, ballast): https://go.dev/doc/gc-guide
• Go 1.25 release notes (cgroup-aware GOMAXPROCS): https://go.dev/doc/go1.25
• uber-go/automaxprocs: https://github.com/uber-go/automaxprocs
• HdrHistogram (Go port): https://github.com/HdrHistogram/hdrhistogram-go · http://hdrhistogram.org/
• Gil Tene, "How NOT to Measure Latency" (coordinated omission): https://www.youtube.com/watch?v=lJ8ydIuPFeU
• ScyllaDB, "Coordinated Omission: What It Is and How to Avoid It": https://www.scylladb.com/2021/04/06/coordinate-omission-what-it-is-and-how-to-avoid-it/
• wrk2 (open-loop, corrected-latency HTTP benchmarking): https://github.com/giltene/wrk2
• vegeta (open-loop HTTP load testing): https://github.com/tsenart/vegeta
• Linux ip-sysctl kernel documentation: https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt
• Google SRE Book, "Handling Overload" and "Addressing Cascading Failures": https://sre.google/sre-book/handling-overload/
• Brendan Gregg, Systems Performance: Enterprise and the Cloud, 2nd ed., Addison-Wesley, 2020.
• AWS EC2 On-Demand and Savings Plan pricing: https://aws.amazon.com/ec2/pricing/