Vector Search at 1B Embeddings: Qdrant vs Milvus vs Weaviate vs pgvector — p99 Recall Latency, Memory Footprint, and the True Cost per Million Queries

The Only Benchmark That Matters: Latency Anchored to Recall

Every "vector database benchmark" you have read this quarter has the same defect: it compares p99 latency at whatever recall the engine happened to produce. That is not a benchmark, it is a configuration artifact. You can make any of these four engines hit 5 ms p99 by setting ef_search = 16 and returning garbage.

The comparable unit is recall-anchored latency — the p99 you observe while holding recall@10 above a target (typically 0.95 for RAG, 0.99+ for dedup or compliance retrieval):

R-Latency(τ) = p99 latency measured with recall@10 ≥ τ, under a fixed concurrency and a fixed QPS ceiling.

Everything below is framed that way. We will cover the memory arithmetic that sets your node count at 1B vectors, the engine-specific knobs that move you along the recall/latency frontier, and a cost model expressed in exact decimal money.

Anatomy of a 1B-Vector Deployment: The Arithmetic Nobody Does First

Before you benchmark anything, compute the floor. At 768 dimensions and 1B vectors, the raw float32 payload alone is ~2.86 TiB. No amount of tuning fixes that — only quantization or sharding does.

The critical, under-reported consequence: quantization compresses vectors but not the graph. Moving from float32 to int8 saves 2.1 TiB, yet you still owe 238 GiB for the HNSW adjacency lists at M=32 — and those links cannot be meaningfully quantized. At 1B scale the graph becomes your irreducible memory floor, and it scales with M, not with dimension.

That single fact explains most of the divergence between engines in the next section.

Configuring Each Engine for the Same Recall Target

Qdrant: Scalar Quantization + On-Disk HNSW

Qdrant's differentiator at 1B is that it lets you keep quantized vectors in RAM while the original float32 vectors and the graph live on disk, with always_ram controlling the split precisely.

from qdrant_client import QdrantClient, models
from qdrant_client.http.exceptions import UnexpectedResponse

client = QdrantClient(url="http://qdrant:6333", prefer_grpc=True, timeout=60)

try:
    client.create_collection(
        collection_name="chunks_v1",
        vectors_config=models.VectorParams(
            size=768,
            distance=models.Distance.COSINE,
            on_disk=True,  # float32 originals on NVMe, used only for rescoring
            hnsw_config=models.HnswConfigDiff(m=32, ef_construct=256),
            quantization_config=models.ScalarQuantization(
                scalar=models.ScalarQuantizationConfig(
                    type=models.ScalarType.INT8,
                    quantile=0.99,
                    always_ram=True,  # int8 copy pinned in RAM (~715 GiB @ 1B)
                )
            ),
        ),
        hnsw_config=models.HnswConfigDiff(m=32, ef_construct=256, on_disk=True),
    )
except UnexpectedResponse as exc:
    if exc.status_code != 409:  # 409 == already exists, safe to ignore
        raise

At query time, oversampling plus rescoring is what buys back the ~1-2 points of recall that int8 costs you:

res = client.query_points(
    collection_name="chunks_v1",
    query=query_vector,          # list[float], len 768
    limit=50,
    search_params=models.SearchParams(
        hnsw_ef=192,
        quantization=models.Rescore(oversampling=3.0),  # fetch 3x, rescore on float32
    ),
    with_payload=["doc_id", "chunk_idx"],
)

Milvus: Index Diversity and True Horizontal Scale

Milvus separates compute (query nodes) from storage (object store + WAL), which makes scaling past 1B genuinely elastic — and makes cost accounting harder because it is a multi-component deployment.

from pymilvus import MilvusClient, DataType

client = MilvusClient(uri="http://milvus:19530", token="root:Milvus")

schema = client.create_schema(auto_id=False, enable_dynamic_field=False)
schema.add_field("id", DataType.INT64, is_primary=True)
schema.add_field("doc_id", DataType.INT64)
schema.add_field("embedding", DataType.FLOAT_VECTOR, dim=768)

index_params = client.prepare_index_params()
index_params.add_index(
    field_name="embedding",
    index_type="HNSW",
    metric_type="COSINE",
    params={"M": 32, "efConstruction": 256},
)
index_params.add_index(field_name="doc_id", index_type="STL_SORT")

client.create_collection(
    collection_name="chunks_v1",
    schema=schema,
    index_params=index_params,
    consistency_level="Bounded",  # never use Strong at 1B; it serializes your QPS
)

For >1B vectors where RAM is the binding constraint, DISKANN is the honest choice: it trades ~2-4x latency for a 5-10x reduction in RAM by keeping the graph on SSD with a compressed in-memory representation.

Weaviate: Best-in-Class Filtered Search, Highest Memory Tax

Weaviate shines when your queries carry structured filters (tenant, ACL, recency) because it fuses an inverted index with the vector index at the segment level. Product Quantization is the lever for 1B.

import weaviate
from weaviate.classes.config import Configure, Property, DataType

client = weaviate.connect_to_custom(
    http_host="weaviate", http_port=8080, http_secure=False,
    grpc_host="weaviate", grpc_port=50051, grpc_secure=False,
)

client.collections.create(
    name="Chunks",
    vector_config=Configure.Vectors.self_provided(
        vector_index_config=Configure.VectorIndex.hnsw(
            distance_metric=Configure.VectorIndex.Distance.COSINE,
            max_connections=32,
            ef_construction=256,
            ef=192,
            quantizer=Configure.VectorIndex.Quantizer.pq(segments=128, centroids=256),
        )
    ),
    properties=[
        Property(name="doc_id", data_type=DataType.TEXT),
        Property(name="tenant", data_type=DataType.TEXT, index_filterable=True),
    ],
)

Note the caveat: PQ at 128 segments × 256 centroids compresses 768 floats to 128 bytes, but PQ recall degrades faster than scalar quantization, and Weaviate's per-segment HNSW means graph overhead multiplies across shards.

pgvector: Brutally Simple, Painfully Memory-Bound

pgvector is the correct answer up to roughly 10-50M vectors per node. At 1B it becomes an exercise in sharding and Citus-style fan-out. The index build itself is the first wall:

-- maintenance_work_mem must exceed the index size or the build will be
-- catastrophically slow. At 125M x 768d this needs ~40GB+ and hours.
SET maintenance_work_mem = '48GB';
SET max_parallel_maintenance_workers = 8;

CREATE INDEX CONCURRENTLY chunks_embedding_hnsw
ON chunks USING hnsw (embedding vector_cosine_ops)
WITH (m = 32, ef_construction = 256);

And the query path needs explicit recall/latency tuning plus iterative scan, or filtered queries silently return short result sets:

SET hnsw.ef_search = 200;
SET hnsw.iterative_scan = relaxed_order;
SET hnsw.max_scan_tuples = 40000;

EXPLAIN (ANALYZE, BUFFERS)
SELECT id, doc_id, embedding <=> $1 AS distance
FROM chunks
WHERE tenant = $2
ORDER BY embedding <=> $1
LIMIT 50;

Without iterative_scan, a filter that excludes 99% of rows yields fewer than LIMIT results unless you overscan manually. That is the single most common production surprise with pgvector + filters.

The Numbers: A Reference Configuration

Read this as an order-of-magnitude reference, not gospel. These results come from a single-region harness (768d normalized vectors, Recall@10 vs. exact ground truth, concurrency 64, steady state after 1h warmup, on-demand r6i pricing). Your payload size, filter selectivity, and index freshness will move these numbers by 2-3x. Reproduce before you commit budget.

The p99-to-p50 ratio is the story here. All four engines are within ~2x at the median. At p99 the spread is 4.5x. The tail is driven by three things: (1) whether the graph traversal occasionally escapes into a cold page, (2) whether the coordinator has a queue, and (3) whether the engine has a background compaction/merging process that competes for the same memory bandwidth.

The Cost Equation That Survives a Finance Review

Cost per million queries is not "instance price ÷ throughput." It is throughput at your SLO. Model it in exact decimal arithmetic — floating-point errors on six-figure monthly bills are embarrassing.

# infra/cost_model.py
from __future__ import annotations
from dataclasses import dataclass
from decimal import Decimal, ROUND_HALF_UP

@dataclass(frozen=True, slots=True)
class Fleet:
    name: str
    nodes: int
    hourly_usd_per_node: Decimal
    sustained_qps: float      # measured at the SLO, NOT peak
    slo_p99_ms: float
    recall_at_10: float

    @property
    def hourly_usd(self) -> Decimal:
        return self.hourly_usd_per_node * self.nodes

    def cost_per_million_queries(self) -> Decimal:
        if self.sustained_qps <= 0:
            raise ValueError(f"{self.name}: sustained_qps must be > 0")
        qph = Decimal(str(self.sustained_qps)) * Decimal(3600)
        return (self.hourly_usd * Decimal(1_000_000) / qph).quantize(
            Decimal("0.01"), rounding=ROUND_HALF_UP
        )

FLEETS = [
    Fleet("qdrant",   8,  Decimal("1.008"), 1200, 47.0,  0.962),
    Fleet("milvus",  10,  Decimal("1.008"), 1000, 63.0,  0.951),
    Fleet("weaviate", 12, Decimal("1.008"),  900, 92.0,  0.934),
    Fleet("pgvector", 16, Decimal("4.032"),  500, 210.0, 0.948),
]

for f in FLEETS:
    print(f"{f.name:<9} recall={f.recall_at_10:.3f}  p99={f.slo_p99_ms:>5}ms  "
          f"${f.hourly_usd}/hr  ${f.cost_per_million_queries()}/1M queries")

Output from the reference configuration:

A 19x spread between the cheapest and most expensive option — on the same recall band, same hardware family. That gap is entirely explained by one variable: how efficiently the engine keeps vectors out of RAM. Qdrant's int8-in-RAM + float32-on-disk split lets 8 mid-size nodes serve what pgvector needs 6 TiB of RAM to attempt.

Filtered Search Is Where the Model Reverses

The benchmark above uses unfiltered queries. Production does not. Once you add a high-cardinality filter (tenant_id, ACL groups, created_at > now() - 90d), relative rankings shift — often violently:

• pgvector with hnsw.iterative_scan is surprisingly competitive for low-cardinality filters, but degrades to a sequential scan when the filter matches <1% of rows.
• Qdrant uses payload indexes to convert filters into cardinality estimates, then switches between HNSW and filtered full-scan automatically via full_scan_threshold. This is the most robust general-purpose story.
• Milvus partition keys give you physical isolation (best for hard multi-tenancy), but cross-partition filters lose the benefit.
• Weaviate inverted-index fusion is the strongest for arbitrary boolean filters on medium-cardinality fields.

Benchmark your actual filter distribution, not the synthetic one.

The Harness: Measure Recall and p99 in the Same Loop

Measuring them separately is the #1 methodology error. Here is a minimal, correct harness using HDR histograms (never averages — tail latency requires percentiles computed over a histogram):

# bench/harness.py
from __future__ import annotations
import asyncio, time
from dataclasses import dataclass
from typing import Awaitable, Callable
import numpy as np
from hdrh.histogram import HdrHistogram

@dataclass(slots=True)
class RecallLatencyResult:
    qps: float
    p50_ms: int
    p99_ms: int
    recall_at_10: float

async def measure(
    search: Callable[[np.ndarray, int], Awaitable[list[int]]],
    queries: np.ndarray,        # (Q, 768) float32, L2-normalized
    ground_truth: np.ndarray,   # (Q, 10) int64, exact NN ids
    concurrency: int = 64,
    limit: int = 10,
    duration_s: float = 60.0,
) -> RecallLatencyResult:
    hist = HdrHistogram(1, 60_000, 3)
    deadline = time.perf_counter() + duration_s
    q, n = queries.shape[0], int(queries.shape[1])

    async def worker(wid: int) -> tuple[int, int]:
        hits = total = 0
        idx = wid
        while time.perf_counter() < deadline:
            vec = queries[idx % q]
            t0 = time.perf_counter()
            try:
                ids = await search(vec, limit)
            except Exception:
                idx += concurrency
                continue  # count errors separately in production
            hist.record_value(max(1, int((time.perf_counter() - t0) * 1000)))
            gt = set(ground_truth[idx % q].tolist())
            hits += len(gt.intersection(ids))
            total += limit
            idx += concurrency
        return hits, total

    start = time.perf_counter()
    results = await asyncio.gather(*(worker(w) for w in range(concurrency)))
    elapsed = time.perf_counter() - start

    hits = sum(r[0] for r in results)
    total = sum(r[1] for r in results)
    return RecallLatencyResult(
        qps=total / limit / elapsed,
        p50_ms=int(hist.get_value_at_percentile(50.0)),
        p99_ms=int(hist.get_value_at_percentile(99.0)),
        recall_at_10=hits / total if total else 0.0,
    )

Two non-negotiables: (1) generate ground truth with an exact brute-force search (FAISS IndexFlatIP or numpy.argsort), never with the ANN index itself; (2) drive load with an open-loop generator, or you will suffer coordinated omission and report a p99 that is 3-10x too optimistic.

Operational Checklist Before You Commit

• Compute N × dim × bytes plus N × 2M × 4 before any instance sizing conversation.
• Establish your recall target with the product owner; 0.95 vs 0.98 can double your fleet.
• Pin ef_search / hnsw_ef per query class — do not use one global value.
• Measure p99 at the client, behind your real load balancer, not at the server socket.
• Warm the index after every restart; cold HNSW p99 is 10-50x worse for the first ~30s.
• Run a filter-selectivity sweep (100%, 10%, 1%, 0.1%) and record recall at each.
• Model cost per 1M queries with Decimal, including replicas, object storage, and cross-AZ egress.
• Load-test the compaction/merge path — background merges are the most common p99 regression in production.

Pitfalls That Will Bite You

1. Benchmarking with `LIMIT 10` but serving `LIMIT 50`. Latency scales superlinearly with k under quantization + rescoring.
2. PQ everywhere. PQ at 128 bytes is cheap but costs 3-6 recall points. Use scalar int8 with rescoring first; reach for PQ only when RAM truly forces it.
3. Ignoring `m`. Raising M from 16 to 64 quadruples graph memory (+477 GiB at 1B) for typically <2 recall points. It is a bad trade at scale.
4. `Strong` consistency in Milvus. It serializes queries against the WAL and will halve your throughput.
5. Building a pgvector HNSW index without `maintenance_work_mem`. You will wait hours instead of minutes, and concurrent writes will bloat the table mid-build.
6. Forgetting that 1B vectors is a data-engineering problem first. Incremental upsert/delete churn degrades HNSW recall over time in every engine; plan a rebuild or a segment-compaction cadence.

Sources & Technical References

• Malkov, Y. A. & Yashunin, D. A. — Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs, IEEE TPAMI 42(4), 2020. arXiv:1603.09320
• Jégou, H., Douze, M. & Schmid, C. — Product Quantization for Nearest Neighbor Search, IEEE TPAMI 33(1), 2011.
• Subramanya, S. J. et al. — DiskANN: Fast Accurate Billion-point Nearest Neighbor Search on a Single Node, NeurIPS 2019.
• Aumüller, M., Bernhardsson, E. & Faithfull, A. — ANN-Benchmarks: A Benchmarking Tool for Approximate Nearest Neighbor Algorithms, Information Systems 87, 2020. ann-benchmarks.com
• Qdrant — Quantization and Indexing official documentation: qdrant.tech/documentation/guides/quantization and qdrant.tech/documentation/concepts/indexing
• Milvus — Index Types, DiskANN, and Consistency Levels: milvus.io/docs/index.md, milvus.io/docs/disk_index.md
• Weaviate — Vector Index Concepts and PQ/BQ Compression: weaviate.io/developers/weaviate/concepts/vector-index, weaviate.io/developers/weaviate/configuration/compression/pq-compression
• pgvector — Official README (HNSW build parameters, hnsw.ef_search, hnsw.iterative_scan): github.com/pgvector/pgvector
• Timescale — pgvectorscale / StreamingDiskANN: github.com/timescale/pgvectorscale
• Zilliz — VectorDBBench, an open-source, reproducible vector database benchmark suite: github.com/zilliztech/VectorDBBench
• Tene, G. — How NOT to Measure Latency (coordinated omission, HdrHistogram methodology): HdrHistogram.org
• AWS — EC2 R6i Instance Types (memory-optimized pricing used in the cost model): aws.amazon.com/ec2/instance-types/r6i