The 1B-Key Benchmark — methodology & raw data.

If you can't reproduce it, it didn't happen.

Most database benchmarks are marketing in a lab coat. They run on hand-tuned hardware, with cherry-picked workloads, and they never release the harness. We meant to do the opposite, but for the 1B-key run we didn't manage it. See the update above.

This post originally led with three numbers: 4.9ms P99, 1 billion keys and 2M ops/sec per node. None of them has been measured reproducibly, and each is a claim that breaks if you change the hardware or the workload. The numbers on the performance page are the ones we can back with a published run and its conditions.

This article documents the methodology of the internal run: the cluster topology, the workload and the percentile distribution. It is kept for the record and clearly labelled. It then gives the reproducible YCSB measurement, and the loadtest commands you can run against the current server yourself.

If anything below smells like a marketing benchmark, we want to hear about it. Send us your questions — we'll either fix the harness or write a follow-up explaining why a number is the way it is.

Commodity NVMe, no exotic silicon.

We deliberately picked the most boring infrastructure the cloud has to offer. If the numbers required Optane or custom FPGAs, they wouldn't be reproducible — and they wouldn't be cheap.

Internal run, unverified. The cluster ran on Google Cloud N2 instances with local NVMe SSDs. Local NVMe is the only storage class we recommend for production VeltrixDB — EBS, Azure managed disks, and any other network-attached storage will break the latency model entirely. That's not us being precious; it's physics.

Total cluster spend at on-demand pricing: ~$5,800/month. (This post also compared that with a DynamoDB invoice estimate. We have removed it: it was never measured and rested on the unverified throughput figure.)

Realistic, not flattering.

A benchmark that runs only point reads against a fully-cached working set will lie to you. Ours doesn't — we keep the writes coming, the cache pressure real, and the key distribution Zipfian.

We use a 70/30 read/write mix with Zipfian key access (α = 0.99) — the same distribution you see in production session stores, ad-tech profile lookups, and rate-limiter buckets. In that run, a reported ~6% of reads missed the cache and fell through to NVMe (unverified).

Workload profile

• Key space · 1,000,000,000 unique 16-byte keys
• Value size · 128 bytes (Zipfian P95: 256 B) — representative of session/profile blobs
• Read/write mix · 70% GET, 25% SET, 5% DEL
• Access distribution · Zipfian α = 0.99 (heavy tail, realistic)
• Concurrency · 1024 client connections per load host × 4 hosts = 4,096 total
• Protocol · TCP binary, pipelined, no batching tricks
• Warm-up · 30 minutes — long enough for the LIRS cache to converge
• Measurement window · 60 minutes of steady state, sampled at 1Hz

Load generator

The harness is an internal Rust-based generator that talks the VeltrixDB binary protocol natively. It has not been published, which is the main reason this run is unverified; the public load generator is cmd/loadtest (§06). It records every operation's wall-clock latency, the kernel's tcp_rtt at submission, and the cache-hit flag returned by the server. Histograms are HdrHistogram-based and merged across all four load hosts every 10 seconds.

# 1B-key benchmark · 70/30 mix · Zipfian α=0.99
[cluster]
endpoints  = ["10.142.0.11:9000", "10.142.0.12:9000", "10.142.0.13:9000"]
protocol   = "binary"
keepalive  = true

[workload]
key_space     = 1_000_000_000
value_size_b  = 128
distribution  = "zipfian"
zipf_alpha    = 0.99
read_pct      = 70
write_pct     = 25
delete_pct    = 5

[concurrency]
connections   = 1024
pipeline_depth = 16

[run]
warmup_min    = 30
measure_min   = 60
sample_hz     = 1

The full percentile distribution.

We don't publish an average. Averages hide tail latency, and tail latency is where databases earn — or lose — their keep.

Every number in this section comes from the internal GKE run and is unverified. It was never reproduced and the harness is not published, so none of these is a headline figure. As reported, each is the merged percentile across the 60-minute measurement window, sampled at 1Hz.

Cache-hit reads — the easy case

As reported for the internal run (unverified), about 94% of reads never touched NVMe and were served from the LIRS cache in under 0.3 ms end-to-end: P99 cache-hit latency 0.31 ms, P99.9 0.42 ms. For comparison, a cache hit costs about 92 ns inside the engine; the rest is network and client.

Cache-miss reads — the only number that matters

The interesting number is what happens when the LIRS cache misses and we have to go to disk. The internal run reported a cache-miss P99 of 4.9 ms, which is unverified. The original text credited io_uring SQPOLL and 1024 NVMe submission queues for it. That was wrong. A miss is served by one 4 KB-aligned Go pread from the VLog, checked with CRC32C (O_DIRECT on Linux), and the index has 8192 shards.

Writes — the durability path

Group commit amortizes the cost of durability across concurrent writers: one WAL write(2) and one fdatasync per window. The internal run reported a P99 write latency of 0.21 ms (unverified). The window setting for that run was not recorded. The reproducible write numbers in §05 were taken with 5 ms windows and fixed group commit, where a writer waits up to one window for its sync. Since v1.12 the default is adaptive group commit: the 15 ms default window is only an upper bound, and a lone writer is synced at once (with an emulated 300 µs sync on macOS, one writer’s P50 went from 16.0 ms to 0.45 ms).

The run we can stand behind.

These are the headline numbers we can back with a recorded run: a single-node YCSB run on June 9, 2026. It is smaller than the 1B-key run, and every condition is stated. The YCSB 0.17.0 binding it used is not in the repo, so it cannot be re-run from there; §06 lists what can.

This run is historical. It used a pure-Go build, before the off-heap native index, the binary WAL, the one-write(2) WAL flusher and the opt-in C++ network front-end. Those changes shipped later, in v1.5–v1.7.

What this run does not show

Because the whole dataset was in cache, this run says nothing about cache-miss latency from NVMe. It also doesn't cover 1 billion keys or a multi-node cluster. Later measurements, each under its own conditions, are listed on the performance page. On v1.7, server batch writes on macOS (8 clients × 1024-key MPUT, 1M-key space) reached 3.54M keys/s at P99 4.2 ms. On a 4-CPU CI runner with tmpfs and the client on the same host, the Go front-end measured 1.77M keys/s batch writes and 177–182K reads/s.

Run it yourself with the tools in the repo.

The internal 1B-key harness was never published, so that run can't be reproduced. What you can run today is the server plus the repo's own load generator, cmd/loadtest, against your own hardware.

You'll need a Linux host with local NVMe (network-attached disks change the latency picture completely), Go, and a checkout of the VeltrixDB repo. cmd/loadtest is not YCSB, so its numbers are not directly comparable with §05. The YCSB client binding used for §05 is not in the repo; for YCSB workloads A–F use go-ycsb in bench/compare (Step 5).

Step 1 — start a single node

One data directory per NVMe disk, and a cache large enough for your dataset. The example uses the 5 ms windows from §05; the default for both is 15 ms.

$ go run ./cmd/server -addr :9000 \
    -data-dirs /mnt/nvme0,/mnt/nvme1,/mnt/nvme2,/mnt/nvme3 \
    -cache 409600 \
    -wal-flush-window-ms 5 -vlog-flush-window-ms 5

Step 2 — tune the host

A few host-level settings matter:

• Memory maps · on cgo builds (the default) the native index mmaps its arrays, so at large key counts set vm.max_map_count ≥ 262144 (see scripts/sysctl.conf); otherwise inserts can fail with ENOMEM
• Swap · off · the whole key index lives in RAM and must not be paged
• CPU governor · performance on all cores — don't let the kernel race-to-idle on tail latency
• Transparent hugepages · echo madvise > /sys/kernel/mm/transparent_hugepage/enabled

Step 3 — load keys

loadtest writes keys key:0 … key:N-1 for a fixed duration. --batch-size above 1 switches to MPUT, which engages VLog block packing.

$ go run ./cmd/loadtest --addr=127.0.0.1:9000 --mode=write \
    --num-keys=100000000 --value-size=100 \
    --concurrency=200 --duration=600

Step 4 — measure

Then run reads (or --mode=mixed --read-ratio=0.7). --safe-reads reads only keys that have already been written. loadtest prints live stats every second and a latency summary at the end.

$ go run ./cmd/loadtest --addr=127.0.0.1:9000 --mode=read \
    --num-keys=100000000 --value-size=100 \
    --concurrency=200 --duration=60

Step 5 — YCSB A–F and the October tables

These are the commands behind the October 2026 numbers on the performance page. bench/compare is a separate Go module (Go ≥ 1.25); with START=0 it uses a server already running at 127.0.0.1:9000.

# go-ycsb A–F against a running server
$ cd bench/compare && START=0 DBS=veltrixdb RECORDS=20000 OPS=20000 \
    THREADS=16 WORKLOADS="a b c d e f" ./compare.sh

# durable single-key writes, adaptive vs fixed group commit (emulated sync)
$ VELTRIX_GC_TABLE=1 go test ./storage -run TestGroupCommit_LatencyTable -v

# vector recall on GloVe-100 (needs h5py + numpy)
$ scripts/ann-dataset.py glove-100-angular /tmp/ann --train 100000 --test 500
$ VELTRIX_ANN_DIR=/tmp/ann go test ./storage -run TestRealEmbeddings -v -timeout 30m