Performance Tuning

VeltrixDB's throughput and latency are governed by a small number of knobs: group commit and the WAL/VLog flush window, cache size, batching, and admission control; vector search adds its own (beam width, quantization, graph placement). This page summarizes the tuning guidance from the project's performance guide (PERFORMANCE.md) and docs/vector-search.md, including the index, C++ storage and network switches added in v1.6–v1.7 and the group-commit and vector knobs of v1.11–v1.12.

The most important knob: group commit Adaptive since v1.12

The WAL and the VLog both use group commit: concurrent writes share one fdatasync, and every write is acknowledged only after the sync that covers it.

Adaptive (default). A writer that is alone (one request in the batch and an EWMA of fewer than 1.5 requests per recent flush) is synced at once, so its latency is about one fdatasync. When writers are concurrent, a batch stays open until no new write has arrived for an idle gap equal to the EWMA of recent fdatasync times (clamped to 20 µs–2 ms), or the window expires, or the --wal-max-batch cap is reached (storage/group_commit.go). Measured with an emulated 300 µs device sync and the 15 ms window on macOS:

Batches are as large as in fixed mode (64 writes per sync at 64 writers), so adaptive does not cost more IOPS under load. With a 2 ms sync (slow or network disk) 64 writers go 3,541 → 11,691 writes/s. Reproduce with VELTRIX_GC_TABLE=1 go test ./storage -run TestGroupCommit_LatencyTable -v; full table and conditions in Benchmarking. In adaptive mode the default 15 ms window is fine for almost everything.

Fixed (--group-commit=fixed). Only useful if you depend on its exact batching. Each writer waits up to one window, so:

writes/s ~ goroutines x (1000 / window_ms)      # fixed mode

P99 figures in the fixed table are Linux NVMe. Do not size against a macOS measurement: on Darwin the engine calls plain fsync(2), which returns at the drive cache in ~0.02 ms without flushing it (the adaptive table above emulates the sync cost for that reason).

./veltrixdb --group-commit=fixed --wal-flush-window-ms 5 --vlog-flush-window-ms 5

Batch operations

Always prefer MultiPut/MultiGet over individual PUT/GET calls when operating on multiple keys — they share a single network round trip.

The MultiPut row is historical (measured before the one-write(2)-per-batch WAL flusher and the ordered-index fix). Current server numbers for 8 clients × 1024-key MPUT: 3.54M keys/s, P99 4.2 ms over a 1M-key space on macOS (1.45M keys/s, P99 9.6 ms before the ordered-index fix), and 1.77M keys/s, P99 12.2–12.7 ms on a 4-CPU CI runner (tmpfs, client on the same host, Go front-end, C++ storage off). New in v1.7

The binary-protocol handler coalesces pipelined PUT frames that are already buffered on one connection into a MultiPut (tryCoalescePuts, cmd/server/main.go), so pipelining binary clients get batch performance without code changes. Text-protocol PUTs, and a client that waits for each reply, are not coalesced.

Disk density and packing

Batched writes pack multiple records per 4 KB VLog block, which materially changes on-disk density at small value sizes:

Packing is automatic for MultiPut and pipelined binary PUTs. A single unbatched Put uses the unpacked lock-free path instead, which pads each record to a full 4 KB block. You can verify actual packing efficiency with veltrixdb_vlog_file_bytes / veltrixdb_storage_writes_total ~ value_size + 24.

Cache tuning

-cache 65536      # 64 GB
--auto-tune       # cache = 85% of an 80%-of-RAM budget (~68% of RAM); ignores -cache
--read-heavy      # 400 GB cache, LIRRatio 0.95, GC every 300 s, TTL scan every 60 s (-cache still wins)

--read-heavy is a preset for workloads dominated by reads where you can afford more RAM and want GC to run less often. An explicit -cache overrides its cache size.

Check cache effectiveness with veltrixdb_cache_hits_total / (hits + misses). Small values (≤256 B) resist eviction more aggressively under the LIRS policy, so workloads with many small values benefit the most from a larger cache.

Index memory and Go GC New in v1.6

On cgo builds (the default, including macOS and the official Docker image) the key index lives off the Go heap in per-shard C++ tables, so the Go garbage collector does not scan it. Measured with 5M keys: full GC 21 ms → 0.27 ms, settled RSS 168 → 142 B/key, at a cost of ~19 ns more per cache-miss lookup. Watch veltrixdb_storage_native_index_bytes: that memory is not in go_memstats_*.

VELTRIXDB_INDEX=map ./veltrixdb ...      # opt out: Go map index (also what CGO_ENABLED=0 builds use)
sudo sysctl -p scripts/sysctl.conf       # includes vm.max_map_count = 262144

Large key counts need vm.max_map_count ≥ 262144 (a large engine holds ~25K memory mappings); below that an insert can fail with ENOMEM.

C++ storage switches New in v1.7

There are no --cgo-batch-engine, --numa-aware or --sqpoll-reader flags; passing one makes the server exit with "flag provided but not defined". Several C++ components in cpp/ — the priority io_uring scheduler, the SQPOLL VLog reader, the ART index, the hugepage slab allocator, mlockall, the eBPF GC throttle — compile but have no Go caller, so hugepage, fixed-buffer or SQPOLL-reader settings have no effect on the server. The read path is Go pread (4 KB-aligned, magic + CRC32C check).

Network front-end New in v1.7

--net=go (default) serves every protocol and command. --net=cpp|uring|poll is an opt-in, experimental C++ event-loop front-end (--net-threads, default one loop per core) that serves only the binary PUT GET DEL PING MPUT MGET commands, in standalone mode without --auth-config. Keys that need a VLog read are finished off the loop in a goroutine (VELTRIXDB_NET_DEFER_READS=0 restores the older inline behaviour for comparison). Load-test it with loadtest --proto=binary.

It is not faster overall. On a 4-CPU CI runner (tmpfs, client on the same host, measured 2026-09-25 before disk reads were moved off the loop): read P50 154 vs 286 µs and poll read throughput 194K vs 177K ops/s, but read P99 ~3.0 vs 1.8 ms and batch writes ~12% lower; io_uring did not beat poll. Compare on your own hardware with scripts/net-bench.sh (see Benchmarking). The server logs a per-stage latency breakdown of the C++ front-end at shutdown. Do not judge network changes on macOS: loopback caps round trips at ~60K/s for the Go and C++ front-ends alike.

Profiling New in v1.7

./veltrixdb --pprof-addr 127.0.0.1:6060   # CPU / heap / trace profiles; separate listener, off by default
go tool pprof http://127.0.0.1:6060/debug/pprof/profile?seconds=30

Admission control

VeltrixDB throttles background GC and, at the extreme, foreground writes based on an exponentially-weighted moving average (EWMA) of read latency, to keep GC from competing with foreground reads for NVMe bandwidth:

The GC bandwidth cap deliberately fires before the full pause, so GC is throttled progressively rather than stopped dead. If veltrixdb_storage_write_admission_throttles_total is rising, read EWMA has crossed 20 ms and writes are being intentionally slowed to protect read latency. Diagnose the underlying cause with veltrixdb_storage_read_latency_seconds before changing any other knob — raising the cache size or flush window will not help if the disk itself is the bottleneck.

Vector search tuning New in v1.11–v1.12

Commands and the data model are in Vector & Hybrid Search. The knobs, with their effect on GloVe-100 (first 100K words, 100-dim, recall@10 / p50, in-process, one query at a time, macOS):

Choosing a layout. Measured Go heap per 768-dim vector (10K synthetic clustered vectors, macOS, VELTRIX_VECTOR_MEMORY=1 go test ./storage -run TestVectorMemoryTable -v): float32 3,383 B, int8 1,078 B, pq (m = 96) 493 B, pq + disk graph 378 B heap + 216 B mapped. At 100-dim the quantized layouts are slower per query than float32 (they re-rank from the VLog) and save less, because graph edges dominate; they pay off at high dimension or large counts. Start with int8; use pq + GRAPH disk when RAM is the constraint and a few points of recall are acceptable, then raise EF or PQM to win recall back.

Sizing estimate (heap, N vectors of dimension d; the constant is graph edges and Go per-node overhead derived from the 768-dim measurement, so treat it as an estimate for other dimensions): float32 ≈ N × (4d + ~310 B), int8 ≈ N × (d + ~310 B), pq ≈ N × (m + ~400 B), pq + disk ≈ N × (m + ~280 B) plus 132 B per vector in the mapped file (up to 2× that while the file grows). Tested up to 100K real vectors and 10K at 768-dim; nothing at 10M+ yet.

Benchmarks and reproduce commands: Benchmarking → Vector search.

Common problems