Introduction to VeltrixDB

VeltrixDB is an NVMe-native distributed key-value database that keeps values on flash and only a small pointer index in DRAM, trading Redis-style RAM costs and LSM-style write amplification for predictable, disk-backed performance.

What is VeltrixDB

VeltrixDB is a key-value store written in Go, with optional C++ components on cgo builds, designed around one idea: values belong on NVMe, not in RAM. Instead of an in-memory hash table (Redis) or a log-structured merge tree that rewrites data repeatedly (RocksDB, LevelDB), VeltrixDB separates keys from values using a WiscKey-style design, hashes every key across 8192 shards, and drives every NVMe disk you give it in parallel (one WAL and one VLog per disk) with no single hot lock.

The same database serves vector, full-text and hybrid search over the records it stores: HNSW vector indexes, a BM25 inverted index, and reciprocal-rank fusion of the two, filtered by the KV record with the same id v1.11–v1.12.

It speaks a simple text protocol (usable directly with nc or telnet) and a binary protocol used by its client SDKs, and it is built to run first-class on Kubernetes via a Helm chart and a CRD-based Operator (both maintained in separate repositories).

Why VeltrixDB

The problem with Redis at scale

Redis keeps every value in RAM, plus its own per-key overhead, and cloud RAM costs far more per GB than NVMe SSD.

VeltrixDB stores values on NVMe and keeps the index (and whatever cache you configure) in DRAM: about 142 B/key with the native index (measured at 5M keys), plus ~90 B/key for the ordered index unless --disable-ordered-index. For 1 billion keys × 1 KB values that is 142–232 GB of RAM plus ~1.4 TB of NVMe, against over 1 TB of RAM for an in-memory store — roughly 4–7× less RAM. With 128-byte values the index is about as large as the values, so the saving grows with value size. This is per-key arithmetic, not a billion-key benchmark.

The problem with LSM trees (RocksDB, LevelDB)

LSM compaction rewrites full key-value records to merge sorted runs, which typically produces 10–30× write amplification — every byte you write eventually lands on disk 10 to 30 times. That burns SSD write endurance and adds latency spikes during heavy compaction.

VeltrixDB uses WiscKey KV-separation: values are written once to an append-only Value Log (VLog) on NVMe, and there is no LSM compaction. GC only copies still-live values out of mostly-dead VLog regions, so write amplification is about 1× by design (not a measured figure).

Predictable P99

Redis P99 latency spikes when AOF rewrite runs; RocksDB P99 spikes during compaction. VeltrixDB's three-tier admission-controlled garbage collector enforces a ceiling: GC bandwidth is rate-limited proportional to read latency, and an emergency mode bypasses pauses before garbage can accumulate. In the YCSB run below there were zero failed writes across the 100M-key load and zero NOT_FOUND across 10M reads; veltrixdb_vlog_gc_emergency_runs_total is the counter to watch for GC falling behind.

How it works

8192 shards, FNV-1a routing. Each key hashes (FNV-1a & 0x1FFF) into 1 of 8192 shards. With 8 NVMe disks, shard N routes to disk N % 8, and all 8 disks write in parallel.

Values on NVMe, index in DRAM. The in-memory index holds a 64-byte record per key (key hash, disk offset, VLog/segment id, sizes, write timestamp, TTL, CRC32C, shard, flags, schema version — no value bytes) — about 142 B/key of RAM with the native index (measured at 5M keys), plus ~90 B/key for the ordered index unless it is disabled. Value bytes go directly to the per-disk append-only VLog. A cache hit is a DRAM lookup (~92 ns in-process, with the sharded LIRS cache); a cache miss is one NVMe random read (a Go pread, CRC32C-verified). On cgo builds the index tables live off the Go heap in C++, so a large index does not slow Go garbage collection.

Adaptive group-commit WAL. New in v1.12 A background flusher amortizes fdatasync across writers — one write(2) and one fdatasync per batch instead of per write. The window (default 15 ms) is only an upper bound: a lone writer is synced immediately, and concurrent writers are synced together after an idle gap of about one fdatasync; --group-commit=fixed restores the full-window wait. Every write is acknowledged only after the sync that covers it. Records are binary with a CRC32C over the whole record New in v1.6; replay still reads legacy text WALs.

LIRS cache. Scan-resistant eviction means large sequential reads don't evict hot keys; small values (≤256 B) get a higher retention priority, so large cold values are evicted first. The cache is split into up to 256 independently locked shards.

Vector, full-text and hybrid search. New in v1.11 HNSW vector indexes per namespace (float32, int8 or product-quantized; graph on the heap or, from v1.12, layer-0 edges in a mapped file), a BM25 inverted index, and hybrid search fused by reciprocal rank. Vectors and documents persist as ordinary keys; the indexes are rebuilt from them at startup, and in a cluster every search fans out to all nodes. See Vector & Hybrid Search.

C++ components (cgo builds). New in v1.6 With CGO_ENABLED=1 (the default for the Docker image) the index moves off the Go heap into C++ shard tables — with 5M keys a full GC went from 21 ms to 0.27 ms and RSS from 168 to 142 B/key. An opt-in io_uring bridge (VELTRIXDB_URING_BRIDGE=on|sqpoll, Linux, default off) can submit VLog batch writes. --net=cpp|uring|poll swaps the Go listener for an experimental C++ event-loop front-end (binary PUT GET DEL PING MPUT MGET, standalone mode only) that is not faster overall. The VLog read path and the LIRS cache stay in Go; the ART index and priority io_uring scheduler under cpp/ have no Go call site and do not run. The Go layer is fully functional and cross-platform without any of it (CGO_ENABLED=0).

Benchmark numbers

The reproducible headline numbers come from YCSB 0.17.0 on a single node (June 9, 2026; AWS EC2 high-memory instance, 4× NVMe raw block VLog, 400 GB LIRS cache, 200 client threads, 100M keys × ~100 B values). Full setup and raw output are in the project's BENCHMARK_RESULTS.md; reproduce with scripts/bench.sh.

Historical single-node figures (pure-Go path, fixed group-commit window, mixed provenance):

New in v1.12 Adaptive group commit, measured on a macOS laptop with an emulated 300 µs fdatasync: one writer's PUT P50 went from 16 ms (fixed window) to 0.45 ms; 64 writers from 4.0K to 37.5K writes/s (P99 2.6 ms). Vector search on GloVe-100 (100K real word vectors, in-process, one query at a time, macOS) reached recall@10 0.906 at ef = 128 with p50 0.42 ms; the full tables are on the search page.

Later server batch writes (8 clients × 1024-key MPUT): 3.54M keys/s, P99 4.2 ms on macOS with a 1M-key space, and 1.77M keys/s, P99 12.2–12.7 ms on a 4-CPU CI runner (tmpfs, same-host client). Write amplification is approximately 1.0× by design (values are written once), versus the 10–30× typical of LSM-tree engines. Full methodology lives in the project's BENCHMARKING.md.

Features

Who should use VeltrixDB

VeltrixDB is a good fit if you:

• Have a large key-value working set (hundreds of millions to billions of keys) where Redis-class RAM costs are the bottleneck.
• Need predictable P99 latency under sustained write load and can't tolerate LSM-style compaction stalls.
• Are comfortable operating a self-hosted database on Kubernetes with local NVMe, using the provided Helm chart or Operator.
• Primarily need plain key-value, atomic (CAS/INCR/DECR/SETNX), and transactional operations, optionally replicated via Raft or primary-copy replication.
• Want vector, text or hybrid search over the same records without running a separate search system, at the scale it has been tested to (100K real vectors; 10M+ is untested).

Who should not use VeltrixDB yet

These are real, current gaps — not hypothetical edge cases: