Introduction to VeltrixDB
VeltrixDB is an NVMe-native distributed key-value database that keeps values on flash and only a small pointer index in DRAM, trading Redis-style RAM costs and LSM-style write amplification for predictable, disk-backed performance.
What is VeltrixDB
VeltrixDB is a key-value store written in Go, with an optional C++ acceleration layer on Linux, designed around one idea: values belong on NVMe, not in RAM. Instead of an in-memory hash table (Redis) or a log-structured merge tree that rewrites data repeatedly (RocksDB, LevelDB), VeltrixDB separates keys from values using a WiscKey-style design, hashes every key across 8192 shards, and drives up to 8 NVMe disks in parallel with no single hot lock.
It speaks a simple text protocol (usable directly with nc or telnet) and a binary protocol used by its client SDKs, and it is built to run first-class on Kubernetes via a Helm chart and a CRD-based Operator.
Why VeltrixDB
The problem with Redis at scale
Redis keeps everything in RAM. At 1 billion keys with 128-byte values, that's roughly 250 GB of RAM — an r6g.16xlarge-class node costing ~$3,000–5,000/month. Cloud RAM costs 15–20× more per GB than NVMe SSD.
VeltrixDB stores values on NVMe with the index and hot data in DRAM. The same 1 billion keys fit in ~160 GB of NVMe, and three nodes cost roughly $300–500/month.
The problem with LSM trees (RocksDB, LevelDB)
LSM compaction rewrites full key-value records to merge sorted runs, which typically produces 10–30× write amplification — every byte you write eventually lands on disk 10 to 30 times. That burns SSD write endurance and adds latency spikes during heavy compaction.
VeltrixDB uses WiscKey KV-separation: values are written once to an append-only Value Log (VLog) on NVMe and never rewritten. Compaction only garbage-collects dead space in the VLog. Write amplification is approximately 1.0×.
Predictable P99
Redis P99 latency spikes when AOF rewrite runs; RocksDB P99 spikes during compaction. VeltrixDB's three-tier admission-controlled garbage collector enforces a ceiling: GC bandwidth is rate-limited proportional to read latency, and an emergency mode bypasses pauses before garbage can accumulate. In a 30-minute sustained benchmark at 1 billion keys, VeltrixDB recorded zero GC emergency events.
How it works
8192 shards, FNV-1a routing. Each key hashes (FNV-1a & 0x1FFF) into 1 of 8192 shards. With 8 NVMe disks, shard N routes to disk N % 8, and all 8 disks write in parallel.
Values on NVMe, index in DRAM. The in-memory index stores only a 64-byte pointer per key (disk offset, shard, size, TTL, version). Value bytes go directly to the per-disk append-only VLog. A cache hit is a DRAM lookup (~220 ns); a cache miss is one NVMe random read (~400 µs).
Group-commit WAL. A background flusher amortizes fdatasync across all writers within a configurable window (default 10 ms) — one fdatasync per batch instead of per write.
LIRS cache. Scan-resistant eviction means large sequential reads don't evict hot keys; small values (≤256 B) get higher eviction priority.
C++ io_uring on Linux. The optional C++ layer adds io_uring SQPOLL + O_DIRECT VLog reads, an Adaptive Radix Tree (ART) index on 2 MB hugepages, and NUMA-aware thread pinning, shaving roughly 25 µs off the NVMe read P99 on production hardware. The Go layer is fully functional and cross-platform without it.
Benchmark numbers
Measured on a 3-node GKE cluster (n2-highmem-64, 8×375 GB NVMe per node), 1 billion keys, 30 minutes sustained:
| Metric | Value |
|---|---|
| Reads/s (internal GKE run — unverified) | 7,200,000 |
| Writes/s (internal GKE run — unverified) | 1,800,000 |
| P50 (80R/20W blended) | ~220 ns |
| P99 (80R/20W blended) | ~510 µs |
| GC emergency events | 0 |
| Errors | 0 |
| Storage density | ~160 GB for 1B × 128 B values |
Verified vs unverified: the measured, reproducible numbers are the YCSB single-node results (427,697 reads/s · 18,064 durable writes/s · AWS EC2, 4×NVMe, 100M keys). The 3-node GKE figures (7.2M reads/s / 1.8M writes/s) come from an internal run without a published harness — treat them as unverified until reproduced.
Single-node figures on Linux NVMe:
| Operation | P50 | P99 |
|---|---|---|
| GET — cache hit | 0.05 ms | 0.28 ms (~1.4M reads/s) |
| PUT — 10 ms flush window | 5 ms | 10.2 ms |
| PUT — 5 ms window, 512 workers | 2.6 ms | 5.2 ms (~102K writes/s) |
| MultiPut 1024 entries | — | ~9.5 ms (~426K entries/s) |
Write amplification is approximately 1.0× versus the 10–30× typical of LSM-tree engines. Full methodology lives in the project's BENCHMARKING.md.
Features
| Area | Details |
|---|---|
| Storage | WiscKey KV-separation, 8192-shard index, LIRS cache, append-only VLog |
| Durability | Group-commit WAL, fdatasync amortization, crash recovery via WAL replay |
| Atomic ops | CAS, INCR, DECR, SETNX — shard-locked read-modify-write, safe under concurrency |
| Data types | Keys with TTL, hash fields with per-field TTL, namespaces |
| Replication | Raft consensus, async / quorum / strong modes, anti-entropy |
| Transactions | Optimistic MVCC with vector clocks |
| Security | AES-256-GCM at-rest encryption, RBAC, mTLS, append-only audit log |
| Quotas | Per-namespace rate limiting (token bucket) + key-count caps |
| CDC | In-process change data capture, repl-ship for cross-process forwarding |
| Backup | Full + incremental chains, S3 + GCS upload/restore |
| Observability | 60+ Prometheus metrics, web dashboard, liveness/readiness probes |
| Compression | Per-record zstd, 256 B threshold, transparent on read |
| Scrubber | Background CRC32C integrity validation at configurable MB/s |
Who should use VeltrixDB
VeltrixDB is a good fit if you:
- Have a large key-value working set (hundreds of millions to billions of keys) where Redis-class RAM costs are the bottleneck.
- Need predictable P99 latency under sustained write load and can't tolerate LSM-style compaction stalls.
- Are comfortable operating a self-hosted database on Kubernetes with local NVMe, using the provided Helm chart or Operator.
- Primarily need plain key-value, atomic (
CAS/INCR/DECR/SETNX), and transactional operations, optionally replicated via Raft or primary-copy replication.
Who should not use VeltrixDB yet
These are real, current gaps — not hypothetical edge cases:
NSSCAN, or secondary indexes.repl-ship is down. A durable WAL-tail mode is future work.raft mode gives linearizable writes via quorum commit, but reads are served from local applied state — there is no read-index or lease-read path yet.replicated mode is primary-copy replication for durability across copies; it has no single-writer ordering, so concurrent writers to the same key are not linearizable. Use raft mode when write linearizability matters.