Server Configuration Reference
Every command-line flag accepted by the veltrixdb server binary (cmd/server), grouped by subsystem, with defaults and behavior confirmed against source.
All flags are standard Go flag package syntax: -flag value, -flag=value, or --flag value are all accepted. Boolean flags can be set bare (-encrypt-at-rest) to mean true. Run veltrixdb -help to print the full built-in flag listing from the running binary, which is always the source of truth if this page and the binary disagree.
Two binaries, one flag surface The
veltrixdb server process (cmd/server/main.go) is the only process that reads these flags at startup. Operator tools like veltrix and the admin API configure themselves separately and talk to a running server over TCP/HTTP.
Network
Flags controlling listen addresses for the data plane (binary/text protocol) and the operator plane (metrics, health, admin API).
| Flag | Default | Description |
|---|---|---|
-addr | :9000 | TCP address for the plaintext client listener (text protocol + binary protocol, auto-detected per connection). |
-metrics-addr | :2112 | HTTP address serving /metrics, /healthz, /readyz, /admin/*, and /traces. This is the "admin port" referenced throughout the docs. /healthz and /readyz answer from the moment the process starts; /metrics, /admin/* and /traces are added once the storage engine has opened (they return 404 before that). |
-tls-addr | :9443 | TCP address for the TLS client listener. Only started when -tls-cert and -tls-key are both set; runs concurrently with the plaintext listener on -addr. |
-net | go | New in v1.7 Network front-end for -addr: go (default; the Go server — every protocol and command, AUTH, all modes) | cpp (C++ event loops in netfront/: io_uring on Linux if available, else poll()) | uring (C++ with io_uring required; fails if unavailable) | poll (C++ with poll(), portable). The C++ front-end serves only the binary protocol's PUT GET DEL PING MPUT MGET, and the server refuses to start with it in -mode raft/replicated or with -auth-config. The text protocol, AUTH, extended binary commands and TLS (-tls-addr) stay on Go. Experimental: in CI measurements it lowered read P50 but had worse read P99 and lower batch-write throughput than go — see Performance Tuning. |
-net-threads | NumCPU | New in v1.7 Number of event loops for -net=cpp|uring|poll (one per core, each with its own SO_REUSEPORT listener). macOS runs a single poll loop. |
-pprof-addr | (empty = off) | New in v1.7 Serves CPU, heap, goroutine and execution-trace profiles (pprof format) on this address, e.g. 127.0.0.1:6060, under /debug/pprof/ (/debug/pprof/profile?seconds=N, /debug/pprof/trace?seconds=N, /debug/pprof/heap, …). Uses its own HTTP mux, never the metrics port — profiles expose internals, so bind it to loopback. |
-pprof-mutex-fraction | 0 (off) | New in v1.16 With -pprof-addr: sample 1 in N mutex contention events for /debug/pprof/mutex (runtime.SetMutexProfileFraction; 5–100 is typical). At 0 that profile stays empty. Ignored (and logged) without -pprof-addr. |
-pprof-block-rate | 0 (off) | New in v1.16 With -pprof-addr: record blocking events of at least this many nanoseconds for /debug/pprof/block (runtime.SetBlockProfileRate; e.g. 10000 = 10 µs). At 0 that profile stays empty. Ignored (and logged) without -pprof-addr. |
-node | node-1 | Unique node ID in the cluster (legacy name). Superseded by -node-id when that is non-empty. |
Storage
Flags controlling data directories, cache sizing, and engine tuning presets.
| Flag | Default | Description |
|---|---|---|
-data | ./veltrixdb-data | Single data directory (WAL + segments). Ignored whenever -data-dirs is set. |
-data-dirs | (empty) | Comma-separated NVMe mount paths, one per physical disk, e.g. -data-dirs=/mnt/nvme0,/mnt/nvme1,...,/mnt/nvme7. Each disk gets its own segment writer and compaction goroutine, and its own WAL and VLog. Overrides -data. |
-raw-vlogs | (empty) | Linux only. Comma-separated raw NVMe block-device paths for VLog backing (e.g. /dev/nvme0n1,/dev/nvme1n1,...), paired index-by-index with -data-dirs. Bypasses XFS for VLog I/O; WAL and segment files remain on -data-dirs. Requires CAP_SYS_RAWIO or root. First open writes a 4 KB superblock at offset 0 of each device. Entry count must exactly match -data-dirs. |
-cache | 256 | LIRS cache size in MB. Ignored when -auto-tune is set (the hardware-derived value wins; this flag is used only if hardware detection fails), and overridden by the -read-heavy preset's 400 GB default unless you explicitly pass a non-default -cache value alongside it. |
-gc-threshold | 0.30 | VLog dead-space ratio (0.0–1.0) that triggers background compaction. Lower values trigger more frequent, smaller GC passes. |
-auto-tune | false | Auto-detects hardware (RAM, CPU cores, disk types) and self-tunes cache size, memtable size, shard count, compaction threads, and SSTable size (the "80% RAM rule"), plus applies OS-level tuning (NVMe I/O scheduler, hugepages, CPU governor). Requires root or CAP_SYS_ADMIN on Linux for the sysctl/rlimit changes; falls back to defaults with a logged warning if hardware detection fails. |
-read-heavy | false | Applies the read-heavy preset: 400 GB cache, LIRS ratio 0.95, 300s GC interval, 60s TTL scan interval. Tuned for P99 < 5 ms at 2M+ ops/sec on n2-highmem-64-class hardware. Overrides DefaultStorageConfig but not -auto-tune or explicit per-flag overrides applied afterward. |
-scrub-mb-per-sec | 50 | Background CRC32C integrity scrubber bandwidth cap, per disk, in MB/s. 0 disables the scrubber entirely. |
-disable-ordered-index | false | Stops maintaining the ordered key index (a skiplist mirroring every live key, ~90 B of RAM per key). Breaking for ordered reads: RANGE, SCANCUR and any RangeScan/ScanCursor caller return ErrOrderedIndexDisabled. Point lookups, namespace prefix scans and every write path are unaffected. Worth it only for point-lookup-only workloads; measured −21.8% allocations and −5.1% wall time on MultiPut-1024. Reallocating the freed RAM to -cache is usually the better use. |
Durability
Flags controlling the write-ahead log, group commit, and point-in-time-recovery (PITR) archiving.
| Flag | Default | Description |
|---|---|---|
-group-commit | adaptive | New in v1.12 How the WAL/VLog flush windows are used. adaptive (default): the window is an upper bound — a lone writer is synced at once, concurrent writers are synced after an idle gap of about one fdatasync. fixed: every batch waits the full window (the behaviour before v1.12). Any other value stops startup. Measured with an emulated 300 µs sync and the 15 ms window: a single writer's Put P50 dropped from 16 ms to 0.45 ms, and 64 writers went from 4.0K to 37.5K durable writes/s (P99 16.8 → 2.6 ms) at the same writes per fdatasync. |
-wal-flush-window-ms | 15 | WAL group-commit flush window in milliseconds (the upper bound in adaptive mode). At the 15 ms default, roughly 200 entries batch per fdatasync at 100K writes/s. Use a smaller window (e.g. 5 ms) for latency-sensitive workloads at the cost of fewer entries per sync. Must match -vlog-flush-window-ms (an enforced invariant). |
-vlog-flush-window-ms | 15 | VLog fdatasync group-commit window in milliseconds. Must equal -wal-flush-window-ms — WAL and VLog flushes race concurrently, and end-to-end P99 is the max of the two. |
-wal-max-batch | 4096 | Maximum WAL entries per group-commit flush before the flush timer is forced to fire early. Raise this if sustained write rate exceeds window × max_batch (e.g., 100K/s × 0.005s = 500, so 4096 gives generous headroom). |
-wal-format | binary | New in v1.6 Encoding for new WAL records: binary (default; 48-byte little-endian header, magic 0xB1, CRC32C over the whole record) or text (legacy). Replay reads both, mixed in one file. text exists only to roll back to a pre-binary build: run once with -wal-format=text, stop cleanly (the shutdown checkpoint rewrites wal.log as text), then downgrade. Fixed in v1.14 a third value, text-legacy, is the rollback mode to v1.13 and older (v1.14 and later, v1.16 included, read the binary WAL directly and need no step — never use text-legacy for them, it only drops TTLs): TTL’d keys’ records now carry their expiry (binary record version 2 / an 11th text field), which older builds cannot read, so text (which keeps that field) is no longer a rollback mode; text-legacy drops TTLs (TTL’d keys become immortal on the older build). The server logs a warning when text is selected, because keys containing | or a newline can lose acknowledged writes on crash replay in that format. Only cmd/server has this flag; veltrixdb-backup and veltrix-repair always write binary. |
-archive-dir | (empty) | Root directory for continuous WAL archiving used by point-in-time recovery. Empty disables archiving. When set, a background archiver copies newly-durable WAL entries into <dir>/disk<N>/. |
-archive-interval-ms | 1000 | Poll interval (ms) for the WAL archiver to check for newly durable entries. |
-archive-max-age-sec | 0 | Prune archived WAL segments older than this many seconds. 0 = keep forever. |
-archive-max-bytes | 0 | Prune the oldest archived segments once total archive size exceeds this many bytes. 0 = unlimited. |
See Backup & Restore for how archived WAL segments combine with a full backup during veltrixdb-backup restore-pitr (an offline restore; take the base backup of a running server with POST /admin/backup).
Security
Flags controlling client TLS, RBAC authentication, the admin-API token, at-rest encryption, and the audit log. Inter-node TLS and the cluster secret are under Cluster / Replication.
| Flag | Default | Description |
|---|---|---|
-tls-cert | (empty) | Path to the TLS server certificate PEM file for client connections. Enables the TLS listener on -tls-addr when set together with -tls-key. |
-tls-key | (empty) | Path to the TLS server private key PEM file. |
-tls-ca | (empty) | Path to a CA certificate for mTLS client verification on the client-facing TLS listener. When set, connecting clients must present a certificate signed by this CA. |
-auth-config | (empty) | Path to an RBAC auth config JSON file (users, password hashes, roles). When set, the AUTH command is required before any operation on that connection. Cannot be combined with -net=cpp|uring|poll (startup fails). When unset, RBAC is disabled and all operations are permitted without credentials. |
-encrypt-at-rest | false | Enables AES-256-GCM encryption of values stored in the VLog. The key is loaded from the VELTRIXDB_ENCRYPTION_KEY environment variable (base64) or from -encryption-key-path. Startup fails fast if the key is missing or not exactly 32 bytes. |
-encryption-key-path | (empty) | Path to a file containing the 32-byte encryption key (raw or base64). Ignored if VELTRIXDB_ENCRYPTION_KEY is set in the environment. |
-audit-log | (empty) | Path to an append-only audit log (JSONL). Empty disables auditing. In v1.12 it covers only single-key PUT and DEL (the flag help also names atomic ops, which v1.12 does not log). Fixed in v1.14 every write path, one record per key — MPUT, pipelined PUTs, transactions, Raft batch apply, and atomic ops as CAS/INCR/DECR/SETNX. Value bytes are never written to the audit log. |
-admin-token | $VELTRIX_ADMIN_TOKEN, else (empty) | v1.1.0 Bearer token guarding every /admin/* endpoint on -metrics-addr. The veltrix CLI and kubectl-veltrix send no token, so they get 401 from a server started with one (use curl with the header, or repl-ship -src-token). When unset, /admin/* accepts loopback connections only (non-loopback gets 403). When set, every /admin/* request — loopback included — must send Authorization: Bearer <token> or X-Admin-Token: <token> or it gets 401; comparison is constant-time. Also readable from the VELTRIX_ADMIN_TOKEN environment variable. /metrics, /healthz, and /readyz are never guarded. See Security. |
Admin API auth is separate from RBAC
-auth-config and -tls-* only protect the client data port (-addr / -tls-addr). The admin/metrics HTTP port (-metrics-addr, default :2112) is guarded separately: /admin/* is loopback-only by default, and -admin-token adds a bearer-token requirement for remote access — see Admin API Reference. Still keep the port behind a network policy on untrusted networks.
Cluster / Replication
Flags controlling deployment mode (standalone, raft, replicated), peer topology, distributed search, and inter-node transport security.
| Flag | Default | Description |
|---|---|---|
-mode | standalone | Deployment mode: standalone | raft | replicated. standalone serves every request from local state (backward compatible, single node). raft commits writes through a Raft log (quorum-committed, linearizable writes; non-leaders answer writes with MOVED <leader-addr> <leader-id>). replicated is primary-copy replication with per-request consistency via -consistency. |
-node-id | (empty) | Node ID for the cluster. Overrides the legacy -node flag when set (non-empty). |
-peers | (empty) | Comma-separated cluster peers: id@host:port,..., where host:port is each peer's client -addr. Every node in the cluster should be listed with the same set of peers. New on main after v1.16 — ships in the next release this is the bootstrap membership: once a membership change was made online, the persisted membership wins over -peers on every restart. |
-join | (empty) | New on main after v1.16 — ships in the next release Join a running cluster instead of bootstrapping one: comma-separated client addresses (host:port) of existing members. The node starts empty and is not a member — in raft mode it never campaigns or votes — until it is added; it asks the members to add it and retries every 2 s. Ignored on a node named in -peers or with persisted membership. Needs -cluster-secret-file or -cluster-mtls. See the membership runbook. |
-advertise-addr | -addr when it names a host | New on main after v1.16 — ships in the next release This node’s client address as other nodes dial it (host:port). Required with -join when -addr has no host (e.g. :9000). |
-force-new-cluster | false | New on main after v1.16 — ships in the next release Dangerous, disaster recovery only. Raft: rewrite this node’s membership to itself alone (a local configuration entry in a new term) so it elects itself after a quorum was lost for good — it commits its whole log, including entries never committed. Replicated: move the persisted cluster_membership.json aside and start from -peers. Remove the flag once the node is up. See Disaster Recovery. |
-seeds | (empty) | Comma-separated bootstrap seed nodes in nodeID=host:port form, used to seed the gossip/partition map at startup (self-entries are skipped automatically). |
-raft-addr | derived | Raft RPC listen address for this node. Empty derives host:(clientPort+2) from -addr. |
-repl-addr | derived | Replication server listen address. Empty derives host:(clientPort+1). |
-gossip-addr | derived | Gossip listener address. Empty derives host:(clientPort+3). |
-transfer-addr | derived | v1.1.0 Partition-transfer (rebalance) HTTP listen address. Empty derives host:(clientPort+5). Used by the TransferAgent to stream keys to their new owners after a ring change; honors the -cluster-tls-* / -cluster-mtls settings when set. |
-linearizable-reads | false | v1.1.0 Raft mode: serve GET and MGET through the ReadIndex fence (quorum-confirmed, never stale). Fixed in v1.14 every read command — namespace, hash, list / set, RANGE / SCANCUR, VER, IDXQUERY, QUERY and searches — goes through it too. Costs one heartbeat round-trip per read; followers redirect to the leader. Logged as having no effect outside -mode raft. Without it (the default), raft-mode reads are local and can be stale on a follower, but since v1.12 a newly elected leader waits until its term's first entry has applied before serving reads, so writes the previous leader acknowledged are visible. |
-rack-id | (empty) | v1.1.0 This node's rack / availability-zone label (e.g. -rack-id=asia-south1-a). When set, partition placement and GetReplicasForKey never place two copies of a partition in the same failure domain — unless the replication factor exceeds the number of distinct racks, in which case placement falls back to distinct nodes while still spanning every rack available. Labels propagate to peers via gossip digests, so each node only needs its own flag. Reported as "rack" per node in the /admin/cluster topology JSON. Unset on all nodes keeps the previous deterministic round-robin placement. |
-auto-rebalance | true | v1.1.0 In distributed modes, automatically rebalance the partition ring and migrate keys to their new owners when nodes join, leave, or fail (3-second debounce per membership-event burst). Set -auto-rebalance=false to keep migration under manual control; the transfer listener still starts. On v1.14 and earlier a node failure triggers a migration that can delete keys, raft mode included — run clusters with -auto-rebalance=false until you upgrade. Fixed in v1.15 replicated mode only: on a membership add or remove, keys are copied to their new RF replicas and a key leaves a node only after every replica acknowledged it; node failure or recovery moves nothing (New on main after v1.16 — ships in the next release membership is added and removed online; on v1.16 and earlier it is static, so a running server never migrates). A logged no-op in -mode raft. See Deployment Modes. |
-raft-pipeline | false | Fixed in v1.15 Raft mode: pipelined replication (env VELTRIX_RAFT_PIPELINE). false: one AppendEntries in flight per follower, sent once per leader fsync. true: up to 8 in flight, sent before the leader's own fsync — faster when every node has its own disk, slower when nodes share one (3 local nodes on one laptop: ~560 vs ~1,040 inserts/s). Durability is the same. Off until measured on separate hosts. Logged as having no effect outside -mode raft. See Cluster & Consensus. |
-raft-snapshot-threshold | 8192 | New on main after v1.16 — ships in the next release Raft mode: retained log entries that trigger a state-machine snapshot and log compaction (env VELTRIX_RAFT_SNAPSHOT_THRESHOLD; 0 = never). Snapshots are streamed to raft_snapshot.dat, have no size cap and are captured without stalling writes. v1.16 and earlier fix the threshold at 8192 and skip snapshots over 256 MB. See Cluster & Consensus. |
-raft-snapshot-chunk-kb | 2048 | New on main after v1.16 — ships in the next release Raft mode: InstallSnapshot chunk size in KiB (env VELTRIX_RAFT_SNAPSHOT_CHUNK_KB). |
-search-fanout | true | New in v1.11 In raft / replicated mode, run every vector / text / hybrid search (and QUERY, IDXQUERY) on all non-failed nodes and merge the results — needed once a rebalance has partitioned the keys. false = local only. Node-to-node search traffic uses the transfer listener (-transfer-addr). |
-search-timeout-ms | 2000 | New in v1.11 Per-peer timeout for one distributed search phase (BM25 has two). A peer that does not answer in time fails the request and is named in the error, unless -search-allow-partial. |
-search-allow-partial | false | New in v1.11 Return a distributed search / QUERY / IDXQUERY result without the peers that failed or timed out (logged). Default: fail the request and name them. Also lets searches answer while the startup index rebuild is still running (otherwise refused; see search_ready in INFO). |
-cluster-secret-file | (empty) | New in v1.11 File holding the shared secret that signs (HMAC-SHA256) every request on the transfer listener: key migration and distributed search (New on main after v1.16 — ships in the next release and membership changes: -join, forwarded changes and every replicated-mode change, which are refused without a secret or -cluster-mtls). Same file on every node; surrounding whitespace is trimmed and a secret shorter than 16 bytes stops startup. When empty, env VELTRIXDB_CLUSTER_SECRET is used. With neither a secret nor -cluster-mtls, the server logs that the transfer listener is unauthenticated — anyone who can reach it can write keys and run searches. |
-consistency | eventual | replicated-mode write consistency: eventual (ACK after the local write, async replication) | quorum (ACK after a majority of replicas apply the write) | strong (ACK after all replicas apply it). quorum/strong return a clear timeout/quorum-not-reached error when the required copies are unreachable. Ignored in standalone and raft modes. |
-cluster-tls-cert | (empty) | PEM certificate for inter-node (Raft/replication) TLS. With -cluster-tls-key it also puts the transfer listener (rebalance and distributed search) on TLS. |
-cluster-tls-key | (empty) | PEM key for inter-node TLS. |
-cluster-tls-ca | (empty) | PEM CA bundle for inter-node TLS peer verification. |
-cluster-mtls | false | Requires and verifies peer client certificates (mutual TLS) for inter-node traffic, including the transfer listener. |
For the semantics of each mode's consistency guarantees — and the remaining node-local caveats (replicated-mode secondary-index metadata; QUERY/RANGE/SCANCUR reads) now that namespace, hash-field, vector, and list/set writes route through the coordinator in all modes — see Deployment Modes.
Environment variables
These are read by the server process at startup (and, for the storage ones, by any process that embeds the storage engine). They are not flags.
| Variable | Default | Description |
|---|---|---|
VELTRIX_ADMIN_TOKEN | (unset) | Default value of -admin-token (the flag wins when passed). Also the default of repl-ship -src-token. |
VELTRIX_RAFT_PIPELINE | (unset = false) | Fixed in v1.15 Default value of -raft-pipeline (the flag wins when passed). |
VELTRIX_RAFT_SNAPSHOT_THRESHOLD | (unset = 8192) | New on main after v1.16 — ships in the next release Default value of -raft-snapshot-threshold. |
VELTRIX_RAFT_SNAPSHOT_CHUNK_KB | (unset = 2048) | New on main after v1.16 — ships in the next release Default value of -raft-snapshot-chunk-kb. |
VELTRIXDB_CLUSTER_SECRET | (unset) | New in v1.11 Shared transfer-listener secret, used when -cluster-secret-file is empty (at least 16 bytes). |
VELTRIXDB_ENCRYPTION_KEY | (unset) | Base64 32-byte key for -encrypt-at-rest. Takes precedence over -encryption-key-path. |
VELTRIXDB_INDEX | native on cgo builds | New in v1.6 Key-index implementation. Unset: the off-heap C++ native index on every cgo build (the official Docker image included), the Go map on CGO_ENABLED=0 builds. map opts out; native forces it (when compiled in) even if VELTRIXDB_DISABLE_CGO_ENGINE is set. At large key counts the native index needs vm.max_map_count ≥ 262144 (see scripts/sysctl.conf) or inserts can fail with ENOMEM. The chosen implementation is reported in the startup log. |
VELTRIXDB_URING_BRIDGE | off | New in v1.7 io_uring VLog write bridge on a Linux cgo build: off (default; pwrite + fdatasync) | on (io_uring, no SQPOLL) | sqpoll (io_uring with an SQPOLL kernel thread per ring). Opt in only after measuring on your hardware; unknown values are logged and treated as off. |
VELTRIXDB_DISABLE_CGO_ENGINE | (unset) | Set to 1/true to skip the C++ layer on a cgo build: forces the io_uring bridge off, does not construct the C++ batch engine, and selects the Go map index (unless VELTRIXDB_INDEX=native). No effect on CGO_ENABLED=0 builds. |
VELTRIXDB_NET_DEFER_READS | (unset = on) | New in v1.7 C++ front-end only. 0 answers every GET/MGET inline on the loop thread, including keys that need a VLog read (the older behaviour, for A/B comparison). By default such keys are finished in a goroutine while the connection waits. |
AWS_*, GCS_ACCESS_TOKEN, GOOGLE_APPLICATION_CREDENTIALS, AZURE_STORAGE_* | (unset) | Cloud credentials used by backup upload/download when not given explicitly. |
All other operational surfaces (backup, restore, chaos testing, migration, load testing) are separate binaries documented in the CLI Reference, not cmd/server flags.
Production example: 8-disk Raft deployment
The following starts one node of a 3-node Raft cluster on an 8-NVMe-disk host, combining storage, durability, security, and cluster flags. Run the equivalent command on each of the three hosts, changing only -node-id and the bind address portion of -addr.
veltrixdb \
-mode raft \
-node-id n1 \
-addr 10.0.1.10:9000 \
-metrics-addr 10.0.1.10:2112 \
-peers n1@10.0.1.10:9000,n2@10.0.1.11:9000,n3@10.0.1.12:9000 \
-data-dirs /mnt/nvme0,/mnt/nvme1,/mnt/nvme2,/mnt/nvme3,/mnt/nvme4,/mnt/nvme5,/mnt/nvme6,/mnt/nvme7 \
-cache 65536 \
-wal-flush-window-ms 15 \
-vlog-flush-window-ms 15 \
-gc-threshold 0.30 \
-scrub-mb-per-sec 50 \
-encrypt-at-rest \
-encryption-key-path /etc/veltrixdb/encryption.key \
-tls-cert /etc/veltrixdb/tls/server.crt \
-tls-key /etc/veltrixdb/tls/server.key \
-auth-config /etc/veltrixdb/auth.json \
-audit-log /var/log/veltrixdb/audit.jsonl \
-cluster-tls-cert /etc/veltrixdb/cluster/node.crt \
-cluster-tls-key /etc/veltrixdb/cluster/node.key \
-cluster-tls-ca /etc/veltrixdb/cluster/ca.crt \
-cluster-mtls \
-cluster-secret-file /etc/veltrixdb/cluster/secret \
-archive-dir /mnt/wal-archive \
-archive-max-age-sec 604800Notes on this configuration: -cache 65536 allocates a 64 GB LIRS cache; -wal-flush-window-ms and -vlog-flush-window-ms are kept equal per the durability invariant; -cluster-mtls requires every peer to present a certificate signed by -cluster-tls-ca, and -cluster-secret-file additionally signs transfer-listener requests (the same file on all three nodes); -group-commit is left at its adaptive default; and -archive-max-age-sec 604800 retains seven days of WAL archive for point-in-time recovery.