Helm Chart

The VeltrixDB Helm chart deploys a production-ready cluster on Kubernetes with per-pod local NVMe storage, Prometheus monitoring, and safe rolling updates — optimized for GKE but portable to any cloud with local NVMe instance types.

Prerequisites

gcloud container node-pools create veltrixdb-pool \
  --cluster=my-cluster \
  --zone=us-central1-a \
  --machine-type=n2-highmem-64 \
  --num-nodes=3 \
  --local-ssd-count=8 \
  --local-ssd-interface=NVME \
  --disk-size=100GB

Installation

# Add the chart repo
helm repo add veltrixdb https://charts.veltrixdb.io
helm repo update

# Install with defaults
helm install veltrixdb veltrixdb/veltrixdb \
  --namespace veltrixdb \
  --create-namespace

# Install with custom values
helm install veltrixdb veltrixdb/veltrixdb \
  --namespace veltrixdb \
  --create-namespace \
  --values my-values.yaml

Verify the deployment

# Wait for PVs (3 nodes x 8 disks = 24 volumes)
kubectl get pv | grep local-nvme

# Check pods are running
kubectl get pods -n veltrixdb -l app.kubernetes.io/name=veltrixdb

# Test connectivity
kubectl exec -n veltrixdb veltrixdb-0 -- \
  sh -c 'echo PING | nc 127.0.0.1 9000'
# Expected: PONG

What the chart deploys

Confirmed against the templates in VeltrixDB-Helm-Chart/templates/: statefulset.yaml, service.yaml, service-headless.yaml, servicemonitor.yaml, pdb.yaml, prometheusrule.yaml, configmap.yaml, secret.yaml, rbac.yaml, and _helpers.tpl.

• StatefulSet — VeltrixDB pods with stable per-pod hostnames (podManagementPolicy: Parallel, one volume-claim template per NVMe disk).
• Headless + client Services — the headless service backs gossip/StatefulSet DNS (publishNotReadyAddresses: true so bootstrap peers are reachable pre-readiness); the client service fronts application traffic.
• ServiceMonitor — Prometheus Operator scrape config, one target per pod.
• PodDisruptionBudget — bounds voluntary disruptions during node drains and upgrades.
• PrometheusRule — a bundled Prometheus alerting ruleset (see below).
• ConfigMap / Secret — engine configuration env vars and the cluster bootstrap token.
• RBAC — ServiceAccount plus any Role/RoleBinding the pods need.

Pod Disruption Budget

The chart's default values.yaml sets podDisruptionBudget.maxUnavailable: 1 — at most one pod may be voluntarily disrupted at a time, which for a 3-node, RF-3 cluster keeps at least 2 nodes available during drains and rolling upgrades.

podDisruptionBudget:
  enabled: true
  maxUnavailable: 1

PrometheusRule alerts

The bundled PrometheusRule (rendered when serviceMonitor.enabled: true) ships 24 alert rules grouped into 10 rule groups, spanning write/read/delete latency, VLog GC health, cluster membership, replication, per-disk health, background scrubbing, and Bloom filter efficiency.

The most operationally important two: VeltrixCorruptionDetected fires the moment the background scrubber finds a CRC32C or magic mismatch (severity critical, for: 0m) — treat as an immediate SMART-check-and-resync trigger. VeltrixGCEmergencyRunsSustained fires when VLog garbage has sat above the emergency threshold (65%) for 30 minutes and GC is bypassing admission control just to keep up — it means sustained write rate is exceeding GC throughput and you need to reduce load or add disks.

Configuration reference

Key settings from values.yaml. Full defaults are tuned for GKE n2-highmem-96 nodes (480 GB RAM, 96 vCPU, 8 x 375 GiB local NVMe SSDs).

Performance tuning

For 100K+ writes/s, lower the flush windows (both must stay equal):

storage:
  walFlushWindowMs: 5
  vlogFlushWindowMs: 5

For maximum cache hit rate, size the cache to roughly 85% of node RAM:

cache:
  maxSizeMB: 409600  # 400 GB on a 512 GB node

Security, audit, and data-integrity features

TLS and RBAC auth

auth:
  enabled: true
  configSecret: veltrixdb-auth

tls:
  enabled: true
  existingSecret: veltrixdb-tls

At-rest encryption (AES-256-GCM)

kubectl create secret generic veltrixdb-encryption \
  -n veltrixdb \
  --from-literal=key=$(openssl rand 32 | base64)

security:
  encryptionAtRest: true
  encryptionSecretName: veltrixdb-encryption

Each pod loads the key from the VELTRIXDB_ENCRYPTION_KEY env var at startup; without the secret, startup fails when security.encryptionAtRest is true. Key rotation is offline-only.

Background scrubber

scrubber:
  enabled: true
  mbPerSec: 50

CRC32C re-validation of every VLog record at a controlled rate; default 50 MB/s/disk gives roughly a 2-hour full pass on a 375 GB SSD. Mismatches surface via the VeltrixCorruptionDetected alert.

Raw NVMe block-device VLog

For latency-critical clusters, enable storage.rawVlog.enabled to write the VLog directly to block devices instead of XFS files. WAL and segment files continue to live on the XFS PVCs.

storage:
  rawVlog:
    enabled: true
    devices:
      - /dev/nvme0n1
      - /dev/nvme1n1
      - /dev/nvme2n1
      - /dev/nvme3n1
      - /dev/nvme4n1
      - /dev/nvme5n1
      - /dev/nvme6n1
      - /dev/nvme7n1

Production values.yaml example

replicaCount: 3

image:
  tag: "1.0.0"

storage:
  disksPerNode: 8
  diskSizeGi: 375

cache:
  maxSizeMB: 262144  # 256 GiB - recommended for 480 GiB nodes

storage:
  walFlushWindowMs: 10
  vlogFlushWindowMs: 10

resources:
  requests:
    memory: 480Gi
    cpu: "60"
  limits:
    hugepages-2Mi: 2Gi

autoTune:
  enabled: false

cluster:
  gossipPort: 7946
  replicationFactor: 3

auth:
  enabled: true
  configSecret: veltrixdb-auth

tls:
  existingSecret: veltrixdb-tls

serviceMonitor:
  enabled: true
  interval: 15s

podDisruptionBudget:
  maxUnavailable: 1

Upgrading

helm upgrade veltrixdb veltrixdb/veltrixdb \
  --namespace veltrixdb \
  --values my-values.yaml

The StatefulSet uses RollingUpdate with maxUnavailable: 1, so pods are replaced one at a time. The PodDisruptionBudget bounds voluntary disruptions from node drains that happen concurrently with the rollout.

Validating a deployment

After install or upgrade, validate the cluster end-to-end with the bench.sh harness shipped in the main VeltrixDB repository. It checks two hard gates: packing density (must stay at or below roughly 152 B/record at 128 B values) and GC emergency runs (must stay at 0 under sustained write stress).

kubectl port-forward -n veltrixdb veltrixdb-0 9000:9000 2112:2112 &

git clone https://github.com/VeltrixDB/veltrixdb.git
cd veltrixdb
ADDR=127.0.0.1:9000 METRICS_ADDR=127.0.0.1:2112 \
NUM_KEYS=10000000 \
BULK_DUR=120 STRESS_DUR=300 \
./scripts/bench.sh

Exit 0 means both gates passed; exit 1 means investigate /tmp/veltrixbench-out/run.log. Re-run after every upgrade that bumps image.tag.

Uninstalling

helm uninstall veltrixdb --namespace veltrixdb

PVCs are not deleted automatically, to protect your data:

kubectl delete pvc -n veltrixdb -l app.kubernetes.io/name=veltrixdb

Troubleshooting

Related pages