First general availability of Tensormesh Platform — self-hosted KV caching for your vLLM
fleet, deployed as a Kubernetes operator.Features
- CPU offloading (L1) — KV cache offloaded from GPU memory into host DRAM; on by default,
sized by
engine.spec.l1.sizeGB. - External storage (L2) — optional durable tier behind L1, via a filesystem adapter, Redis, or object storage, with prefetch back into L1.
- CacheBlend (Preview) — non-prefix KV reuse for RAG and shared-block workloads.
- KV cache encryption at rest —
aesgcmencryption of the L2 tier with a master key you supply as a Kubernetes Secret. - P2P KV transfer (Preview) — peer-to-peer KV movement between engine pods.
- Prefill/decode disaggregation (Preview) — split prefill and decode roles across pods.
- Multi-tenancy — per-tenant cache isolation via
cache_salt. - Observability — Prometheus metrics from every cache-engine pod, plus an optional OpenTelemetry Collector path for metrics and traces (Tempo).
tmo-cli— administrator CLI to observe and operateLMCacheEnginefleets.- Operator UI — a read-only console for fleet health, cache capacity, GPU telemetry, thresholds, and alerts.
- NVIDIA Dynamo — offload KV cache from Dynamo vLLM workers to the Tensormesh engine over cross-pod CUDA IPC.

