> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tensormesh.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Welcome to Tensormesh Platform

> Self-hosted KV caching for your vLLM fleet on Kubernetes.

**Tensormesh Platform** runs a KV-cache engine on your own Kubernetes cluster to cut
time-to-first-token (TTFT) and free GPU memory. It installs as a Kubernetes operator: a single
controller manages a fleet of cache engines next to your vLLM pods, so repeated and long-context
prompts are served from cache instead of recomputed on the GPU — and that cache survives across
requests, pods, and optionally nodes.

<CardGroup cols={2}>
  <Card title="Getting Started" icon="play" href="/installation/getting-started">
    Preflight your cluster and pick an install method.
  </Card>

  <Card title="End-to-End Example" icon="rocket-launch" href="/installation/example">
    Install → run inference → benchmark the cache win.
  </Card>
</CardGroup>

## Benefits

<CardGroup cols={4}>
  <Card title="Faster prefills" icon="bolt">
    Cached prefixes skip recomputation, so warm requests return their first token sooner.
  </Card>

  <Card title="More GPU headroom" icon="microchip">
    KV cache lives in CPU memory and on disk, freeing HBM for active batches.
  </Card>

  <Card title="Reuse that persists" icon="recycle">
    Cache outlives a single request — across pods, restarts, and optionally across nodes.
  </Card>

  <Card title="Kubernetes-native" icon="dharmachakra">
    One operator, standard CRs, Helm/GitOps install, and metrics through Prometheus.
  </Card>
</CardGroup>

## How caching tiers fit together

Each tier is larger and cheaper than the one above it; the engine promotes hot data upward and
spills cold data downward:

| Tier                      | Where it lives                     | Configured by                                                   | Best for                                           |
| ------------------------- | ---------------------------------- | --------------------------------------------------------------- | -------------------------------------------------- |
| **GPU HBM (L0)**          | vLLM's own paged KV cache          | vLLM                                                            | The active batch                                   |
| **CPU offloading (L1)**   | Host DRAM (system RAM)             | [`engine.spec.l1.sizeGB`](/configuration/cpu-offloading)        | Fast warm reuse beyond what fits on the GPU        |
| **External storage (L2)** | Local disk, NFS, or a remote store | [`engine.spec.l2Backend`](/configuration/filesystem-offloading) | Large working sets and reuse across restarts/nodes |

Start with **[CPU offloading](/configuration/cpu-offloading)** — it's on by default. Add
**[external storage](/configuration/filesystem-offloading)** when your working set outgrows RAM or
must survive restarts.

## What gets deployed

<CardGroup cols={4}>
  <Card title="Operator" icon="gear">
    A controller-manager Deployment that watches the Tensormesh CRs cluster-wide and reconciles
    them into running cache infrastructure.
  </Card>

  <Card title="Cache engine" icon="layer-group">
    A DaemonSet — one cache-engine pod per GPU node — that stores and serves KV cache to the
    vLLM pods on that node.
  </Card>

  <Card title="Coordinator" icon="sitemap">
    A fleet-wide service the engines register with, enabling cross-node cache awareness.
  </Card>

  <Card title="Observability" icon="chart-line">
    An optional OpenTelemetry stack that ships cache metrics and traces to Prometheus and Tempo.
  </Card>
</CardGroup>

## Explore the docs

<CardGroup cols={4}>
  <Card title="Install with Helm" icon="circle-nodes" href="/installation/helm">
    The primary install path, plus upgrades and rollbacks.
  </Card>

  <Card title="CPU offloading" icon="memory" href="/configuration/cpu-offloading">
    The default host-DRAM cache tier.
  </Card>

  <Card title="Metrics" icon="gauge-high" href="/observability/metrics">
    The signals that show whether the cache is reducing recompute.
  </Card>

  <Card title="Operator UI" icon="browser" href="/ui/introduction">
    A console for monitoring your cache fleet, thresholds, and alerts.
  </Card>
</CardGroup>