> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tensormesh.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Filesystem Offloading

> Use a filesystem-backed L2 cache with the Tensormesh Operator by mounting storage into the engine pod and configuring the raw L2 adapter.

Filesystem offloading means using LMCache's **filesystem-backed L2 adapter** as a backing store
behind the in-memory L1 cache. In operator terms, this is an `engine.spec.l2Backend.raw`
configuration plus a mounted path inside the engine pod.

Use this when you want:

* a simple on-cluster L2 tier without Redis or object storage
* warm reuse across engine restarts
* a shared POSIX-style backing store, if your cluster can provide one

## What this is in the operator model

The chart does **not** expose a first-class `filesystem.enabled=true` value.
Filesystem offloading is configured through the LMCacheEngine CR passthrough:

* `engine.spec.l2Backend.raw`
* `engine.spec.volumes`
* `engine.spec.volumeMounts`

That is because filesystem L2 is currently treated as a **raw adapter** rather than a typed
operator field like `l2Backend.resp`.

## Important distinction: shared vs node-local storage

This is the main thing to get right:

* if you mount a **shared filesystem** at the same path on every engine pod, the L2 cache can be
  shared across nodes
* if you mount a **node-local disk** or node-local PVC, the L2 cache only helps that node

So:

* `ReadWriteMany` network storage is the correct choice when you want cluster-wide reuse
* local NVMe, hostPath, or `ReadWriteOnce` PVCs are still valid, but they behave like
  **node-local offload**, not a shared cache

## Minimal example

This example mounts a PVC at `/data/lmcache/l2` and configures the LMCache `fs` adapter:

```yaml my-values.yaml theme={null}
engine:
  enabled: true
  spec:
    l1:
      sizeGB: 60
    image:
      repository: lmcache/vllm-openai
      tag: v0.4.5
      pullPolicy: IfNotPresent
    l2Backend:
      raw:
        type: fs
        config:
          base_path: /data/lmcache/l2
      storePolicy: default
      prefetchPolicy: default
      prefetchMaxInFlight: 8
    volumes:
      - name: lmcache-l2
        persistentVolumeClaim:
          claimName: lmcache-l2
    volumeMounts:
      - name: lmcache-l2
        mountPath: /data/lmcache/l2
```

Apply it with:

```bash theme={null}
helm upgrade --install tensormesh-operator \
  oci://ghcr.io/tensormesh-production/charts/tensormesh-operator \
  --version 0.4.5 \
  -n tensormesh-operator --create-namespace \
  -f my-values.yaml --wait
```

## What the fields mean

* `l2Backend.raw.type: fs`
  * selects the filesystem-backed L2 adapter
* `l2Backend.raw.config.base_path`
  * the directory inside the engine container where LMCache stores L2 files
* `volumes` / `volumeMounts`
  * make that path real and writable inside the engine pod
* `storePolicy: default`
  * keep normal L1 behavior and also store evicted/eligible keys into L2
* `prefetchPolicy: default`
  * allow misses to load data back from L2 into L1
* `prefetchMaxInFlight`
  * caps concurrent L2-to-L1 loads to avoid flooding L1 memory

## Recommended storage choices

### Shared cache across nodes

Use a storage class or backing system that gives you:

* `ReadWriteMany`
* the same mounted path in every engine pod
* enough throughput for concurrent reads during warm prefills

Examples:

* NFS
* EFS / Filestore / Azure Files
* CephFS

This is the right shape when you want one node to benefit from data another node stored.

### Fast local spill on each node

Use:

* local SSD / NVMe
* hostPath
* `ReadWriteOnce` PVC pinned to the node

This is still useful if your main goal is to extend capacity beyond RAM on a single node, but it is
not a cross-node shared L2.

## Optional `fs` adapter knobs

The `fs` adapter accepts a few useful extra fields:

```yaml theme={null}
l2Backend:
  raw:
    type: fs
    config:
      base_path: /data/lmcache/l2
      relative_tmp_dir: tmp
      read_ahead_size: 1048576
      use_odirect: false
```

What they do:

* `relative_tmp_dir`
  * subdirectory under `base_path` for temporary write files
* `read_ahead_size`
  * issue a small initial read to encourage filesystem readahead
* `use_odirect`
  * bypass the OS page cache; only use this if you understand the alignment/performance tradeoff

If you need adapter-specific features beyond these, pass them through under `raw.config`. The
operator forwards that JSON to LMCache as-is.

## Persistence and restart behavior

Filesystem L2 is useful because it survives engine pod restarts as long as the mounted storage
survives.

That means:

* `emptyDir` is **not** a good choice if you want persistence across pod restarts
* a PVC or durable host-backed path is the right choice if restart survival matters

## Storage Sizing

Sizing is the next question after “should I use filesystem offloading at all?”

The fastest way to pick L1 and L2 capacities is the interactive
**[Cache Sizing](/operator/configuration/cache-sizing)** guide: paste a workload trace and it
simulates LRU reuse distance to show the L1/L2 hit rate you would get at any capacity, measured in
characters, tokens, or prefix hashes.

To convert those units into bytes of cache, the upstream
[LMCache KV Cache Size Calculator](https://docs.lmcache.ai/getting_started/kv_cache_calculator.html)
gives the per-token KV size for a given model.

### L1 sizing

Use L1 when you want the fastest possible warm-hit behavior. In the operator, L1 is:

* `engine.spec.l1.sizeGB`

General guidance:

* size L1 for the hot working set you expect to be reused frequently
* if L1 is too small, warm entries will evict before they are reused
* if L1 comfortably holds the hot set, repeat traffic can stay in memory and avoid L2 reads

To size L1 to your workload's hot set, use the
[Cache Sizing](/operator/configuration/cache-sizing) simulator and set `l1.sizeGB` near the knee of
the hit-rate curve.

### L2 sizing

Use L2 when you want capacity beyond RAM or warm reuse across restarts and, if storage is shared,
across nodes.

General guidance:

* size L2 for the larger working set that does not fit in L1
* if L2 is too small, you will churn older entries and lose warm benefit between bursts
* durable L2 is most useful when prompt reuse is real and frequent enough to pay back the extra I/O

The [Cache Sizing](/operator/configuration/cache-sizing) simulator shows how much extra hit rate each
additional GB of L2 buys for your trace — size L2 to where the curve reaches the hit rate you are
willing to pay for.

### Expected performance by hit rate

The performance you should expect depends heavily on where hits land:

* mostly **L1 hits**
  * best TTFT improvement
  * lowest warm-request latency
* mostly **L2 hits**
  * still useful, but slower than L1 because data must be loaded back into memory
* mostly **misses**
  * little or no cache benefit

To estimate the L1/L2/miss split for your own traffic, simulate it in the
[Cache Sizing](/operator/configuration/cache-sizing) guide.

## Verification

After rollout:

```bash theme={null}
kubectl get lmcacheengine -n tensormesh-operator
kubectl describe lmcacheengine -n tensormesh-operator <engine-name>
kubectl get pods -n tensormesh-operator -l app.kubernetes.io/component=cache-engine
kubectl logs -n tensormesh-operator -l app.kubernetes.io/component=cache-engine --tail=200
```

What to check:

* the engine pod is `Running`
* the mounted path exists and is writable
* the engine does not fail while parsing `--l2-adapter`
* warm requests eventually produce L2-related store/load activity rather than only L1 behavior

If you want a stronger functional check, combine this with a small L1 and repeated long prompts so
you can force eviction and subsequent reload from L2.

## Common mistakes

* mounting a PVC but forgetting to set `l2Backend.raw`
* setting `base_path` to a directory that is not actually mounted into the pod
* assuming node-local storage is shared across nodes
* using `emptyDir` and expecting data to survive a pod restart
* mounting anything at `/dev/shm`
  * this is unrelated to filesystem L2 and can break CUDA IPC for MP mode

## When to use something else

Choose a RESP backend instead when you want:

* a clearly shared remote cache tier
* centralized capacity management
* auth-managed network storage with fewer filesystem semantics to think about

Choose filesystem offloading when you want:

* the simplest durable L2 on Kubernetes
* no Redis dependency
* direct use of an existing POSIX storage system