Skip to main content
Filesystem offloading means using LMCache’s filesystem-backed L2 adapter as a backing store behind the in-memory L1 cache. In operator terms, this is an engine.spec.l2Backend.raw configuration plus a mounted path inside the engine pod. Use this when you want:
  • a simple on-cluster L2 tier without Redis or object storage
  • warm reuse across engine restarts
  • a shared POSIX-style backing store, if your cluster can provide one

​
What this is in the operator model

The chart does not expose a first-class filesystem.enabled=true value. Filesystem offloading is configured through the LMCacheEngine CR passthrough:
  • engine.spec.l2Backend.raw
  • engine.spec.volumes
  • engine.spec.volumeMounts
That is because filesystem L2 is currently treated as a raw adapter rather than a typed operator field like l2Backend.resp.

​
Important distinction: shared vs node-local storage

This is the main thing to get right:
  • if you mount a shared filesystem at the same path on every engine pod, the L2 cache can be shared across nodes
  • if you mount a node-local disk or node-local PVC, the L2 cache only helps that node
So:
  • ReadWriteMany network storage is the correct choice when you want cluster-wide reuse
  • local NVMe, hostPath, or ReadWriteOnce PVCs are still valid, but they behave like node-local offload, not a shared cache

​
Minimal example

This example mounts a PVC at /data/lmcache/l2 and configures the LMCache fs adapter:
my-values.yaml
engine:
  enabled: true
  spec:
    l1:
      sizeGB: 60
    image:
      repository: lmcache/vllm-openai
      tag: v0.4.5
      pullPolicy: IfNotPresent
    l2Backend:
      raw:
        type: fs
        config:
          base_path: /data/lmcache/l2
      storePolicy: default
      prefetchPolicy: default
      prefetchMaxInFlight: 8
    volumes:
      - name: lmcache-l2
        persistentVolumeClaim:
          claimName: lmcache-l2
    volumeMounts:
      - name: lmcache-l2
        mountPath: /data/lmcache/l2
Apply it with: 
helm upgrade --install tensormesh-operator \
  oci://ghcr.io/tensormesh-production/charts/tensormesh-operator \
  --version 0.4.5 \
  -n tensormesh-operator --create-namespace \
  -f my-values.yaml --wait

​
What the fields mean

  • l2Backend.raw.type: fs
    • selects the filesystem-backed L2 adapter
  • l2Backend.raw.config.base_path
    • the directory inside the engine container where LMCache stores L2 files
  • volumes / volumeMounts
    • make that path real and writable inside the engine pod
  • storePolicy: default
    • keep normal L1 behavior and also store evicted/eligible keys into L2
  • prefetchPolicy: default
    • allow misses to load data back from L2 into L1
  • prefetchMaxInFlight
    • caps concurrent L2-to-L1 loads to avoid flooding L1 memory

​
Shared cache across nodes

Use a storage class or backing system that gives you:
  • ReadWriteMany
  • the same mounted path in every engine pod
  • enough throughput for concurrent reads during warm prefills
Examples:
  • NFS
  • EFS / Filestore / Azure Files
  • CephFS
This is the right shape when you want one node to benefit from data another node stored.

​
Fast local spill on each node

Use:
  • local SSD / NVMe
  • hostPath
  • ReadWriteOnce PVC pinned to the node
This is still useful if your main goal is to extend capacity beyond RAM on a single node, but it is not a cross-node shared L2.

​
Optional fs adapter knobs

The fs adapter accepts a few useful extra fields: 
l2Backend:
  raw:
    type: fs
    config:
      base_path: /data/lmcache/l2
      relative_tmp_dir: tmp
      read_ahead_size: 1048576
      use_odirect: false
What they do:
  • relative_tmp_dir
    • subdirectory under base_path for temporary write files
  • read_ahead_size
    • issue a small initial read to encourage filesystem readahead
  • use_odirect
    • bypass the OS page cache; only use this if you understand the alignment/performance tradeoff
If you need adapter-specific features beyond these, pass them through under raw.config. The operator forwards that JSON to LMCache as-is.

​
Persistence and restart behavior

Filesystem L2 is useful because it survives engine pod restarts as long as the mounted storage survives. That means:
  • emptyDir is not a good choice if you want persistence across pod restarts
  • a PVC or durable host-backed path is the right choice if restart survival matters

​
Storage Sizing

Sizing is the next question after “should I use filesystem offloading at all?” The fastest way to pick L1 and L2 capacities is the interactive Cache Sizing guide: paste a workload trace and it simulates LRU reuse distance to show the L1/L2 hit rate you would get at any capacity, measured in characters, tokens, or prefix hashes. To convert those units into bytes of cache, the upstream LMCache KV Cache Size Calculator gives the per-token KV size for a given model.

​
L1 sizing

Use L1 when you want the fastest possible warm-hit behavior. In the operator, L1 is:
  • engine.spec.l1.sizeGB
General guidance:
  • size L1 for the hot working set you expect to be reused frequently
  • if L1 is too small, warm entries will evict before they are reused
  • if L1 comfortably holds the hot set, repeat traffic can stay in memory and avoid L2 reads
To size L1 to your workload’s hot set, use the Cache Sizing simulator and set l1.sizeGB near the knee of the hit-rate curve.

​
L2 sizing

Use L2 when you want capacity beyond RAM or warm reuse across restarts and, if storage is shared, across nodes. General guidance:
  • size L2 for the larger working set that does not fit in L1
  • if L2 is too small, you will churn older entries and lose warm benefit between bursts
  • durable L2 is most useful when prompt reuse is real and frequent enough to pay back the extra I/O
The Cache Sizing simulator shows how much extra hit rate each additional GB of L2 buys for your trace — size L2 to where the curve reaches the hit rate you are willing to pay for.

​
Expected performance by hit rate

The performance you should expect depends heavily on where hits land:
  • mostly L1 hits
    • best TTFT improvement
    • lowest warm-request latency
  • mostly L2 hits
    • still useful, but slower than L1 because data must be loaded back into memory
  • mostly misses
    • little or no cache benefit
To estimate the L1/L2/miss split for your own traffic, simulate it in the Cache Sizing guide.

​
Verification

After rollout: 
kubectl get lmcacheengine -n tensormesh-operator
kubectl describe lmcacheengine -n tensormesh-operator <engine-name>
kubectl get pods -n tensormesh-operator -l app.kubernetes.io/component=cache-engine
kubectl logs -n tensormesh-operator -l app.kubernetes.io/component=cache-engine --tail=200
What to check:
  • the engine pod is Running
  • the mounted path exists and is writable
  • the engine does not fail while parsing --l2-adapter
  • warm requests eventually produce L2-related store/load activity rather than only L1 behavior
If you want a stronger functional check, combine this with a small L1 and repeated long prompts so you can force eviction and subsequent reload from L2.

​
Common mistakes

  • mounting a PVC but forgetting to set l2Backend.raw
  • setting base_path to a directory that is not actually mounted into the pod
  • assuming node-local storage is shared across nodes
  • using emptyDir and expecting data to survive a pod restart
  • mounting anything at /dev/shm
    • this is unrelated to filesystem L2 and can break CUDA IPC for MP mode

​
When to use something else

Choose a RESP backend instead when you want:
  • a clearly shared remote cache tier
  • centralized capacity management
  • auth-managed network storage with fewer filesystem semantics to think about
Choose filesystem offloading when you want:
  • the simplest durable L2 on Kubernetes
  • no Redis dependency
  • direct use of an existing POSIX storage system