Skip to main content
CacheBlend is a technique introduced by the Tensormesh team, enabling reuse of precomputed KV caches beyond prefix boundaries. Rather than requiring an exact prefix match, CacheBlend identifies and reuses any contiguous block of previously computed tokens regardless of position, then selectively recomputes the KV values of a small subset of tokens to partially update each reused cache block. CacheBlend is most useful when:
  • you serve RAG workloads where the same documents are retrieved in varying order across requests
  • prompt reuse is non-prefix — shared blocks sit after a per-request preamble, system prompt, or other documents
  • prefix caching alone leaves you re-prefilling content you have already seen

​
Prerequisites

CacheBlend requires the following minimum versions, plus cert-manager:
ComponentMinimum version
Tensormesh Operator helm chart0.5.1-rc1
LMCache Operator (lmcache/lmcache-operator)v0.5.0
LMCache vLLM (lmcache/vllm-openai)v0.5.0
cacheblend-plugin (private image)latest-nightly
cert-managerany recent release

​
Configuration

CacheBlend needs two namespaces:
  • the operator namespace (the Helm release namespace)
  • a workload namespace — PSS-privileged; holds the CacheBlendEngine, its connection ConfigMap, the opt-in vLLM pods, and the cacheblend-plugin private-image pull secret
Pre-create the workload namespace and the pull secret: 
# operator namespace
kubectl create ns tensormesh-operator

# workload namespace: PSS-privileged + private-image pull secret
kubectl create ns cacheblend-workload
kubectl label ns cacheblend-workload pod-security.kubernetes.io/enforce=privileged
kubectl -n cacheblend-workload create secret docker-registry cacheblend-plugin-pull \
  --docker-server=docker.io --docker-username=<user> --docker-password=<token>
Then install with CacheBlend enabled, pointing the engine at the workload namespace:
my-values.yaml
# REQUIRED — the mutating webhook injects the plugin (needs cert-manager).
webhook:
  enabled: true

cacheBlend:
  enabled: true
  name: tensormesh-cacheblend
  # MUST differ from the release namespace and be PSS-privileged.
  namespace: cacheblend-workload
  spec:
    l1:
      sizeGB: 60
    image:
      repository: lmcache/vllm-openai
      tag: <matched-tag>            # same lmcache build as the plugin below
      pullPolicy: IfNotPresent
    injection:
      payloadImage:
        repository: tensormesh/cacheblend-plugin   # PRIVATE
        tag: <matched-tag>
        pullPolicy: IfNotPresent
      imagePullSecrets:
        - name: cacheblend-plugin-pull             # must exist in the workload ns

helm upgrade --install tensormesh-operator \
  oci://ghcr.io/tensormesh-production/charts/tensormesh-operator \
  -n tensormesh-operator --create-namespace \
  -f my-values.yaml --wait

​
Opting a vLLM pod in

CacheBlend attaches to vLLM pods you deploy — the operator does not create them. In the workload namespace, add two things to your vLLM pod template, and launch vLLM args-only ( command: override makes the webhook skip injection): 
metadata:
  labels:
    lmcache.ai/cacheblend-inject: "true"                    # opt in
  annotations:
    lmcache.ai/cacheblend-engine: "tensormesh-cacheblend"   # engine in this namespace
Everything else (image, model, resources, replicas) stays your normal vLLM config. At admission the webhook injects the plugin init container, hostIPC, the CUSTOM attention backend, and the --kv-transfer-config (CBKVConnector) pointing at the node-local engine; blend tunables (cb.check_layer, cb.recomp_ratio) come from the engine config.

​
Helm values reference

ValueTypeDefaultDescription
cacheBlend.enabledboolfalseCreate a CacheBlendEngine CR. Requires webhook.enabled: true + cert-manager.
cacheBlend.namestring""Engine CR name. Empty = <fullname>-cacheblend.
cacheBlend.namespacestring""Namespace for the engine + its connection ConfigMap. Use a non-release, PSS-privileged workload namespace. Empty = release namespace.
cacheBlend.spec.l1.sizeGBint60L1 (host-DRAM) chunk cache size, in GiB.
cacheBlend.spec.imageobjectlmcache/vllm-openaiBlend-engine image (the lmcache server).
cacheBlend.spec.injection.payloadImageobject—PRIVATE cacheblend-plugin image injected as the init container. Must match the engine’s lmcache build.
cacheBlend.spec.injection.imagePullSecretslist[]Pull secret names appended to opt-in pods for the private payload image. The Secret(s) must already exist in the vLLM pod’s namespace.
webhook.enabledbooltrueEnable the CacheBlend mutating webhook. Requires cert-manager.

​
Verification

​
Engine reconciled

kubectl -n cacheblend-workload get cacheblendengine tensormesh-cacheblend
kubectl -n cacheblend-workload get configmap tensormesh-cacheblend-connection
The <engine>-connection ConfigMap is the reconcile proof — and the gate the webhook reads. If it is missing, injection silently fail-opens (the pod runs without CacheBlend).

​
Webhook wired

kubectl get mutatingwebhookconfiguration tensormesh-operator-mutating-webhook \
  -o jsonpath='{.webhooks[0].clientConfig.caBundle}' | head -c 20; echo
A non-empty caBundle means cert-manager issued the serving cert and the webhook can be called.

​
Injection happened

After creating an opt-in vLLM pod in the workload namespace: 
P=$(kubectl -n cacheblend-workload get pod -l lmcache.ai/cacheblend-inject=true -o name | head -1)
kubectl -n cacheblend-workload get $P -o jsonpath='{.metadata.annotations.lmcache\.ai/cacheblend-injected}{"\n"}'  # present = injected
kubectl -n cacheblend-workload get $P -o jsonpath='{.spec.initContainers[*].name}{"\n"}'   # cb-plugin-stage
kubectl -n cacheblend-workload get $P -o jsonpath='{.spec.hostIPC}{"\n"}'                   # true

​
Non-prefix blend actually engaged (optional)

Send two requests that share a block after different-length preambles, then check the engine logs. A shifted retrieve means a chunk was reused at a different position than it was cached — i.e. a non-prefix blend: 
kubectl -n cacheblend-workload logs -l app.kubernetes.io/component=cache-engine --tail=500 \
  | grep -E "\[match_probe\] .* matches=[1-9]|Retrieved pre-computed for [1-9].* shifted=[1-9]"

​
Common mistakes

  • Running the opt-in vLLM pod in the operator (release) namespace — the webhook excludes it, so injection never happens
  • Setting a command: on the vLLM container — the webhook skips injection (appended args would never reach vllm serve)
  • Workload namespace not PSS-privileged — Pod Security rejects the injected hostIPC
  • The cacheblend-plugin-pull secret missing from the workload namespace — the injected init container ImagePullBackOffs (the secret must live where the pod runs, not in the operator namespace)
  • Mismatched cacheblend-plugin and vllm-openai images — vLLM crashes at startup with an ImportError; pin a matched pair
  • webhook.enabled: false or no cert-manager — the engine deploys but nothing injects the plugin