Non-Prefix KV Caching

CacheBlend is a technique introduced by the Tensormesh team that reuses precomputed KV caches beyond prefix boundaries. Rather than requiring an exact prefix match, it reuses any contiguous block of previously computed tokens regardless of position, then selectively recomputes a small subset of tokens to partially update each reused block. CacheBlend is most useful when:

prompt reuse is non-prefix — shared blocks sit after a per-request preamble, system prompt, or other documents
you serve RAG workloads where the same documents are retrieved in varying order across requests

Prerequisites

Component	Minimum version
Tensormesh Operator helm chart	`0.5.3`
LMCache Operator (`lmcache/lmcache-operator`)	`v0.5.3`
LMCache vLLM (`lmcache/vllm-openai`)	`v0.5.3`
cacheblend-plugin (`private image`)	`v0.5.3`
cert-manager	any recent release

Install Chart

Install the chart with CacheBlend enabled. This creates the CacheBlendEngine — a per-node DaemonSet — in the tensormesh-operator namespace:

my-values.yaml

# REQUIRED — the mutating webhook injects the plugin (needs cert-manager).
webhook:
  enabled: true

cacheBlend:
  enabled: true
  name: tensormesh-cacheblend        # engine created in the release namespace
  spec:
    l1:
      sizeGB: 200
    image:
      repository: lmcache/vllm-openai
      tag: <matched-tag>             # same lmcache build as the plugin below
      pullPolicy: IfNotPresent
    injection:
      payloadImage:
        repository: tensormesh/cacheblend-plugin   # PRIVATE
        tag: <matched-tag>
        pullPolicy: IfNotPresent
      imagePullSecrets:
        - name: cacheblend-plugin-pull             # must exist in each workload ns

helm upgrade --install tensormesh-operator \
  oci://ghcr.io/tensormesh-production/charts/tensormesh-operator \
  -n tensormesh-operator --create-namespace \
  -f my-values.yaml --wait

Opting a vLLM pod in

CacheBlend plugin attaches to vLLM pods you deploy — the operator does not create them. First create the workload namespace (PSS-privileged) and the plugin pull secret — the private init container is pulled in the pod’s namespace, so the secret must live here:

kubectl create ns cacheblend-workload
kubectl label ns cacheblend-workload pod-security.kubernetes.io/enforce=privileged
kubectl -n cacheblend-workload create secret docker-registry cacheblend-plugin-pull \
  --docker-server=docker.io --docker-username=<user> --docker-password=<token>

Then add a label + annotation to your pod template and launch vLLM args-only (a command: override makes the webhook skip injection):

metadata:
  labels:
    lmcache.ai/cacheblend-inject: "true"                                        # opt in
  annotations:
    lmcache.ai/cacheblend-engine: "tensormesh-operator/tensormesh-cacheblend"   # <release-ns>/<name>

Everything else (image, model, resources, replicas) stays your normal vLLM config. At admission the webhook injects the plugin init container, hostIPC, the CUSTOM attention backend, and the --kv-transfer-config (CBKVConnector) pointing at the node-local engine; blend tunables (cb.check_layer, cb.recomp_ratio) come from the engine config.

Helm values reference

Value	Type	Default	Description
`cacheBlend.enabled`	bool	`false`	Create a `CacheBlendEngine` CR. Requires `webhook.enabled: true` + cert-manager.
`cacheBlend.name`	string	`""`	Engine CR name. Empty = `<fullname>-cacheblend`.
`cacheBlend.namespace`	string	`""`	Namespace for the engine + its connection ConfigMap. Empty (recommended) = release namespace; opt-in pods bind it cross-namespace.
`cacheBlend.spec.l1.sizeGB`	int	`60`	L1 (host-DRAM) chunk cache size, in GiB.
`cacheBlend.spec.image`	object	`lmcache/vllm-openai`	Blend-engine image (the lmcache server).
`cacheBlend.spec.injection.payloadImage`	object	—	PRIVATE `cacheblend-plugin` image injected as the init container. Must match the engine’s lmcache build.
`cacheBlend.spec.injection.imagePullSecrets`	list	`[]`	Pull secret names appended to opt-in pods for the private payload image. The Secret(s) must already exist in the vLLM pod’s namespace.
`webhook.enabled`	bool	`true`	Enable the CacheBlend mutating webhook. Requires cert-manager.

Verification

Engine reconciled

kubectl -n tensormesh-operator get cacheblendengine tensormesh-cacheblend
kubectl -n tensormesh-operator get configmap tensormesh-cacheblend-connection

The <engine>-connection ConfigMap is the reconcile proof — and the gate the webhook reads. If it is missing, injection silently fail-opens (the pod runs without CacheBlend).

Webhook wired

kubectl get mutatingwebhookconfiguration tensormesh-operator-mutating-webhook \
  -o jsonpath='{.webhooks[0].clientConfig.caBundle}' | head -c 20; echo

A non-empty caBundle means cert-manager issued the serving cert and the webhook can be called.

Injection happened

After creating an opt-in vLLM pod in the workload namespace:

P=$(kubectl -n cacheblend-workload get pod -l lmcache.ai/cacheblend-inject=true -o name | head -1)
kubectl -n cacheblend-workload get $P -o jsonpath='{.metadata.annotations.lmcache\.ai/cacheblend-injected}{"\n"}'  # present = injected
kubectl -n cacheblend-workload get $P -o jsonpath='{.spec.initContainers[*].name}{"\n"}'   # cb-plugin-stage
kubectl -n cacheblend-workload get $P -o jsonpath='{.spec.hostIPC}{"\n"}'                   # true

Non-prefix blend actually engaged

Send two requests that share a block after different-length preambles, then check the engine logs. A shifted retrieve means a chunk was reused at a different position than it was cached — i.e. a non-prefix blend (select the engine by instance to skip any regular LMCacheEngine in the same ns):

kubectl -n tensormesh-operator logs -l app.kubernetes.io/instance=tensormesh-cacheblend --tail=500 \
  | grep -E "\[match_probe\] .* matches=[1-9]|Retrieved pre-computed for [1-9].* shifted=[1-9]"

Common mistakes

Running the opt-in vLLM pod in the operator (release) namespace — the webhook excludes it, so injection never happens
A bare engine ref (tensormesh-cacheblend) instead of <release-ns>/tensormesh-cacheblend — the webhook looks in the pod’s own namespace and finds no engine
Setting a command: on the vLLM container — the webhook skips injection (appended args would never reach vllm serve)
Workload namespace not PSS-privileged — Pod Security rejects the injected hostIPC
The cacheblend-plugin-pull secret missing from the workload namespace — the injected init container ImagePullBackOffs (the secret must live where the pod runs, not in the operator namespace)
Mismatched cacheblend-plugin and vllm-openai images — vLLM crashes at startup with an ImportError; pin a matched pair
webhook.enabled: false or no cert-manager — the engine deploys but nothing injects the plugin

Installation

Feature List

Integrations

Management

Operator UI

Reference

Prerequisites

Install Chart

Opting a vLLM pod in

Helm values reference

Verification

Engine reconciled

Webhook wired

Injection happened

Non-prefix blend actually engaged

Common mistakes

​Prerequisites

​Install Chart

​Opting a vLLM pod in

​Helm values reference

​Verification

​Engine reconciled

​Webhook wired

​Injection happened

​Non-prefix blend actually engaged

​Common mistakes

Prerequisites

Install Chart

Opting a vLLM pod in

Helm values reference

Verification

Engine reconciled

Webhook wired

Injection happened

Non-prefix blend actually engaged

Common mistakes