> ## Documentation Index > Fetch the complete documentation index at: https://docs.tensormesh.ai/llms.txt > Use this file to discover all available pages before exploring further. # CacheBlend > Non-Prefix KV Cache Reuse [CacheBlend](https://arxiv.org/abs/2405.16444) is a technique introduced by the Tensormesh team, enabling reuse of precomputed KV caches beyond prefix boundaries. Rather than requiring an exact prefix match, CacheBlend identifies and reuses any contiguous block of previously computed tokens regardless of position, then selectively recomputes the KV values of a small subset of tokens to partially update each reused cache block. CacheBlend is most useful when: * you serve **RAG** workloads where the same documents are retrieved in varying order across requests * prompt reuse is **non-prefix** — shared blocks sit after a per-request preamble, system prompt, or other documents * prefix caching alone leaves you re-prefilling content you have already seen ## Prerequisites CacheBlend requires the following minimum versions, plus cert-manager: | Component | Minimum version | | --------------------------------------------- | ------------------ | | Tensormesh Operator helm chart | `0.5.1-rc1` | | LMCache Operator (`lmcache/lmcache-operator`) | `v0.5.0` | | LMCache vLLM (`lmcache/vllm-openai`) | `v0.5.0` | | cacheblend-plugin (`private image`) | latest-nightly | | cert-manager | any recent release | ## Configuration CacheBlend needs **two namespaces**: * the **operator** namespace (the Helm release namespace) * a **workload** namespace — PSS-privileged; holds the `CacheBlendEngine`, its connection ConfigMap, the opt-in vLLM pods, and the cacheblend-plugin private-image pull secret Pre-create the workload namespace and the pull secret: ```bash theme={null} # operator namespace kubectl create ns tensormesh-operator # workload namespace: PSS-privileged + private-image pull secret kubectl create ns cacheblend-workload kubectl label ns cacheblend-workload pod-security.kubernetes.io/enforce=privileged kubectl -n cacheblend-workload create secret docker-registry cacheblend-plugin-pull \ --docker-server=docker.io --docker-username= --docker-password= ``` Then install with CacheBlend enabled, pointing the engine at the workload namespace: ```yaml my-values.yaml theme={null} # REQUIRED — the mutating webhook injects the plugin (needs cert-manager). webhook: enabled: true cacheBlend: enabled: true name: tensormesh-cacheblend # MUST differ from the release namespace and be PSS-privileged. namespace: cacheblend-workload spec: l1: sizeGB: 60 image: repository: lmcache/vllm-openai tag: # same lmcache build as the plugin below pullPolicy: IfNotPresent injection: payloadImage: repository: tensormesh/cacheblend-plugin # PRIVATE tag: pullPolicy: IfNotPresent imagePullSecrets: - name: cacheblend-plugin-pull # must exist in the workload ns ``` ```bash theme={null} helm upgrade --install tensormesh-operator \ oci://ghcr.io/tensormesh-production/charts/tensormesh-operator \ -n tensormesh-operator --create-namespace \ -f my-values.yaml --wait ``` ## Opting a vLLM pod in CacheBlend attaches to vLLM pods **you deploy** — the operator does not create them. In the **workload namespace**, add two things to your vLLM pod template, and launch vLLM **args-only** ( `command:` override makes the webhook skip injection): ```yaml theme={null} metadata: labels: lmcache.ai/cacheblend-inject: "true" # opt in annotations: lmcache.ai/cacheblend-engine: "tensormesh-cacheblend" # engine in this namespace ``` Everything else (image, model, resources, replicas) stays your normal vLLM config. At admission the webhook injects the plugin init container, `hostIPC`, the `CUSTOM` attention backend, and the `--kv-transfer-config` (`CBKVConnector`) pointing at the node-local engine; blend tunables (`cb.check_layer`, `cb.recomp_ratio`) come from the engine config. ## Helm values reference | Value | Type | Default | Description | | -------------------------------------------- | ------ | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | | `cacheBlend.enabled` | bool | `false` | Create a `CacheBlendEngine` CR. Requires `webhook.enabled: true` + cert-manager. | | `cacheBlend.name` | string | `""` | Engine CR name. Empty = `-cacheblend`. | | `cacheBlend.namespace` | string | `""` | Namespace for the engine + its connection ConfigMap. Use a non-release, PSS-privileged workload namespace. Empty = release namespace. | | `cacheBlend.spec.l1.sizeGB` | int | `60` | L1 (host-DRAM) chunk cache size, in GiB. | | `cacheBlend.spec.image` | object | `lmcache/vllm-openai` | Blend-engine image (the lmcache server). | | `cacheBlend.spec.injection.payloadImage` | object | — | PRIVATE `cacheblend-plugin` image injected as the init container. Must match the engine's lmcache build. | | `cacheBlend.spec.injection.imagePullSecrets` | list | `[]` | Pull secret names appended to opt-in pods for the private payload image. The Secret(s) must already exist in the vLLM pod's namespace. | | `webhook.enabled` | bool | `true` | Enable the CacheBlend mutating webhook. Requires cert-manager. | ## Verification ### Engine reconciled ```bash theme={null} kubectl -n cacheblend-workload get cacheblendengine tensormesh-cacheblend kubectl -n cacheblend-workload get configmap tensormesh-cacheblend-connection ``` The `-connection` ConfigMap is the reconcile proof — and the gate the webhook reads. If it is missing, injection silently fail-opens (the pod runs without CacheBlend). ### Webhook wired ```bash theme={null} kubectl get mutatingwebhookconfiguration tensormesh-operator-mutating-webhook \ -o jsonpath='{.webhooks[0].clientConfig.caBundle}' | head -c 20; echo ``` A non-empty `caBundle` means cert-manager issued the serving cert and the webhook can be called. ### Injection happened After creating an opt-in vLLM pod in the workload namespace: ```bash theme={null} P=$(kubectl -n cacheblend-workload get pod -l lmcache.ai/cacheblend-inject=true -o name | head -1) kubectl -n cacheblend-workload get $P -o jsonpath='{.metadata.annotations.lmcache\.ai/cacheblend-injected}{"\n"}' # present = injected kubectl -n cacheblend-workload get $P -o jsonpath='{.spec.initContainers[*].name}{"\n"}' # cb-plugin-stage kubectl -n cacheblend-workload get $P -o jsonpath='{.spec.hostIPC}{"\n"}' # true ``` ### Non-prefix blend actually engaged (optional) Send two requests that share a block **after different-length preambles**, then check the engine logs. A `shifted` retrieve means a chunk was reused at a different position than it was cached — i.e. a non-prefix blend: ```bash theme={null} kubectl -n cacheblend-workload logs -l app.kubernetes.io/component=cache-engine --tail=500 \ | grep -E "\[match_probe\] .* matches=[1-9]|Retrieved pre-computed for [1-9].* shifted=[1-9]" ``` ## Common mistakes * Running the opt-in vLLM pod in the **operator (release) namespace** — the webhook excludes it, so injection never happens * Setting a `command:` on the vLLM container — the webhook skips injection (appended args would never reach `vllm serve`) * Workload namespace **not PSS-privileged** — Pod Security rejects the injected `hostIPC` * The `cacheblend-plugin-pull` secret missing from the **workload** namespace — the injected init container `ImagePullBackOff`s (the secret must live where the pod runs, not in the operator namespace) * **Mismatched** `cacheblend-plugin` and `vllm-openai` images — vLLM crashes at startup with an `ImportError`; pin a matched pair * `webhook.enabled: false` or no cert-manager — the engine deploys but nothing injects the plugin