Modify an Existing vLLM Deployment

Most customers already run vLLM some other way before they install the Tensormesh Platform. This page covers the minimum changes needed to patch an existing vLLM Deployment so it uses the operator-managed LMCache engine in MP mode.

Enable injection in the chart

Injection is configured once, on the operator release — nothing here goes into your vLLM Deployment. Two values matter: the webhook must be on (it is by default), and the engine must carry a payload image if you want payload injection as well as connection wiring.

my-values.yaml

webhook:
  enabled: true                 # default; requires cert-manager

engine:
  enabled: true
  namespace: my-workload        # NOT the release namespace — see the warning below
  spec:
    l1:
      sizeGB: 60
    injection:
      payloadImage:
        repository: lmcache/lmcache-payload
        tag: "v0.5.4"           # must match the engine's lmcache — same wire protocol
        pullPolicy: IfNotPresent

helm upgrade --install tensormesh-operator \
  oci://ghcr.io/tensormesh-production/charts/tensormesh-operator \
  --namespace tensormesh-operator --create-namespace \
  -f my-values.yaml

Webhook injection

Add two lines of metadata on the vLLM pod template:

spec:
  template:
    metadata:
      labels:
        lmcache.ai/lmcache-inject: "true"      
      annotations:
        lmcache.ai/lmcache-engine: "tensormesh-operator-default-engine"  

At admission the webhook adds, to the target container:

Injected	Purpose
`--kv-transfer-config <json>`	the engine’s connection contract, read from `<engine>-connection`
`/dev/shm` sharing (or `hostIPC`)	MP-mode shared memory, mirroring the engine’s `spec.hostIPC`
`PYTHONHASHSEED=0`	deterministic token hashing, so vLLM and the engine derive the same cache keys
`lmcache.ai/lmcache-injected: "true"`	idempotency stamp — a second admission is a no-op

The engine must be in the same namespace as the pod. The injector resolves <engine>-connection in the pod’s own namespace (req.Namespace), so the annotation takes a bare engine name, not <namespace>/<name>. The chart’s own release namespace is excluded from the webhook, so run the vLLM Deployment in a namespace alongside an engine — see Namespace rule.

Payload injection: stop matching lmcache versions by hand

Ordinarily your vLLM image must already carry a compatible external lmcache, matched to the engine’s version. That is the awkward part of running an existing Deployment against a managed engine, and payload injection removes it. When the engine carries an injection.payloadImage (set it in the chart), the webhook injects these as well as the connection wiring:

Injected	Purpose
`lmcache-payload-stage` init container	copies the engine’s `lmcache` into a shared volume
`lmcache-payload` emptyDir + read-only mount	where the staged runtime lands
`PYTHONPATH=/lmcache-payload`	makes vLLM import the staged lmcache ahead of its baked-in one

So an older vLLM image runs the engine’s lmcache without being rebuilt. The payload image is published alongside every engine release and nightly, with matching tags (v0.5.4, latest-nightly, …) — pin it to the engine’s tag, because client and server share a versioned protocol.

Verify the injection

Confirm the pod spec was rewritten:

kubectl get pod <pod> -o jsonpath='{.metadata.annotations}' | grep lmcache
kubectl get pod <pod> -o jsonpath='{.spec.initContainers[*].name}'
kubectl get pod <pod> -o jsonpath='{.spec.containers[0].env[?(@.name=="PYTHONPATH")].value}'

Then prove which lmcache is actually imported at runtime — the staged one, and the baked-in one for contrast:

# staged (what vLLM uses)
kubectl exec <pod> -c vllm -- python3 -c 'import lmcache; print(lmcache.__version__, lmcache.__file__)'
#   0.5.4 /lmcache-payload/lmcache/__init__.py

# baked into the image
kubectl exec <pod> -c vllm -- env -u PYTHONPATH python3 -c 'import lmcache; print(lmcache.__version__, lmcache.__file__)'
#   0.5.2 /opt/venv/lib/python3.12/site-packages/lmcache/__init__.py

Different paths and versions means payload injection is doing its job.

When the webhook skips your pod

Injection fails open: the pod is always admitted, and the reason is stamped on lmcache.ai/lmcache-skip-reason. Read it before assuming the webhook is broken:

kubectl get pod <pod> -o jsonpath='{.metadata.annotations.lmcache\.ai/lmcache-skip-reason}'

Skip reason	Meaning	What to do
`command-override`	the container sets `command:`	move the flags into `args:` so the image’s entrypoint runs — see below
`engine-not-found`	no `<engine>-connection` in the pod’s namespace	fix the annotation, or move the pod to the engine’s namespace
`kv-transfer-config-present`	you already pass `--kv-transfer-config`	intentional; yours is never clobbered
`target-container-not-found`	multi-container pod, wrong target	name it with `lmcache.ai/lmcache-container`
`payload-image-unset`	engine has no `injection.payloadImage`	set it, or supply `lmcache` in your image

OpenShift-specific patch

The webhook injects the shared-memory wiring, but it cannot grant your pod permission to use it. On OpenShift the vLLM pod still needs an SCC that permits what was injected — hostIPC, or the /dev/shm hostPath the webhook uses when the engine runs with hostIPC: false. The chart handles only the engine pod’s ServiceAccount; it does not patch your pre-existing vLLM Deployment. A pod whose SCC forbids the injected wiring is rejected after a successful injection, so the skip-reason annotation will be absent — look at the pod’s events instead. Your vLLM pod therefore needs:

hostIPC: true
a serviceAccountName bound to a suitable SCC

If your vLLM Deployment is in the same namespace as the chart-managed engine, you can usually reuse the chart-created privileged ServiceAccount:

spec:
  template:
    spec:
      hostIPC: true
      serviceAccountName: tensormesh-operator-engine-privileged

If the Deployment is in another namespace, that namespace needs its own ServiceAccount plus SCC binding.

Verify the patched deployment

After patching and rolling out the Deployment:

kubectl rollout status deploy/<your-vllm-deployment> --timeout=15m
kubectl get pod -l app=<your-label> -o wide
kubectl exec -it <vllm-pod> -- \
  python3 -c 'import lmcache.integration.vllm.lmcache_mp_connector as m; print(m.__file__)'

Then send two identical long prompts through the vLLM Service and check the engine logs. Port-forward the Service in one terminal:

kubectl port-forward -n <your-namespace> svc/<your-vllm-service> 8000:8000

In a second terminal, build the request once and send it twice:

PROMPT=$(python3 -c "print('Tell me a long story about a brave knight. ' * 30)")
PAYLOAD=$(jq -n --arg p "$PROMPT" \
  '{model:"Qwen/Qwen3-0.6B", prompt:$p, max_tokens:20, temperature:0}')

# Call 1 — cold, should STORE KV into LMCache
time curl -sS http://localhost:8000/v1/completions \
  -H 'Content-Type: application/json' -d "$PAYLOAD" | jq -r '.choices[0].text'

# Call 2 — identical, should RETRIEVE prefix KV from LMCache
time curl -sS http://localhost:8000/v1/completions \
  -H 'Content-Type: application/json' -d "$PAYLOAD" | jq -r '.choices[0].text'

Then inspect the engine logs:

# select by the engine CR's name — the component label (cache-engine) also
# matches CacheBlend engine pods when those are enabled
kubectl logs -n <your-namespace> \
  -l app.kubernetes.io/instance=tensormesh-operator-default-engine --tail=200 \
  | grep -E "Stored [0-9]+ tokens|Prefetch request completed.*prefix hits"

Healthy behavior is:

call 1 prints a normal completion and the engine logs Stored N tokens
call 2 sends the exact same payload and the engine logs Prefetch request completed ... prefix hits=N
call 2 is usually noticeably faster than call 1 for a long enough prompt

Overview

Installation

Features

Operations

Integrations

Operator UI

Reference

Enable injection in the chart

Webhook injection

Payload injection: stop matching lmcache versions by hand

Verify the injection

When the webhook skips your pod

OpenShift-specific patch

Verify the patched deployment

​Enable injection in the chart

​Webhook injection

​Payload injection: stop matching lmcache versions by hand

​Verify the injection

​When the webhook skips your pod

​OpenShift-specific patch

​Verify the patched deployment

Enable injection in the chart

Webhook injection

Payload injection: stop matching lmcache versions by hand

Verify the injection

When the webhook skips your pod

OpenShift-specific patch

Verify the patched deployment