Modify an Existing vLLM Deployment

Most customers already run vLLM some other way before they install the Tensormesh Operator. This page covers the minimum changes needed to patch an existing vLLM Deployment so it uses the operator-managed LMCache engine in MP mode.

The vLLM pod and the LMCache engine must run a compatible LMCache connector/runtime pair. Do not rely on latest. Pin explicit image tags, or build your own vLLM image with the external lmcache package installed and tested.

What the Deployment must do

Your existing vLLM Deployment must do all of the following:

Mount the engine connection ConfigMap and pass it to --kv-transfer-config.
Run with hostIPC: true so MP-mode shared memory works.
Set PYTHONHASHSEED=0 on the vLLM container — LMCache requires deterministic token hashing across processes for consistent cache keys.
Schedule onto a node that also runs an LMCache engine pod.
Use a vLLM image that can import the external lmcache package successfully.
On OpenShift, run under a ServiceAccount bound to an SCC that allows hostIPC.

If any of those is missing, the deployment may start but LMCache integration will either fail outright or silently fall back to the wrong connector path.

The minimum patch

Assume the chart created an engine named tensormesh-operator-default-engine. That engine creates a same-namespace ConfigMap named:

tensormesh-operator-default-engine-connection

The vLLM pod must mount that ConfigMap and pass its JSON contents to --kv-transfer-config. The desired steady-state contract is:

{
  "kv_connector": "LMCacheMPConnector",
  "kv_connector_module_path": "lmcache.integration.vllm.lmcache_mp_connector",
  "kv_role": "kv_both",
  "kv_connector_extra_config": {
    "lmcache.mp.host": "tcp://tensormesh-operator-default-engine.<namespace>.svc.cluster.local",
    "lmcache.mp.port": "5555"
  }
}

Every released operator (v0.5.0 and later) generates this full JSON — including kv_connector_module_path — so you never hand-write it; the pod just passes the ConfigMap’s contents through. Example Deployment fragment:

spec:
  template:
    spec:
      hostIPC: true
      nodeSelector:
        nvidia.com/gpu.present: "true"
      containers:
        - name: vllm
          image: lmcache/vllm-openai:v0.5.2   # pin the tag the Compatibility Matrix pairs with your chart
          env:
            - name: PYTHONHASHSEED
              value: "0"
          args:
            - |
              exec python3 -m vllm.entrypoints.openai.api_server \
                --model Qwen/Qwen3-0.6B \
                --port 8000 \
                --no-enable-prefix-caching \
                --kv-transfer-config "$(cat /etc/lmcache/kv-transfer-config.json)"
          volumeMounts:
            - name: kv-transfer-config
              mountPath: /etc/lmcache
              readOnly: true
      volumes:
        - name: kv-transfer-config
          configMap:
            name: tensormesh-operator-default-engine-connection

Why each change exists

hostIPC: true
- required for LMCache MP mode
PYTHONHASHSEED=0
- deterministic token hashing so vLLM and the engine derive the same cache keys
--kv-transfer-config ...
- tells vLLM how to reach the engine service and which connector settings to use
--no-enable-prefix-caching
- disables vLLM’s own local prefix cache so LMCache is the cache path being exercised
nodeSelector
- ensures the pod lands on a GPU node where an engine pod also exists

Image requirement: external `lmcache` must be installed

This is the easy point to miss. For the MP connector path, the safe target state is to point vLLM at the external lmcache connector module explicitly. That avoids silently using an older vendored builtin path, but it also means your vLLM build must be new enough to load that module correctly. The safe default is the serving image the Compatibility Matrix pairs with your chart version — it bundles a matched vLLM + lmcache. If you must keep your own image, it needs a vLLM recent enough to honor kv_connector_module_path; when unsure, contact the Tensormesh team before patching the deployment. Safe options:

use a pinned Tensormesh/LMCache image that already bundles the intended lmcache version
or build your own vLLM image with lmcache installed explicitly

Unsafe options:

vllm/vllm-openai:latest with no LMCache install
mutable tags where the real digest is unknown
assuming matching YAML tags are enough without checking the running pod image

If you build your own image, verify the import inside the running pod:

kubectl exec -it <vllm-pod> -- \
  python3 -c 'import lmcache.integration.vllm.lmcache_mp_connector as m; print(m.__file__)'

That command should resolve to the external lmcache package, not to a vendored fallback inside vLLM. If the import works but the pod still fails during LMCache registration, check the vLLM build date or commit next. A working lmcache import is necessary, but for MP mode it is not sufficient if the vLLM connector-loading behavior is too old.

Namespace rule

The connection ConfigMap is namespaced. A pod can only mount a ConfigMap from its own namespace. That means:

easiest path: run the vLLM Deployment in the same namespace as the engine
if you must run vLLM elsewhere, you need to copy/replicate the connection ConfigMap into that namespace and keep it in sync

For most installs, same namespace is the correct default.

OpenShift-specific patch

If your existing Deployment runs on OpenShift, hostIPC: true means the vLLM pod also needs an SCC that permits it. The chart only handles the engine pod’s ServiceAccount automatically; it does not magically patch your pre-existing vLLM Deployment. Your vLLM pod therefore needs:

hostIPC: true
a serviceAccountName bound to a suitable SCC

If your vLLM Deployment is in the same namespace as the chart-managed engine, you can usually reuse the chart-created privileged ServiceAccount:

spec:
  template:
    spec:
      hostIPC: true
      serviceAccountName: tensormesh-operator-engine-privileged

If the Deployment is in another namespace, that namespace needs its own ServiceAccount plus SCC binding.

Other settings that are optional

These are often useful, but they are not the core LMCache wiring:

HF_HOME=/tmp/hf
- useful on OpenShift because random UIDs need a writable cache path
HF_HUB_DISABLE_TELEMETRY=1
- hygiene only
readiness/liveness tuning
- depends on model size and boot time

What usually breaks

The common failure modes are:

mounting the ConfigMap but forgetting hostIPC: true (or PYTHONHASHSEED=0)
pinning the engine image but leaving the vLLM pod on latest
updating the Deployment spec but still having an old ReplicaSet pod alive
running vLLM in a different namespace and trying to mount a cross-namespace ConfigMap
using OpenShift without a ServiceAccount/SCC that permits hostIPC
assuming the vLLM image has external lmcache installed when it does not
using a vLLM build too old to honor kv_connector_module_path correctly

Verify the patched deployment

After patching and rolling out the Deployment:

kubectl rollout status deploy/<your-vllm-deployment> --timeout=15m
kubectl get pod -l app=<your-label> -o wide
kubectl exec -it <vllm-pod> -- \
  python3 -c 'import lmcache.integration.vllm.lmcache_mp_connector as m; print(m.__file__)'

Then send two identical long prompts through the vLLM Service and check the engine logs. Port-forward the Service in one terminal:

kubectl port-forward -n <your-namespace> svc/<your-vllm-service> 8000:8000

In a second terminal, build the request once and send it twice:

PROMPT=$(python3 -c "print('Tell me a long story about a brave knight. ' * 30)")
PAYLOAD=$(jq -n --arg p "$PROMPT" \
  '{model:"Qwen/Qwen3-0.6B", prompt:$p, max_tokens:20, temperature:0}')

# Call 1 — cold, should STORE KV into LMCache
time curl -sS http://localhost:8000/v1/completions \
  -H 'Content-Type: application/json' -d "$PAYLOAD" | jq -r '.choices[0].text'

# Call 2 — identical, should RETRIEVE prefix KV from LMCache
time curl -sS http://localhost:8000/v1/completions \
  -H 'Content-Type: application/json' -d "$PAYLOAD" | jq -r '.choices[0].text'

Then inspect the engine logs:

# select by the engine CR's name — the component label (cache-engine) also
# matches CacheBlend engine pods when those are enabled
kubectl logs -n <your-namespace> \
  -l app.kubernetes.io/instance=tensormesh-operator-default-engine --tail=200 \
  | grep -E "Stored [0-9]+ tokens|Prefetch request completed.*prefix hits"

Healthy behavior is:

call 1 prints a normal completion and the engine logs Stored N tokens
call 2 sends the exact same payload and the engine logs Prefetch request completed ... prefix hits=N
call 2 is usually noticeably faster than call 1 for a long enough prompt

Keep PROMPT and PAYLOAD in the same shell as the curl commands. If $PAYLOAD is empty, the request carries no prompt and you will see no LMCache store/prefetch markers.

Installation

Feature List

Integrations

Management

Operator UI

Reference

What the Deployment must do

The minimum patch

Why each change exists

Image requirement: external `lmcache` must be installed

Namespace rule

OpenShift-specific patch

Other settings that are optional

What usually breaks

Verify the patched deployment

Next steps

Install on OpenShift

End-to-end example

​What the Deployment must do

​The minimum patch

​Why each change exists

​Image requirement: external lmcache must be installed

​Namespace rule

​OpenShift-specific patch

​Other settings that are optional

​What usually breaks

​Verify the patched deployment

​Next steps

Install on OpenShift

End-to-end example

What the Deployment must do

The minimum patch

Why each change exists

Image requirement: external `lmcache` must be installed

Namespace rule

OpenShift-specific patch

Other settings that are optional

What usually breaks

Verify the patched deployment

Next steps