> ## Documentation Index > Fetch the complete documentation index at: https://docs.tensormesh.ai/llms.txt > Use this file to discover all available pages before exploring further. # Modify an Existing vLLM Deployment > Patch an existing vLLM deployment to consume the LMCache engine created by the Tensormesh Operator. Most customers already run vLLM some other way before they install the Tensormesh Operator. This page covers the **minimum changes** needed to patch an existing vLLM Deployment so it uses the operator-managed LMCache engine in MP mode. The vLLM pod and the LMCache engine must run a **compatible LMCache connector/runtime pair**. Do not rely on `latest`. Pin explicit image tags, or build your own vLLM image with the external `lmcache` package installed and tested. ## What the Deployment must do Your existing vLLM Deployment must do all of the following: 1. Mount the engine connection `ConfigMap` and pass it to `--kv-transfer-config`. 2. Run with `hostIPC: true` so MP-mode shared memory works. 3. Set `PYTHONHASHSEED=0` on the vLLM container — LMCache requires deterministic token hashing across processes for consistent cache keys. 4. Schedule onto a node that also runs an LMCache engine pod. 5. Use a vLLM image that can import the **external** `lmcache` package successfully. 6. On OpenShift, run under a ServiceAccount bound to an SCC that allows `hostIPC`. If any of those is missing, the deployment may start but LMCache integration will either fail outright or silently fall back to the wrong connector path. ## The minimum patch Assume the chart created an engine named `tensormesh-operator-default-engine`. That engine creates a same-namespace `ConfigMap` named: ```text theme={null} tensormesh-operator-default-engine-connection ``` The vLLM pod must mount that `ConfigMap` and pass its JSON contents to `--kv-transfer-config`. The desired steady-state contract is: ```json theme={null} { "kv_connector": "LMCacheMPConnector", "kv_connector_module_path": "lmcache.integration.vllm.lmcache_mp_connector", "kv_role": "kv_both", "kv_connector_extra_config": { "lmcache.mp.host": "tcp://tensormesh-operator-default-engine..svc.cluster.local", "lmcache.mp.port": "5555" } } ``` Every released operator (`v0.5.0` and later) generates this full JSON — including `kv_connector_module_path` — so you never hand-write it; the pod just passes the ConfigMap's contents through. Example Deployment fragment: ```yaml theme={null} spec: template: spec: hostIPC: true nodeSelector: nvidia.com/gpu.present: "true" containers: - name: vllm image: lmcache/vllm-openai:v0.5.2 # pin the tag the Compatibility Matrix pairs with your chart env: - name: PYTHONHASHSEED value: "0" args: - | exec python3 -m vllm.entrypoints.openai.api_server \ --model Qwen/Qwen3-0.6B \ --port 8000 \ --no-enable-prefix-caching \ --kv-transfer-config "$(cat /etc/lmcache/kv-transfer-config.json)" volumeMounts: - name: kv-transfer-config mountPath: /etc/lmcache readOnly: true volumes: - name: kv-transfer-config configMap: name: tensormesh-operator-default-engine-connection ``` ### Why each change exists * `hostIPC: true` * required for LMCache MP mode * `PYTHONHASHSEED=0` * deterministic token hashing so vLLM and the engine derive the same cache keys * `--kv-transfer-config ...` * tells vLLM how to reach the engine service and which connector settings to use * `--no-enable-prefix-caching` * disables vLLM's own local prefix cache so LMCache is the cache path being exercised * `nodeSelector` * ensures the pod lands on a GPU node where an engine pod also exists ## Image requirement: external `lmcache` must be installed This is the easy point to miss. For the MP connector path, the safe target state is to point vLLM at the **external** `lmcache` connector module explicitly. That avoids silently using an older vendored builtin path, but it also means your vLLM build must be new enough to load that module correctly. The safe default is the serving image the [Compatibility Matrix](/installation/compatibility) pairs with your chart version — it bundles a matched vLLM + `lmcache`. If you must keep your own image, it needs a vLLM recent enough to honor `kv_connector_module_path`; when unsure, contact the Tensormesh team before patching the deployment. Safe options: * use a pinned Tensormesh/LMCache image that already bundles the intended `lmcache` version * or build your own vLLM image with `lmcache` installed explicitly Unsafe options: * `vllm/vllm-openai:latest` with no LMCache install * mutable tags where the real digest is unknown * assuming matching YAML tags are enough without checking the running pod image If you build your own image, verify the import inside the running pod: ```bash theme={null} kubectl exec -it -- \ python3 -c 'import lmcache.integration.vllm.lmcache_mp_connector as m; print(m.__file__)' ``` That command should resolve to the external `lmcache` package, not to a vendored fallback inside vLLM. If the import works but the pod still fails during LMCache registration, check the vLLM build date or commit next. A working `lmcache` import is necessary, but for MP mode it is not sufficient if the vLLM connector-loading behavior is too old. ## Namespace rule The connection `ConfigMap` is namespaced. A pod can only mount a `ConfigMap` from its own namespace. That means: * easiest path: run the vLLM Deployment in the **same namespace** as the engine * if you must run vLLM elsewhere, you need to copy/replicate the connection `ConfigMap` into that namespace and keep it in sync For most installs, same namespace is the correct default. ## OpenShift-specific patch If your existing Deployment runs on OpenShift, `hostIPC: true` means the vLLM pod also needs an SCC that permits it. The chart only handles the engine pod's ServiceAccount automatically; it does not magically patch your pre-existing vLLM Deployment. Your vLLM pod therefore needs: * `hostIPC: true` * a `serviceAccountName` bound to a suitable SCC If your vLLM Deployment is in the same namespace as the chart-managed engine, you can usually reuse the chart-created privileged ServiceAccount: ```yaml theme={null} spec: template: spec: hostIPC: true serviceAccountName: tensormesh-operator-engine-privileged ``` If the Deployment is in another namespace, that namespace needs its own ServiceAccount plus SCC binding. ## Other settings that are optional These are often useful, but they are not the core LMCache wiring: * `HF_HOME=/tmp/hf` * useful on OpenShift because random UIDs need a writable cache path * `HF_HUB_DISABLE_TELEMETRY=1` * hygiene only * readiness/liveness tuning * depends on model size and boot time ## What usually breaks The common failure modes are: * mounting the ConfigMap but forgetting `hostIPC: true` (or `PYTHONHASHSEED=0`) * pinning the engine image but leaving the vLLM pod on `latest` * updating the Deployment spec but still having an old ReplicaSet pod alive * running vLLM in a different namespace and trying to mount a cross-namespace ConfigMap * using OpenShift without a ServiceAccount/SCC that permits `hostIPC` * assuming the vLLM image has external `lmcache` installed when it does not * using a vLLM build too old to honor `kv_connector_module_path` correctly ## Verify the patched deployment After patching and rolling out the Deployment: ```bash theme={null} kubectl rollout status deploy/ --timeout=15m kubectl get pod -l app= -o wide kubectl exec -it -- \ python3 -c 'import lmcache.integration.vllm.lmcache_mp_connector as m; print(m.__file__)' ``` Then send two identical long prompts through the vLLM Service and check the engine logs. Port-forward the Service in one terminal: ```bash theme={null} kubectl port-forward -n svc/ 8000:8000 ``` In a second terminal, build the request once and send it twice: ```bash theme={null} PROMPT=$(python3 -c "print('Tell me a long story about a brave knight. ' * 30)") PAYLOAD=$(jq -n --arg p "$PROMPT" \ '{model:"Qwen/Qwen3-0.6B", prompt:$p, max_tokens:20, temperature:0}') # Call 1 — cold, should STORE KV into LMCache time curl -sS http://localhost:8000/v1/completions \ -H 'Content-Type: application/json' -d "$PAYLOAD" | jq -r '.choices[0].text' # Call 2 — identical, should RETRIEVE prefix KV from LMCache time curl -sS http://localhost:8000/v1/completions \ -H 'Content-Type: application/json' -d "$PAYLOAD" | jq -r '.choices[0].text' ``` Then inspect the engine logs: ```bash theme={null} # select by the engine CR's name — the component label (cache-engine) also # matches CacheBlend engine pods when those are enabled kubectl logs -n \ -l app.kubernetes.io/instance=tensormesh-operator-default-engine --tail=200 \ | grep -E "Stored [0-9]+ tokens|Prefetch request completed.*prefix hits" ``` Healthy behavior is: * call 1 prints a normal completion and the engine logs `Stored N tokens` * call 2 sends the exact same payload and the engine logs `Prefetch request completed ... prefix hits=N` * call 2 is usually noticeably faster than call 1 for a long enough prompt Keep `PROMPT` and `PAYLOAD` in the same shell as the `curl` commands. If `$PAYLOAD` is empty, the request carries no prompt and you will see no LMCache store/prefetch markers. ## Next steps Full install plus demo deployment, smoke test, and benchmark.