> ## Documentation Index > Fetch the complete documentation index at: https://docs.tensormesh.ai/llms.txt > Use this file to discover all available pages before exploring further. # Modify an Existing Deployment > Patch an existing vLLM deployment to consume the LMCache engine created by the Tensormesh Operator. Most customers already run vLLM some other way before they install the Tensormesh Operator. This page covers the **minimum changes** needed to patch an existing vLLM Deployment so it uses the operator-managed LMCache engine in MP mode. The vLLM pod and the LMCache engine must run a **compatible LMCache connector/runtime pair**. Do not rely on `latest`. Pin explicit image tags, or build your own vLLM image with the external `lmcache` package installed and tested. Current released operator builds may **not** include `kv_connector_module_path=lmcache.integration.vllm.lmcache_mp_connector` in the generated connection `ConfigMap` yet. That change is in flight. Until your installed operator release explicitly includes it, add the field yourself in your vLLM `--kv-transfer-config` JSON or contact the Tensormesh team for the supported patch path. Your vLLM build must also be new enough to honor that field: use a build that includes the vLLM connector-loading fix from **April 7, 2026** or newer, and prefer builds from **May 15, 2026** or newer. ## What has to change Your existing vLLM Deployment must do all of the following: 1. Mount the engine connection `ConfigMap` and pass it to `--kv-transfer-config`. 2. Run with `hostIPC: true` so MP-mode shared memory works. 3. Schedule onto a node that also runs an LMCache engine pod. 4. Use a vLLM image that can import the **external** `lmcache` package successfully. 5. On OpenShift, run under a ServiceAccount bound to an SCC that allows `hostIPC`. If any of those is missing, the deployment may start but LMCache integration will either fail outright or silently fall back to the wrong connector path. ## The minimum patch Assume the chart created an engine named `tensormesh-operator-default-engine`. That engine creates a same-namespace `ConfigMap` named: ```text theme={null} tensormesh-operator-default-engine-connection ``` The vLLM pod must mount that `ConfigMap` and pass its JSON contents to `--kv-transfer-config`. The desired steady-state contract is: ```json theme={null} { "kv_connector": "LMCacheMPConnector", "kv_connector_module_path": "lmcache.integration.vllm.lmcache_mp_connector", "kv_role": "kv_both", "kv_connector_extra_config": { "lmcache.mp.host": "tcp://tensormesh-operator-default-engine..svc.cluster.local", "lmcache.mp.port": "5555" } } ``` If your installed operator release already generates that full JSON, you do **not** need to hand-write `kv_connector_module_path` yourself. If it does **not**, add that field explicitly when you construct `--kv-transfer-config` for the vLLM pod. Example Deployment fragment: ```yaml theme={null} spec: template: spec: hostIPC: true nodeSelector: nvidia.com/gpu.present: "true" containers: - name: vllm image: lmcache/vllm-openai:v0.4.5 args: - | exec python3 -m vllm.entrypoints.openai.api_server \ --model Qwen/Qwen3-0.6B \ --port 8000 \ --no-enable-prefix-caching \ --kv-transfer-config "$(cat /etc/lmcache/kv-transfer-config.json)" volumeMounts: - name: kv-transfer-config mountPath: /etc/lmcache readOnly: true volumes: - name: kv-transfer-config configMap: name: tensormesh-operator-default-engine-connection ``` ### Why each change exists * `hostIPC: true` * required for LMCache MP mode * `--kv-transfer-config ...` * tells vLLM how to reach the engine service and which connector settings to use * `--no-enable-prefix-caching` * disables vLLM's own local prefix cache so LMCache is the cache path being exercised * `nodeSelector` * ensures the pod lands on a GPU node where an engine pod also exists ## Image requirement: external `lmcache` must be installed This is the easy point to miss. For the MP connector path, the safe target state is to point vLLM at the **external** `lmcache` connector module explicitly. That avoids silently using an older vendored builtin path, but it also means your vLLM build must be new enough to load that module correctly. Minimum support expectation: * supported: vLLM builds that include the explicit external-module selection fix merged on **April 7, 2026** * preferred: vLLM builds that include the LMCache MP external-by-default/fallback behavior merged on **May 15, 2026** * if your existing build predates those changes, or you are unsure, contact the Tensormesh team before patching the deployment Safe options: * use a pinned Tensormesh/LMCache image that already bundles the intended `lmcache` version * or build your own vLLM image with `lmcache` installed explicitly Unsafe options: * `vllm/vllm-openai:latest` with no LMCache install * mutable tags where the real digest is unknown * assuming matching YAML tags are enough without checking the running pod image If you build your own image, verify the import inside the running pod: ```bash theme={null} kubectl exec -it -- \ python3 -c 'import lmcache.integration.vllm.lmcache_mp_connector as m; print(m.__file__)' ``` That command should resolve to the external `lmcache` package, not to a vendored fallback inside vLLM. If the import works but the pod still fails during LMCache registration, check the vLLM build date or commit next. A working `lmcache` import is necessary, but for MP mode it is not sufficient if the vLLM connector-loading behavior is too old. ## Namespace rule The connection `ConfigMap` is namespaced. A pod can only mount a `ConfigMap` from its own namespace. That means: * easiest path: run the vLLM Deployment in the **same namespace** as the engine * if you must run vLLM elsewhere, you need to copy/replicate the connection `ConfigMap` into that namespace and keep it in sync For most installs, same namespace is the correct default. ## OpenShift-specific patch If your existing Deployment runs on OpenShift, `hostIPC: true` means the vLLM pod also needs an SCC that permits it. The chart only handles the engine pod's ServiceAccount automatically; it does not magically patch your pre-existing vLLM Deployment. Your vLLM pod therefore needs: * `hostIPC: true` * a `serviceAccountName` bound to a suitable SCC If your vLLM Deployment is in the same namespace as the chart-managed engine, you can usually reuse the chart-created privileged ServiceAccount: ```yaml theme={null} spec: template: spec: hostIPC: true serviceAccountName: tensormesh-operator-engine-privileged ``` If the Deployment is in another namespace, that namespace needs its own ServiceAccount plus SCC binding. ## Other settings that are optional These are often useful, but they are not the core LMCache wiring: * `HF_HOME=/tmp/hf` * useful on OpenShift because random UIDs need a writable cache path * `HF_HUB_DISABLE_TELEMETRY=1` * hygiene only * readiness/liveness tuning * depends on model size and boot time ## What usually breaks The common failure modes are: * mounting the ConfigMap but forgetting `hostIPC: true` * pinning the engine image but leaving the vLLM pod on `latest` * updating the Deployment spec but still having an old ReplicaSet pod alive * running vLLM in a different namespace and trying to mount a cross-namespace ConfigMap * using OpenShift without a ServiceAccount/SCC that permits `hostIPC` * assuming the vLLM image has external `lmcache` installed when it does not * using a vLLM build too old to honor `kv_connector_module_path` correctly ## Verify the patched deployment After patching and rolling out the Deployment: ```bash theme={null} kubectl rollout status deploy/ --timeout=15m kubectl get pod -l app= -o wide kubectl exec -it -- \ python3 -c 'import lmcache.integration.vllm.lmcache_mp_connector as m; print(m.__file__)' ``` Then send two identical long prompts through the vLLM Service and check the engine logs. Port-forward the Service in one terminal: ```bash theme={null} kubectl port-forward -n svc/ 8000:8000 ``` In a second terminal, build the request once and send it twice: ```bash theme={null} PROMPT=$(python3 -c "print('Tell me a long story about a brave knight. ' * 30)") PAYLOAD=$(jq -n --arg p "$PROMPT" \ '{model:"Qwen/Qwen3-0.6B", prompt:$p, max_tokens:20, temperature:0}') # Call 1 — cold, should STORE KV into LMCache time curl -sS http://localhost:8000/v1/completions \ -H 'Content-Type: application/json' -d "$PAYLOAD" | jq -r '.choices[0].text' # Call 2 — identical, should RETRIEVE prefix KV from LMCache time curl -sS http://localhost:8000/v1/completions \ -H 'Content-Type: application/json' -d "$PAYLOAD" | jq -r '.choices[0].text' ``` Then inspect the engine logs: ```bash theme={null} kubectl logs -n -l app.kubernetes.io/component=cache-engine --tail=200 \ | grep -E "Stored [0-9]+ tokens|Prefetch request completed.*prefix hits" ``` Healthy behavior is: * call 1 prints a normal completion and the engine logs `Stored N tokens` * call 2 sends the exact same payload and the engine logs `Prefetch request completed ... prefix hits=N` * call 2 is usually noticeably faster than call 1 for a long enough prompt Keep `PROMPT` and `PAYLOAD` in the same shell as the `curl` commands. If `$PAYLOAD` is empty, the request carries no prompt and you will see no LMCache store/prefetch markers. ## Next steps OpenShift-specific SCC and ServiceAccount requirements. Full install plus demo deployment, smoke test, and benchmark.