Skip to main content
Most customers already run vLLM some other way before they install the Tensormesh Operator. This page covers the minimum changes needed to patch an existing vLLM Deployment so it uses the operator-managed LMCache engine in MP mode.
The vLLM pod and the LMCache engine must run a compatible LMCache connector/runtime pair. Do not rely on latest. Pin explicit image tags, or build your own vLLM image with the external lmcache package installed and tested.
Current released operator builds may not include kv_connector_module_path=lmcache.integration.vllm.lmcache_mp_connector in the generated connection ConfigMap yet. That change is in flight. Until your installed operator release explicitly includes it, add the field yourself in your vLLM --kv-transfer-config JSON or contact the Tensormesh team for the supported patch path. Your vLLM build must also be new enough to honor that field: use a build that includes the vLLM connector-loading fix from April 7, 2026 or newer, and prefer builds from May 15, 2026 or newer.

​
What has to change

Your existing vLLM Deployment must do all of the following:
  1. Mount the engine connection ConfigMap and pass it to --kv-transfer-config.
  2. Run with hostIPC: true so MP-mode shared memory works.
  3. Schedule onto a node that also runs an LMCache engine pod.
  4. Use a vLLM image that can import the external lmcache package successfully.
  5. On OpenShift, run under a ServiceAccount bound to an SCC that allows hostIPC.
If any of those is missing, the deployment may start but LMCache integration will either fail outright or silently fall back to the wrong connector path.

​
The minimum patch

Assume the chart created an engine named tensormesh-operator-default-engine. That engine creates a same-namespace ConfigMap named: 
tensormesh-operator-default-engine-connection
The vLLM pod must mount that ConfigMap and pass its JSON contents to --kv-transfer-config. The desired steady-state contract is: 
{
  "kv_connector": "LMCacheMPConnector",
  "kv_connector_module_path": "lmcache.integration.vllm.lmcache_mp_connector",
  "kv_role": "kv_both",
  "kv_connector_extra_config": {
    "lmcache.mp.host": "tcp://tensormesh-operator-default-engine.<namespace>.svc.cluster.local",
    "lmcache.mp.port": "5555"
  }
}
If your installed operator release already generates that full JSON, you do not need to hand-write kv_connector_module_path yourself. If it does not, add that field explicitly when you construct --kv-transfer-config for the vLLM pod. Example Deployment fragment: 
spec:
  template:
    spec:
      hostIPC: true
      nodeSelector:
        nvidia.com/gpu.present: "true"
      containers:
        - name: vllm
          image: lmcache/vllm-openai:v0.4.5
          args:
            - |
              exec python3 -m vllm.entrypoints.openai.api_server \
                --model Qwen/Qwen3-0.6B \
                --port 8000 \
                --no-enable-prefix-caching \
                --kv-transfer-config "$(cat /etc/lmcache/kv-transfer-config.json)"
          volumeMounts:
            - name: kv-transfer-config
              mountPath: /etc/lmcache
              readOnly: true
      volumes:
        - name: kv-transfer-config
          configMap:
            name: tensormesh-operator-default-engine-connection

​
Why each change exists

  • hostIPC: true
    • required for LMCache MP mode
  • --kv-transfer-config ...
    • tells vLLM how to reach the engine service and which connector settings to use
  • --no-enable-prefix-caching
    • disables vLLM’s own local prefix cache so LMCache is the cache path being exercised
  • nodeSelector
    • ensures the pod lands on a GPU node where an engine pod also exists

​
Image requirement: external lmcache must be installed

This is the easy point to miss. For the MP connector path, the safe target state is to point vLLM at the external lmcache connector module explicitly. That avoids silently using an older vendored builtin path, but it also means your vLLM build must be new enough to load that module correctly. Minimum support expectation:
  • supported: vLLM builds that include the explicit external-module selection fix merged on April 7, 2026
  • preferred: vLLM builds that include the LMCache MP external-by-default/fallback behavior merged on May 15, 2026
  • if your existing build predates those changes, or you are unsure, contact the Tensormesh team before patching the deployment
Safe options:
  • use a pinned Tensormesh/LMCache image that already bundles the intended lmcache version
  • or build your own vLLM image with lmcache installed explicitly
Unsafe options:
  • vllm/vllm-openai:latest with no LMCache install
  • mutable tags where the real digest is unknown
  • assuming matching YAML tags are enough without checking the running pod image
If you build your own image, verify the import inside the running pod: 
kubectl exec -it <vllm-pod> -- \
  python3 -c 'import lmcache.integration.vllm.lmcache_mp_connector as m; print(m.__file__)'
That command should resolve to the external lmcache package, not to a vendored fallback inside vLLM. If the import works but the pod still fails during LMCache registration, check the vLLM build date or commit next. A working lmcache import is necessary, but for MP mode it is not sufficient if the vLLM connector-loading behavior is too old.

​
Namespace rule

The connection ConfigMap is namespaced. A pod can only mount a ConfigMap from its own namespace. That means:
  • easiest path: run the vLLM Deployment in the same namespace as the engine
  • if you must run vLLM elsewhere, you need to copy/replicate the connection ConfigMap into that namespace and keep it in sync
For most installs, same namespace is the correct default.

​
OpenShift-specific patch

If your existing Deployment runs on OpenShift, hostIPC: true means the vLLM pod also needs an SCC that permits it. The chart only handles the engine pod’s ServiceAccount automatically; it does not magically patch your pre-existing vLLM Deployment. Your vLLM pod therefore needs:
  • hostIPC: true
  • a serviceAccountName bound to a suitable SCC
If your vLLM Deployment is in the same namespace as the chart-managed engine, you can usually reuse the chart-created privileged ServiceAccount: 
spec:
  template:
    spec:
      hostIPC: true
      serviceAccountName: tensormesh-operator-engine-privileged
If the Deployment is in another namespace, that namespace needs its own ServiceAccount plus SCC binding.

​
Other settings that are optional

These are often useful, but they are not the core LMCache wiring:
  • HF_HOME=/tmp/hf
    • useful on OpenShift because random UIDs need a writable cache path
  • HF_HUB_DISABLE_TELEMETRY=1
    • hygiene only
  • readiness/liveness tuning
    • depends on model size and boot time

​
What usually breaks

The common failure modes are:
  • mounting the ConfigMap but forgetting hostIPC: true
  • pinning the engine image but leaving the vLLM pod on latest
  • updating the Deployment spec but still having an old ReplicaSet pod alive
  • running vLLM in a different namespace and trying to mount a cross-namespace ConfigMap
  • using OpenShift without a ServiceAccount/SCC that permits hostIPC
  • assuming the vLLM image has external lmcache installed when it does not
  • using a vLLM build too old to honor kv_connector_module_path correctly

​
Verify the patched deployment

After patching and rolling out the Deployment: 
kubectl rollout status deploy/<your-vllm-deployment> --timeout=15m
kubectl get pod -l app=<your-label> -o wide
kubectl exec -it <vllm-pod> -- \
  python3 -c 'import lmcache.integration.vllm.lmcache_mp_connector as m; print(m.__file__)'
Then send two identical long prompts through the vLLM Service and check the engine logs. Port-forward the Service in one terminal: 
kubectl port-forward -n <your-namespace> svc/<your-vllm-service> 8000:8000
In a second terminal, build the request once and send it twice: 
PROMPT=$(python3 -c "print('Tell me a long story about a brave knight. ' * 30)")
PAYLOAD=$(jq -n --arg p "$PROMPT" \
  '{model:"Qwen/Qwen3-0.6B", prompt:$p, max_tokens:20, temperature:0}')

# Call 1 — cold, should STORE KV into LMCache
time curl -sS http://localhost:8000/v1/completions \
  -H 'Content-Type: application/json' -d "$PAYLOAD" | jq -r '.choices[0].text'

# Call 2 — identical, should RETRIEVE prefix KV from LMCache
time curl -sS http://localhost:8000/v1/completions \
  -H 'Content-Type: application/json' -d "$PAYLOAD" | jq -r '.choices[0].text'
Then inspect the engine logs: 
kubectl logs -n <your-namespace> -l app.kubernetes.io/component=cache-engine --tail=200 \
  | grep -E "Stored [0-9]+ tokens|Prefetch request completed.*prefix hits"
Healthy behavior is:
  • call 1 prints a normal completion and the engine logs Stored N tokens
  • call 2 sends the exact same payload and the engine logs Prefetch request completed ... prefix hits=N
  • call 2 is usually noticeably faster than call 1 for a long enough prompt
Keep PROMPT and PAYLOAD in the same shell as the curl commands. If $PAYLOAD is empty, the request carries no prompt and you will see no LMCache store/prefetch markers.

​
Next steps

Install on OpenShift

OpenShift-specific SCC and ServiceAccount requirements.

End-to-end example

Full install plus demo deployment, smoke test, and benchmark.