Skip to main content
The Tensormesh Operator (TMO) is a Kubernetes controller that reconciles LMCacheEngine custom resources into a DaemonSet of LMCache cache servers — one per GPU node — that back vLLM (or any inference runtime) with shared KV cache reuse. This page covers what you need before installing and helps you pick an install method. Each method ends up deploying the same operator and the same CRD; the difference is how you manage the release.

​
What gets installed

Operator

A small controller-manager Deployment that watches LMCacheEngine resources cluster-wide.

CRD

LMCacheEngine (lmcache.lmcache.ai/v1alpha1) — the spec the operator reconciles.

Engine DaemonSet

One LMCache cache server pod per GPU node, created by the operator from your CR.

​
Prerequisites

The chart’s kubeVersion constraint is >=1.28.0-0. OpenShift 4.16+ and EKS / GKE / AKS on a supported control plane all qualify.
The engine DaemonSet schedules onto nodes labeled nvidia.com/gpu.present=true. On clusters with the NVIDIA GPU Operator installed, this label is set automatically. On clusters without it, label your GPU nodes manually:
kubectl label node <node-name> nvidia.com/gpu.present=true
On a cluster with no GPU nodes the install still succeeds — the DaemonSet stays at 0 desired replicas until a matching node appears.
The NVIDIA GPU Operator exposes GPUs via the Container Device Interface (CDI). Before GPU Operator v25.10.0, CDI was off by default (cdi.enabled: false) — and with it off, the engine/vLLM pods get stuck in ContainerCreating (no clear error). On those versions, enable CDI in the GPU ClusterPolicy:
oc patch clusterpolicy gpu-cluster-policy --type=merge -p '{
  "spec": { "cdi": { "enabled": true, "default": true } }
}'
GPU Operator v25.10.0+ enables CDI by default — no action needed. See Troubleshooting → Pod stuck in ContainerCreating if you hit this.
Installation creates cluster-scoped objects (a CRD, a ClusterRole, and a ClusterRoleBinding). You need permission to create those.
kubectl on Kubernetes, oc on OpenShift. The two are interchangeable for everything in these docs — examples are shown with kubectl unless an OpenShift-specific command is required.

​
Pick an install method

Helm

The primary path. Single helm install command, simple upgrades and rollbacks.

Modify Existing vLLM Deployment

Patch an existing vLLM Deployment to mount the engine connection ConfigMap and use LMCache MP mode safely.

GitOps (Argo CD)

Declarative, Git-driven installs across many clusters. Best for fleets and audited environments.

OpenShift

OpenShift adds one extra flag for the privileged SCC required by hostIPC. Read this after picking Helm or GitOps if your target is OpenShift.

Kustomize

Render the chart once with helm template and patch it with Kustomize overlays. For teams that standardize on Kustomize as their final renderer.

​
Verify your cluster is ready

Run these three preflight checks before installing.
1

Kubernetes is 1.28 or newer

kubectl version
Look at the Server Version line — anything v1.28.0 or newer qualifies. OpenShift users can also run oc version; the Kubernetes Version line is what matters.
2

At least one GPU node is labeled (or you accept an empty DaemonSet)

kubectl get nodes -l nvidia.com/gpu.present=true
A non-empty list means the engine DaemonSet has somewhere to schedule. An empty list is fine for an operator-only install — see Install modes.
3

You can create cluster-scoped resources

kubectl auth can-i create customresourcedefinitions.apiextensions.k8s.io
kubectl auth can-i create clusterroles.rbac.authorization.k8s.io
kubectl auth can-i create clusterrolebindings.rbac.authorization.k8s.io
All three must return yes. As a shortcut, kubectl auth can-i '*' '*' --all-namespaces returning yes means you’re effectively cluster-admin.Each command also prints a Warning: resource '...' is not namespace scoped line — that’s expected (these are cluster-scoped resources) and harmless. The yes/no answer is what matters.
If any check fails: bump the cluster version, label the GPU nodes (see the prereqs accordion above), or ask a cluster admin for the missing RBAC.

​
Next steps

Install with Helm

Five-minute install with helm install.

Modify Existing vLLM Deployment

Patch the workload you already run instead of using the demo deployment.

Troubleshooting

Common install and day-2 issues.