Install from the OpenShift software catalog (certified operator)

Tensormesh Platform is a Red Hat Certified Operator: on OpenShift you install it from the in-cluster software catalog, and Operator Lifecycle Manager (OLM) installs and upgrades the operator. Its images are certified by Red Hat and distributed from the Tensormesh registry, so you add your registry access token to the cluster before installing. New to Tensormesh Platform? Start with the Introduction.

This is a different install path from Helm. OLM installs the operator, the CRDs, the mutating webhooks and their serving certificates, so cert-manager is not needed. It does not create the resources the Helm chart creates for you; this page covers them. Don’t install from the catalog on a cluster that already runs the Helm chart: both own the same CRDs and webhooks. Uninstall the chart first, and note that deleting its CRDs also deletes every engine and coordinator it created.

Step	Required	Why you do it yourself
1–3: Registry access token, operator, namespace	Yes	Every image is pulled from the Tensormesh registry. Without the token the operator pod cannot start.
4: Coordinator and engine	Yes	The Helm chart creates a coordinator and an engine by default. On OLM you create them yourself.
5: Connect vLLM	Yes	You opt your vLLM pods in with a label and an annotation.
6: CacheBlend or 7: SGLang	Alternative to steps 4–5	Their engines share the host IPC namespace with the serving pods, so a cluster administrator grants a least-privilege SCC.
8: Network access	Recommended	Limits who can reach the cache server port.
9: Observability	Optional	The OpenTelemetry Collector needs an operator you install.
Upgrade and uninstall	Day 2	Image tags live in your custom resources, and custom resources must be deleted before the operator.

Prerequisites

OpenShift 4.20 or later

The certified bundle supports OpenShift 4.20 and later. On an older cluster the operator does not appear in the catalog.

NVIDIA GPU nodes and the GPU Operator, driver R580 or later

GPU nodes are x86_64 with NVIDIA data-center GPUs. Install the NVIDIA GPU Operator (with its Node Feature Discovery dependency) from the software catalog and create its default ClusterPolicy. The serving image is built on CUDA 13.0, so GPU nodes need an NVIDIA driver R580 or later. The GPU Operator installs the driver too, and which version depends on the GPU Operator release: older releases default to an earlier branch (v25.3 defaults to R570), so check the driver it deployed. The GPU Operator also labels GPU nodes nvidia.com/gpu.present=true and registers the nvidia RuntimeClass, which the cache engines use. On GPU Operator versions before v25.10.0, also enable CDI.Check that your GPU nodes are labeled and that the driver is R580 or later:

oc get nodes -l nvidia.com/gpu.present=true   # must list your GPU nodes
oc exec -n nvidia-gpu-operator -c nvidia-driver-ctr \
  "$(oc get pod -n nvidia-gpu-operator -l app.kubernetes.io/component=nvidia-driver -o name | head -1)" \
  -- nvidia-smi --query-gpu=driver_version --format=csv,noheader   # 580 or later

Node resources

Each cache engine runs one pod on every GPU node. The pod requests 4 CPUs and l1.sizeGB + 5 GiB of memory (limit 1.5×), on top of your vLLM pods. The serving image is about 10 GB, so allow at least 30 GB of free container storage on each GPU node.

cluster-admin

Installing an operator, editing the cluster pull secret and granting an SCC are cluster-admin actions.

A Tensormesh registry access token

The access token from the Tensormesh team, the same one used for Helm installs and the CLI. If you don’t have it, contact Tensormesh. Every Tensormesh image (the operator, the serving image that also runs the cache engines and the coordinator, the LMCache payload and the CacheBlend plugin) is pulled from artifacts.tensormesh.ai.

Network, proxy and FIPS

The examples download model weights from Hugging Face (huggingface.co). The operator does not pass the cluster-wide proxy settings to the pods it creates or connects: if your cluster uses a proxy, set HTTP_PROXY, HTTPS_PROXY and NO_PROXY on your vLLM Deployment and through spec.env on the engines, and include .svc, .cluster.local and your cluster’s pod and service networks in NO_PROXY, so that traffic between vLLM, the engines and the coordinator stays inside the cluster. Disconnected (air-gapped) clusters are not supported in this release, and Tensormesh Platform is not supported on clusters installed in FIPS mode: its images are not built for FIPS 140 compliance.

1. Add the Tensormesh registry access token

Do this first. OLM pulls the operator image as soon as you install, so without the token the operator pod stays in ImagePullBackOff. Add the token to the cluster’s global pull secret, which covers every node and every namespace. The pull secret holds every registry credential of the cluster, so the commands run in a private temporary directory, and the token is read from a prompt rather than typed on the command line:

read -rs -p "Tensormesh registry access token: " TOKEN; echo
(
  cd "$(mktemp -d)" && umask 077
  oc get secret/pull-secret -n openshift-config \
    --template='{{index .data ".dockerconfigjson" | base64decode}}' > pull-secret.json
  oc registry login --registry="artifacts.tensormesh.ai" \
    --auth-basic="x-access-token:$TOKEN" --to=pull-secret.json
  oc set data secret/pull-secret -n openshift-config \
    --from-file=.dockerconfigjson=pull-secret.json
  rm -f pull-secret.json
)
unset TOKEN

This is Red Hat’s procedure for updating the global cluster pull secret, with the Tensormesh registry filled in. The Machine Config Operator rolls the new secret out to every node without draining or rebooting them. Check the entry, then wait until every machine config pool has finished updating:

oc get secret/pull-secret -n openshift-config \
  --template='{{index .data ".dockerconfigjson" | base64decode}}' | grep -c artifacts.tensormesh.ai   # 1
oc get mcp   # wait until every pool shows UPDATED True and UPDATING False

On hosted control planes (ROSA with HCP, HyperShift), follow Red Hat’s procedure for adding a pull secret to a hosted cluster instead.

2. Install the operator

In the OpenShift web console, go to Ecosystem → Software Catalog.
Search for Tensormesh Platform; the listing carries the Certified badge. Click Install.
Keep the stable channel and the default installation mode, All namespaces on the cluster, and confirm. For Update approval, see Upgrade: with Manual, you decide when the operator upgrades, but OLM then applies Manual approval to every operator in the openshift-operators namespace and upgrades them together.

OLM installs the operator into the openshift-operators namespace, together with the three CRDs and the mutating webhooks. Check that it is running:

oc get csv -n openshift-operators | grep tensormesh-platform   # PHASE: Succeeded
oc get pods -n openshift-operators | grep tensormesh-platform  # STATUS: Running
oc get crd | grep lmcache.lmcache.ai
# cacheblendengines.lmcache.lmcache.ai
# lmcachecoordinators.lmcache.lmcache.ai
# lmcacheengines.lmcache.lmcache.ai

If the operator pod is in ImagePullBackOff, check step 1, then delete the pod once the pull-secret rollout has finished; it restarts and pulls the image.

3. Prepare a workload namespace

The coordinator, the cache engine and the vLLM pods that use it must all run in the same namespace: an engine finds its coordinator in its own namespace, the webhook looks up the engine in the pod’s namespace, and OpenShift gives each namespace its own UID range, so pods in different namespaces can’t share memory with an engine. This page uses llm-serving; if you use another namespace, replace llm-serving everywhere below. Create the namespace and a ServiceAccount for the Tensormesh workloads:

oc create namespace llm-serving
oc create serviceaccount tensormesh -n llm-serving

The coordinator, the engines and the vLLM examples below all run as the tensormesh ServiceAccount. The default engine needs no extra permissions; CacheBlend and SGLang need host IPC, which you grant to this ServiceAccount there.

4. Create the coordinator and the engine

The LMCacheCoordinator tracks which cache engines are alive and drives cache eviction across them; engines register with it at startup. Every setup needs one coordinator per namespace. Then choose one engine for the namespace. Each engine runs a pod on every GPU node and reserves its l1.sizeGB of memory there:

the default LMCacheEngine below, for vLLM (steps 4 and 5);
a CacheBlendEngine, for RAG workloads (step 6);
an LMCacheEngine in host-IPC mode, for SGLang (step 7).

You can run two engines side by side only if both memory reservations fit on every GPU node. Set spec.image on every resource as shown. If it is unset, the operator uses a latest tag, which for engines is the public community image, not the certified build. The samples in the console’s Create instance form carry the certified images but no serviceAccountName: add serviceAccountName: tensormesh when you start from them.

coordinator.yaml

apiVersion: lmcache.lmcache.ai/v1alpha1
kind: LMCacheCoordinator
metadata:
  name: coordinator
  namespace: llm-serving
spec:
  serviceAccountName: tensormesh
  image:
    repository: artifacts.tensormesh.ai/tensormesh-production/images/coordinator-controllers
    tag: v0.5.6-ubi

engine.yaml

apiVersion: lmcache.lmcache.ai/v1alpha1
kind: LMCacheEngine
metadata:
  name: engine
  namespace: llm-serving
spec:
  gpuVendor: nvidia
  serviceAccountName: tensormesh
  image:
    repository: artifacts.tensormesh.ai/tensormesh-production/images/vllm-openai
    tag: v0.5.6-ubi
  coordinator:
    ref:
      name: coordinator
  l1:
    sizeGB: 40           # first cache tier: CPU memory reserved on each GPU node

With this l1, the engine pod requests 45 GiB of memory on every GPU node (see Node resources); size l1 so that it fits next to your vLLM pods. On NVIDIA GPUs the engine runs in isolated IPC mode by default (isolatedIPC unset): the engine and the connected vLLM pods exchange KV cache over CUDA IPC, and each vLLM pod gets a private in-memory /dev/shm. Isolated IPC needs no host IPC and no extra SCC. The coordinator and engine images are built on the serving image, so the first start on a node can take 15 to 20 minutes while it pulls. Wait for both resources to report Running:

oc apply -f coordinator.yaml -f engine.yaml
oc -n llm-serving wait lmcachecoordinator/coordinator lmcacheengine/engine \
  --for=jsonpath='{.status.phase}'=Running --timeout=30m
oc -n llm-serving get lmcachecoordinator,lmcacheengine
# NAME                                                PHASE     READY   REPLICAS   ENDPOINT
# lmcachecoordinator.lmcache.lmcache.ai/coordinator   Running   1       1          http://coordinator.llm-serving.svc:9300
# NAME                                      PHASE     READY   DESIRED
# lmcacheengine.lmcache.lmcache.ai/engine   Running   1       1

The engine’s READY count should equal your number of GPU nodes. If no node carries nvidia.com/gpu.present=true, the engine stays Pending and the wait times out. For storage tiers and sizing, see filesystem offloading, KV cache encryption and cache sizing.

5. Connect your vLLM deployment

Add one label and one annotation to the vLLM pod template; the labels call this injection. When the pod is created, the operator’s webhook adds the LMCache connection settings and the shared-memory access, so you don’t write any LMCache configuration yourself. Create the vLLM Deployment after the engine is Running: vLLM connects to the engine when it starts. If the webhook can’t connect a pod, it lets the pod start without LMCache rather than block it, so check the result below.

vllm.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: vllm
  namespace: llm-serving
spec:
  replicas: 1
  progressDeadlineSeconds: 1800             # the first start pulls a large image and the model
  strategy:
    type: Recreate                          # free the GPU before starting the new pod
  selector:
    matchLabels: { app: vllm }
  template:
    metadata:
      labels:
        app: vllm
        lmcache.ai/lmcache-inject: "true"     # opt in to the webhook
      annotations:
        lmcache.ai/lmcache-engine: "engine"   # the LMCacheEngine in this namespace
    spec:
      serviceAccountName: tensormesh
      containers:
        - name: vllm
          image: artifacts.tensormesh.ai/tensormesh-production/images/vllm-openai:v0.5.6-ubi
          args: ["--model", "Qwen/Qwen3-8B", "--port", "8000"]
          ports: [{ containerPort: 8000 }]
          readinessProbe:                     # Ready only once the model is loaded
            httpGet: { path: /health, port: 8000 }
            periodSeconds: 10
            failureThreshold: 90
          resources:
            limits: { nvidia.com/gpu: 1 }
          volumeMounts:
            - { name: model-cache, mountPath: /tmp/huggingface }   # the image's HF_HOME
      volumes:
        - name: model-cache
          emptyDir: {}                        # or a PersistentVolumeClaim to keep weights across restarts

Leave command unset on the vLLM container: the webhook skips containers that override it. The image’s entrypoint is vllm serve, so args starts with the model options.

oc apply -f vllm.yaml
oc -n llm-serving rollout status deployment/vllm --timeout=30m   # downloads the model on first start

Check that the pod was connected:

oc get pod -n llm-serving -l app=vllm \
  -o jsonpath='{.items[0].metadata.annotations}' | tr ',' '\n' | grep lmcache
# "lmcache.ai/lmcache-injected":"true"   -> connected
# "lmcache.ai/lmcache-skip-reason":"..."  -> not connected; see Troubleshooting

If the skip reason is engine-not-found because you created the Deployment before the engine, run oc -n llm-serving rollout restart deployment/vllm once the engine is Running. Then check that the cache works end to end. Send a long prompt, restart vLLM so that its GPU memory is empty, and send the same prompt again: the engine stores the KV cache the first time and serves it back to the new pod the second time. The new pod downloads the model again unless model-cache is a PersistentVolumeClaim.

ASK='import json, urllib.request
prompt = "Tell me a long story about a brave knight. " * 60
req = urllib.request.Request("http://localhost:8000/v1/completions",
    data=json.dumps({"model": "Qwen/Qwen3-8B", "prompt": prompt, "max_tokens": 20}).encode(),
    headers={"Content-Type": "application/json"})
print(json.load(urllib.request.urlopen(req))["usage"])'

oc -n llm-serving exec deploy/vllm -- python3 -c "$ASK"
oc -n llm-serving rollout restart deployment/vllm
oc -n llm-serving rollout status deployment/vllm --timeout=30m
oc -n llm-serving exec deploy/vllm -- python3 -c "$ASK"

# read the log of the engine on the vLLM pod's node
NODE=$(oc -n llm-serving get pod -l app=vllm -o jsonpath='{.items[0].spec.nodeName}')
ENGINE_POD=$(oc -n llm-serving get pod -l app.kubernetes.io/instance=engine \
  --field-selector spec.nodeName="$NODE" -o name)
oc -n llm-serving logs "$ENGINE_POD" | grep -E "Stored|Retrieved"
# ... LMCache INFO: Stored 512 tokens in 0.004 seconds ...      <- first request
# ... LMCache INFO: Retrieved 512 tokens in 0.003 seconds ...   <- after the restart

By default each node’s engine keeps its own cache, so on a cluster with several GPU nodes the restarted pod may land on a node whose engine hasn’t seen the prompt, and you see only Stored there. A shared filesystem cache or P2P lets engines reuse each other’s cache. The example uses the certified serving image, which already contains LMCache. To keep your own vLLM image, add the LMCache payload image to engine.yaml; the webhook then adds the matching LMCache client to your pods (see payload injection for how it works; the Helm and cert-manager steps there don’t apply to OLM):

spec:
  injection:
    payloadImage:
      repository: artifacts.tensormesh.ai/tensormesh-production/images/lmcache-payload
      tag: v0.5.6-ubi

6. CacheBlend (optional)

CacheBlend reuses KV cache for text that is not at the start of the prompt, such as retrieved documents in RAG.

Allow host IPC

A CacheBlend engine always shares /dev/shm with its vLLM pods. On OpenShift it does that through the host IPC namespace (hostIPC: true), and the webhook gives the connected vLLM pods host IPC too. OpenShift’s default restricted-v2 SCC forbids host IPC, so apply the least-privilege lmcache-engine SCC. It allows host IPC and nothing more: non-root, all capabilities dropped, no privileged containers, no host network, PID or ports, no hostPath volumes, seccomp RuntimeDefault. GPUs reach the pods through the NVIDIA RuntimeClass and CDI, not host paths.

lmcache-engine-scc.yaml

apiVersion: security.openshift.io/v1
kind: SecurityContextConstraints
metadata:
  name: lmcache-engine
  annotations:
    kubernetes.io/description: >-
      Least-privilege SCC for Tensormesh Platform cache engines and the vLLM pods connected
      to them: host IPC, and otherwise no more than restricted-v2 allows.
allowHostIPC: true
allowHostNetwork: false
allowHostPID: false
allowHostPorts: false
allowPrivilegedContainer: false
allowPrivilegeEscalation: false
allowHostDirVolumePlugin: false
defaultAddCapabilities: null
allowedCapabilities: null
requiredDropCapabilities: ["ALL"]
runAsUser: { type: MustRunAsRange }
seLinuxContext: { type: MustRunAs }
fsGroup: { type: MustRunAs }
supplementalGroups: { type: RunAsAny }
seccompProfiles: ["runtime/default"]
readOnlyRootFilesystem: false
volumes: [configMap, downwardAPI, emptyDir, persistentVolumeClaim, projected, secret]
users: []
groups: []

A cluster administrator applies it once per cluster, then lets the tensormesh ServiceAccount use it in each namespace that runs CacheBlend or SGLang. Only pods that run as that ServiceAccount get host IPC, but anyone who can create pods in the namespace can choose that ServiceAccount, so grant it only in namespaces you trust:

oc apply -f lmcache-engine-scc.yaml

oc create role lmcache-engine-scc --verb=use \
  --resource=scc --resource-name=lmcache-engine -n llm-serving
oc create rolebinding lmcache-engine-scc --role=lmcache-engine-scc \
  --serviceaccount=llm-serving:tensormesh -n llm-serving

OpenShift admits pods through SCCs, and the namespace-level Pod Security enforce mode is privileged by default, so no namespace label is needed. If your cluster policy sets a stricter pod-security.kubernetes.io/enforce label on namespaces, set it to privileged on this one; host IPC is outside the baseline and restricted profiles.

Create the CacheBlend engine

If you followed steps 4 and 5 and are switching to CacheBlend, delete the step-5 vLLM Deployment and the step-4 engine first, to free the GPU and the engine’s memory: oc delete deployment vllm -n llm-serving and oc delete lmcacheengine engine -n llm-serving.

cacheblend.yaml

apiVersion: lmcache.lmcache.ai/v1alpha1
kind: CacheBlendEngine
metadata:
  name: cacheblend
  namespace: llm-serving
spec:
  hostIPC: true                               # share /dev/shm through host IPC; the SCC forbids the hostPath alternative
  gpuVendor: nvidia
  serviceAccountName: tensormesh
  image:
    repository: artifacts.tensormesh.ai/tensormesh-production/images/vllm-openai
    tag: v0.5.6-ubi
  coordinator:
    ref:
      name: coordinator
  l1:
    sizeGB: 40
  blend:
    checkLayer: 1                             # depends on the model: 1 for Qwen3-8B
  injection:
    payloadImage:                             # the CacheBlend plugin; required
      repository: artifacts.tensormesh.ai/tensormesh-production/images/cacheblend-plugin
      tag: v0.5.6-ubi

Set blend.checkLayer for your model from the Model Configurations table. A wrong value causes no errors, but some responses come back incorrect.

oc apply -f cacheblend.yaml
oc -n llm-serving wait cacheblendengine/cacheblend --for=jsonpath='{.status.phase}'=Running --timeout=30m

Connect vLLM to CacheBlend

Use the CacheBlend label and annotation instead of the step-5 ones. This is the step-5 Deployment with the opt-in swapped; the webhook adds the CacheBlend plugin and vLLM flags:

vllm-cacheblend.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: vllm-cacheblend
  namespace: llm-serving
spec:
  replicas: 1
  progressDeadlineSeconds: 1800             # the first start pulls a large image and the model
  strategy:
    type: Recreate                          # free the GPU before starting the new pod
  selector:
    matchLabels: { app: vllm-cacheblend }
  template:
    metadata:
      labels:
        app: vllm-cacheblend
        lmcache.ai/cacheblend-inject: "true"    # opt in to CacheBlend
      annotations:
        lmcache.ai/cacheblend-engine: "cacheblend"
    spec:
      serviceAccountName: tensormesh            # bound to the lmcache-engine SCC above
      containers:
        - name: vllm
          image: artifacts.tensormesh.ai/tensormesh-production/images/vllm-openai:v0.5.6-ubi
          args: ["--model", "Qwen/Qwen3-8B", "--port", "8000"]
          ports: [{ containerPort: 8000 }]
          readinessProbe:
            httpGet: { path: /health, port: 8000 }
            periodSeconds: 10
            failureThreshold: 90
          resources:
            limits: { nvidia.com/gpu: 1 }
          volumeMounts:
            - { name: model-cache, mountPath: /tmp/huggingface }
      volumes:
        - name: model-cache
          emptyDir: {}

oc apply -f vllm-cacheblend.yaml
oc -n llm-serving rollout status deployment/vllm-cacheblend --timeout=30m
oc get pod -n llm-serving -l app=vllm-cacheblend \
  -o jsonpath='{.items[0].metadata.annotations}' | tr ',' '\n' | grep cacheblend
# "lmcache.ai/cacheblend-injected":"true"   -> connected

If no pod appears, run oc -n llm-serving describe replicaset -l app=vllm-cacheblend. The message unable to validate against any security context constraint means the SCC binding above is missing. Once your RAG traffic repeats documents at different positions in the prompt, the counters of the engine on the vLLM pod’s node show how many prompt tokens CacheBlend reused at non-prefix positions:

NODE=$(oc -n llm-serving get pod -l app=vllm-cacheblend -o jsonpath='{.items[0].spec.nodeName}')
CB_POD=$(oc -n llm-serving get pod -l app.kubernetes.io/instance=cacheblend \
  --field-selector spec.nodeName="$NODE" -o name)
oc -n llm-serving exec "$CB_POD" -- curl -s localhost:8080/metrics \
  | grep -E '^lmcache_blend_lookup_(non_prefix_hit|requested)_tokens'
# lmcache_blend_lookup_requested_tokens_total 71680.0
# lmcache_blend_lookup_non_prefix_hit_tokens_total 33792.0   <- greater than 0: blending is active

The CacheBlend guide covers the CacheBlend settings; its Helm steps don’t apply here.

7. SGLang (optional)

The webhook also connects SGLang servers. SGLang’s LMCache connector supports only CUDA IPC over a shared /dev/shm, not isolated IPC, so on OpenShift it needs an engine with isolatedIPC: false and hostIPC: true, and the host IPC grant from step 6. The webhook skips SGLang pods whose engine runs isolated IPC. Like CacheBlend, this engine is usually created instead of the step-4 one: if you are switching from step 5 or 6, delete that vLLM Deployment and engine first.

sglang-engine.yaml

apiVersion: lmcache.lmcache.ai/v1alpha1
kind: LMCacheEngine
metadata:
  name: sglang-engine
  namespace: llm-serving
spec:
  isolatedIPC: false
  hostIPC: true                               # share /dev/shm through host IPC; the SCC forbids the hostPath alternative
  gpuVendor: nvidia
  serviceAccountName: tensormesh
  image:
    repository: artifacts.tensormesh.ai/tensormesh-production/images/vllm-openai
    tag: v0.5.6-ubi
  coordinator:
    ref:
      name: coordinator
  l1:
    sizeGB: 40

The SGLang image is yours to supply; it is not one of the certified images. It must contain LMCache at the engine’s version, such as the community image lmcache/sglang:v0.5.6. Opt SGLang pods in with the lmcache.ai/sglang-inject label and the lmcache.ai/lmcache-engine annotation; the webhook adds --enable-lmcache and the engine address. OpenShift runs the pod as a non-root user, so point HOME at a writable directory:

sglang.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sglang
  namespace: llm-serving
spec:
  replicas: 1
  progressDeadlineSeconds: 1800             # the first start pulls a large image and the model
  strategy:
    type: Recreate                          # free the GPU before starting the new pod
  selector:
    matchLabels: { app: sglang }
  template:
    metadata:
      labels:
        app: sglang
        lmcache.ai/sglang-inject: "true"
      annotations:
        lmcache.ai/lmcache-engine: "sglang-engine"
    spec:
      serviceAccountName: tensormesh            # bound to the lmcache-engine SCC
      containers:
        - name: sglang
          image: lmcache/sglang:v0.5.6
          command: ["python3", "-m", "sglang.launch_server"]
          args: ["--model-path", "Qwen/Qwen3-8B", "--host", "0.0.0.0", "--port", "30000"]
          env:
            - { name: HOME, value: /tmp }
            - { name: HF_HOME, value: /tmp/huggingface }
          ports: [{ containerPort: 30000 }]
          readinessProbe:
            httpGet: { path: /health, port: 30000 }
            periodSeconds: 10
            failureThreshold: 90
          resources:
            limits: { nvidia.com/gpu: 1 }
          volumeMounts:
            - { name: model-cache, mountPath: /tmp/huggingface }
      volumes:
        - name: model-cache
          emptyDir: {}

oc apply -f sglang-engine.yaml
oc -n llm-serving wait lmcacheengine/sglang-engine --for=jsonpath='{.status.phase}'=Running --timeout=30m
oc apply -f sglang.yaml
oc -n llm-serving rollout status deployment/sglang --timeout=30m
oc get pod -n llm-serving -l app=sglang \
  -o jsonpath='{.items[0].metadata.annotations}' | tr ',' '\n' | grep lmcache
# "lmcache.ai/sglang-injected":"true"   -> connected

After the first long requests, the log of the sglang-engine pod on the SGLang pod’s node shows Stored lines, as in step 5.

8. Restrict network access (recommended)

The cache server port accepts connections from any pod in the cluster unless you restrict it. Set spec.networkPolicy.enabled: true on each LMCacheEngine and CacheBlendEngine, in the YAML or with a patch:

oc -n llm-serving patch lmcacheengine engine --type=merge \
  -p '{"spec":{"networkPolicy":{"enabled":true}}}'

The operator then creates a NetworkPolicy named <engine>-ingress for the engine’s pods:

The cache server port (5555 by default) accepts only pods that carry one of the webhook’s opt-in labels set to "true", from any namespace: lmcache.ai/lmcache-inject or lmcache.ai/sglang-inject for an LMCacheEngine, lmcache.ai/cacheblend-inject for a CacheBlendEngine. This limits who can reach the cache, but it is not tenant isolation: any pod with that label can connect.
The HTTP port (8080), which serves status, metrics and the cache operations the CLI uses, the metrics port (9090) and, when enabled, the P2P port stay open to all pods. Add your own NetworkPolicy to restrict them.
All other incoming connections to the engine pods are blocked.

Setting the field back to false deletes the policy. Check that it took effect:

oc get lmcacheengine engine -n llm-serving \
  -o jsonpath='{.status.conditions[?(@.type=="NetworkPolicyEffective")].status}'   # True

The policy needs a network plugin that enforces NetworkPolicy; OVN-Kubernetes, the OpenShift default, does. It has no effect on pods that use hostNetwork: true.

9. Observability (optional)

The OLM install does not create the OpenTelemetry Collector or the ServiceMonitors that helm install --set observability.enabled=true does. Without a collector, each engine serves Prometheus metrics at /metrics on its HTTP port (8080). To collect engine metrics and traces centrally, install the Red Hat build of OpenTelemetry operator from the software catalog and create a collector in the workload namespace:

collector.yaml

apiVersion: opentelemetry.io/v1beta1
kind: OpenTelemetryCollector
metadata:
  name: tensormesh
  namespace: llm-serving
spec:
  mode: deployment
  config:
    receivers:
      otlp:
        protocols:
          grpc: { endpoint: "0.0.0.0:4317" }
    exporters:
      prometheus: { endpoint: "0.0.0.0:8889" }
      debug: {}
    service:
      pipelines:
        metrics: { receivers: [otlp], exporters: [prometheus] }
        traces: { receivers: [otlp], exporters: [debug] }

Then point the engine at it: --otlp-endpoint sends the engine’s metrics and traces to the collector, and --enable-tracing turns on tracing. Scrape the collector (port 8889) instead of the engine:

oc apply -f collector.yaml
oc -n llm-serving patch lmcacheengine engine --type=merge \
  -p '{"spec":{"extraArgs":["--enable-tracing","--otlp-endpoint=http://tensormesh-collector.llm-serving.svc:4317"]}}'

To send traces to a tracing backend such as Tempo, replace the debug exporter with that backend’s OTLP exporter. To scrape the collector with the cluster’s monitoring stack, enable monitoring for user-defined projects and create a ServiceMonitor for the collector’s tensormesh-collector Service. The metrics themselves are described in Observability.

Upgrade

The operator, its CRDs and its webhooks upgrade through OLM, according to the Update approval you chose in step 2: with Automatic, OLM upgrades the operator as soon as a new version reaches the stable channel; with Manual, you approve the install plan in Ecosystem → Installed Operators → Tensormesh Platform. The engine, coordinator and serving images don’t upgrade with the operator: they are the tags set in your custom resources and Deployments. After each operator upgrade:

Check the OpenShift certified bundle table for the image tags of the new version.
Update spec.image.tag, and spec.injection.payloadImage.tag where set, on every LMCacheCoordinator, LMCacheEngine and CacheBlendEngine, the image of your vLLM Deployments, and your SGLang image (LMCache at the new engine version). Mixing versions across the operator and the engines is not supported.

OLM does not roll an operator back to an earlier version; if an upgrade causes problems, contact Tensormesh.

Uninstall

Delete the custom resources before the operator. OLM does not delete them when you uninstall the operator, so their engine DaemonSets and coordinator would keep running, unmanaged, until you delete them.

First delete your vLLM and SGLang Deployments, then the custom resources in every namespace, and wait until they are gone:

oc delete cacheblendengine,lmcacheengine,lmcachecoordinator --all -A
oc get cacheblendengine,lmcacheengine,lmcachecoordinator -A   # expect: No resources found

Then uninstall the operator. In the console, open Ecosystem → Installed Operators → Tensormesh Platform and choose Actions → Uninstall Operator. Or use the CLI, following Red Hat’s procedure for deleting an Operator:

CSV=$(oc get subscription.operators.coreos.com tensormesh-platform -n openshift-operators \
  -o jsonpath='{.status.currentCSV}')
oc delete subscription.operators.coreos.com tensormesh-platform -n openshift-operators
oc delete csv "$CSV" -n openshift-operators

The CRDs, the lmcache-engine SCC, the namespace grants and your registry access token stay in place, so a reinstall picks up where you left off. Remove them only if you are done with Tensormesh Platform:

oc delete crd lmcacheengines.lmcache.lmcache.ai \
  lmcachecoordinators.lmcache.lmcache.ai cacheblendengines.lmcache.lmcache.ai
oc delete rolebinding,role lmcache-engine-scc -n llm-serving
oc delete serviceaccount tensormesh -n llm-serving
oc delete scc lmcache-engine

To remove the registry access token, edit the global pull secret as in step 1 and delete the artifacts.tensormesh.ai entry.

Troubleshooting

Symptom	Check
Operator or engine pod in `ImagePullBackOff`	The registry access token (step 1) and that the pull-secret rollout finished (`oc get mcp`).
An engine stays `Pending`, or its `READY` count is lower than your GPU nodes	`oc -n llm-serving describe daemonset <engine>`; a pod stuck `Pending` usually lacks CPU or memory for the engine’s request (see Node resources). With no GPU-labeled node, the engine stays `Pending`.
No CacheBlend or SGLang pod appears	`oc -n llm-serving describe replicaset -l app=<name>`; `unable to validate against any security context constraint` means the host IPC grant is missing.
A pod has `lmcache.ai/lmcache-skip-reason` (vLLM), `lmcache.ai/cacheblend-skip-reason` or `lmcache.ai/sglang-skip-reason`	`engine-not-found`: no engine with that name exists in the pod’s namespace (check the annotation, the namespace, and that the engine was created first). `command-override`: the vLLM container sets `command`. `target-container-not-found`: the container named by the pod’s annotation doesn’t exist. `isolated-ipc-unsupported` (SGLang): the engine runs isolated IPC. `payload-image-unset` (CacheBlend): set `injection.payloadImage`. Fix it, then restart the Deployment. `kv-transfer-config-present` is informational: the pod is connected and your own `--kv-transfer-config` was kept.

For more, see Troubleshooting; its Helm commands don’t apply to an OLM install.

Next steps

CacheBlend

Model settings and how to check that blending is active.

CLI

tmo-cli checks engine status, clears the cache and runs diagnostics. It installs separately, with the same registry access token.

Overview

Installation

Features

Operations

Integrations

Operator UI

Reference

Install from the OpenShift software catalog (certified operator)

Prerequisites

1. Add the Tensormesh registry access token

2. Install the operator

3. Prepare a workload namespace

4. Create the coordinator and the engine

5. Connect your vLLM deployment

6. CacheBlend (optional)

Allow host IPC

Create the CacheBlend engine

Connect vLLM to CacheBlend

7. SGLang (optional)

8. Restrict network access (recommended)

9. Observability (optional)

Upgrade

Uninstall

Troubleshooting

Next steps

CacheBlend

CLI

​Prerequisites

​1. Add the Tensormesh registry access token

​2. Install the operator

​3. Prepare a workload namespace

​4. Create the coordinator and the engine

​5. Connect your vLLM deployment

​6. CacheBlend (optional)

​Allow host IPC

​Create the CacheBlend engine

​Connect vLLM to CacheBlend

​7. SGLang (optional)

​8. Restrict network access (recommended)

​9. Observability (optional)

​Upgrade

​Uninstall

​Troubleshooting

​Next steps

CacheBlend

CLI

Prerequisites

1. Add the Tensormesh registry access token

2. Install the operator

3. Prepare a workload namespace

4. Create the coordinator and the engine

5. Connect your vLLM deployment

6. CacheBlend (optional)

Allow host IPC

Create the CacheBlend engine

Connect vLLM to CacheBlend

7. SGLang (optional)

8. Restrict network access (recommended)

9. Observability (optional)

Upgrade

Uninstall

Troubleshooting

Next steps