Tensormesh / LMCache
| Term | Meaning |
|---|---|
| LMCache | A multi-process KV (key-value) cache for LLM inference. When requests share a prefix (system prompt, RAG context, prior turn), LMCache lets later requests reuse the earlier request’s attention KV tensors instead of recomputing them. |
| Tensormesh Operator (TMO) | The Kubernetes controller this chart installs. It watches LMCacheEngine resources and reconciles them into a DaemonSet of cache servers. |
| Engine | A running LMCache cache server pod (one per GPU node, via a DaemonSet). Created by the operator from an LMCacheEngine CR. |
LMCacheEngine | The custom resource you write to declare an engine. Group/version lmcache.lmcache.ai/v1alpha1. |
MP mode / LMCacheMPConnector | ”Multi-process” mode — vLLM connects to the engine on the same node over shared-memory IPC (hostIPC). The connector ships KV blocks between vLLM and the engine. |
| L1 / L2 | Cache tiers. L1 is local node memory (sized by engine.spec.l1.sizeGB). L2 is an optional shared backend (Redis, S3, filesystem). |
| KV cache hit | A later request reused stored KV blocks for a shared prefix — the win LMCache delivers. Visible as prefix hits in engine logs and faster TTFT. |
Kubernetes
| Term | Meaning |
|---|---|
| CRD (CustomResourceDefinition) | Extends the Kubernetes API with a new resource kind. Cluster-scoped. Here: lmcacheengines.lmcache.lmcache.ai. |
| CR (Custom Resource) | An instance of a CRD-defined kind, e.g. an LMCacheEngine. |
| Operator / controller | A program that watches CRs and reconciles them into lower-level objects (Deployments, DaemonSets, …). |
| DaemonSet | Runs one pod per matching node — the shape used for the engine (one per GPU node). |
| Deployment | Maintains N identical pods with rolling updates — the shape used for the operator. |
| Finalizer | A marker on a resource that blocks its deletion until a controller runs cleanup. The LMCacheEngine’s lmcache.ai/cleanup finalizer can block a namespace delete if the operator is gone. |
| Namespaced vs cluster-scoped | Namespaced resources are removed when their namespace is deleted; cluster-scoped ones (CRD, ClusterRole, ClusterRoleBinding) survive and must be cleaned up explicitly. |
| SCC (Security Context Constraint) | OpenShift’s per-pod security policy. Engine pods need the privileged SCC because they use hostIPC. |
| ServiceMonitor | A Prometheus Operator CR telling Prometheus what to scrape. |
Helm
| Term | Meaning |
|---|---|
| Chart | A versioned, packaged bundle of Kubernetes templates + default values. |
| Release | A named installation of a chart in a namespace. Helm stores its history (as Secrets) to enable upgrade/rollback. |
| Values / values file | User configuration that overrides chart defaults — supplied via -f my-values.yaml (preferred) or --set. |
| appVersion vs version | version is the chart’s version; appVersion is the version of the app it deploys. The operator image tag defaults to appVersion. |
| OCI artifact | A chart stored in a container registry (here GHCR) using OCI media types — pulled with oci://… instead of a classic HTTP repo. |
| Hook | A Job/pod Helm runs at a lifecycle point (pre-install, pre-delete, test). The chart’s pre-delete hook removes the engine CR before the operator. |
Observability
| Term | Meaning |
|---|---|
| OTLP | OpenTelemetry Protocol — how the engine ships metrics/traces to the Collector (gRPC on :4317). |
| OTel Collector | Receives OTLP from the engine and exports onward (Prometheus, Tempo, external backends). Managed by the OpenTelemetry Operator from a CR the chart creates. |
Prometheus exporter (:8889) | The Collector port that re-exposes LMCache application metrics (lmcache_*) for scraping. |
Collector internals (:8888) | The Collector’s own metrics (otelcol_*) — used to confirm data is flowing. |
| Tempo | A trace storage backend. The chart can deploy a TempoMonolithic; you view traces in your own Grafana via a Tempo data source. |
remoteWrite | Pushing metrics to an external Prometheus-compatible endpoint (Grafana Cloud, Mimir, Thanos) instead of (or alongside) in-cluster scraping. |