Glossary / Vocabulary - Tensormesh Platform Documentation

Short definitions of the terms you’ll meet in these docs, grouped by area.

Tensormesh

Term	Meaning
Tensormesh Platform	The product: self-hosted KV caching for your vLLM fleet, deployed on Kubernetes as an operator that reconciles your cache resources (`LMCacheEngine`, `LMCacheCoordinator`, `CacheBlendEngine`) into a running engine fleet.
vLLM	The open-source LLM inference server that serves your models. The cache engine runs alongside vLLM and offloads its KV cache, so shared prefixes are served from cache instead of recomputed on the GPU.
LMCache	The open-source KV (key-value) cache for LLM inference that the engine runs. When requests share a prefix (system prompt, RAG context, prior turn), it lets later requests reuse the earlier request’s attention KV tensors instead of recomputing them.
Engine	A running LMCache cache server pod (one per GPU node, via a DaemonSet). Created by the operator from an `LMCacheEngine` CR.
Coordinator / `LMCacheCoordinator`	A fleet-wide service the engines register with on startup (a Deployment plus a ClusterIP Service on port 9300), enabling cross-node cache awareness. Reconciled from an `LMCacheCoordinator` CR; enabled by default with server-side defaults, so no configuration is required.
`LMCacheEngine`	The custom resource you write to declare an engine. Group/version `lmcache.lmcache.ai/v1alpha1`.
CacheBlend / `CacheBlendEngine`	A Tensormesh technique (and its per-node CR) that reuses KV cache beyond exact prefix matches (any shared block of tokens, even after a per-request preamble), then recomputes a small subset to stitch the blocks together. Useful for RAG. See Non-Prefix KV Caching.
MP mode / `LMCacheMPConnector`	”Multi-process” mode: vLLM connects to the engine on the same node over shared-memory IPC (`hostIPC`). The connector ships KV blocks between vLLM and the engine.
L0 / L1 / L2	Cache tiers. L0 is GPU HBM (vLLM’s own paged KV cache). L1 is CPU offloading: host DRAM, sized by `engine.spec.l1.sizeGB`. L2 is external storage: an optional backend (Redis, S3, filesystem) set via `engine.spec.l2Backend`.
Prefetch	Loading KV blocks from L2 (external storage) back into L1 (host DRAM) on a miss, so the next reuse is served from memory. Tuned by `engine.spec.l2Backend.prefetchPolicy` and `prefetchMaxInFlight`.
KV cache	The key/value attention tensors vLLM computes for the tokens in a prompt. Reusing them across requests instead of recomputing is what saves GPU work and lowers TTFT.
KV cache hit	A later request reused stored KV blocks for a shared prefix. Visible as `prefix hits` in engine logs and lower TTFT.
TTFT	Time to first token — how long a request waits before the model emits its first output token. Cache hits cut it by skipping prefix recomputation.

Kubernetes

Term	Meaning
CRD (CustomResourceDefinition)	Extends the Kubernetes API with a new resource kind. Cluster-scoped. Here: `lmcacheengines.lmcache.lmcache.ai`.
CR (Custom Resource)	An instance of a CRD-defined kind, e.g. an `LMCacheEngine`.
Operator / controller	A program that watches CRs and reconciles them into lower-level objects (Deployments, DaemonSets, …).
DaemonSet	Runs one pod per matching node — the shape used for the engine (one per GPU node).
Deployment	Maintains N identical pods with rolling updates — the shape used for the operator.
Finalizer	A marker on a resource that blocks its deletion until a controller runs cleanup. The `LMCacheEngine`’s `lmcache.ai/cleanup` finalizer can block a namespace delete if the operator is gone.
Namespaced vs cluster-scoped	Namespaced resources are removed when their namespace is deleted; cluster-scoped ones (CRD, ClusterRole, ClusterRoleBinding) survive and must be cleaned up explicitly.
SCC (Security Context Constraint)	OpenShift’s per-pod security policy. Engine pods need the `privileged` SCC because they use `hostIPC`.
ServiceMonitor	A Prometheus Operator CR telling Prometheus what to scrape.

Helm

Term	Meaning
Chart	A versioned, packaged bundle of Kubernetes templates + default values.
Release	A named installation of a chart in a namespace. Helm stores its history (as Secrets) to enable upgrade/rollback.
Values / values file	User configuration that overrides chart defaults, supplied via `-f my-values.yaml` (preferred) or `--set`.
appVersion vs version	`version` is the chart’s version; `appVersion` is the version of the app it deploys. The operator image tag defaults to `appVersion`.
OCI artifact	A chart stored in a container registry (here GHCR) using OCI media types, pulled with `oci://…` instead of a classic HTTP repo.
Hook	A Job/pod Helm runs at a lifecycle point (pre-install, pre-delete, test). The chart’s pre-delete hook removes the engine CR before the operator.

Observability

Term	Meaning
OTLP	OpenTelemetry Protocol — how the engine ships metrics/traces to the Collector (gRPC on `:4317`).
OTel Collector	Receives OTLP from the engine and exports onward (Prometheus, Tempo, external backends). Managed by the OpenTelemetry Operator from a CR the chart creates.
Prometheus exporter (`:8889`)	The Collector port that re-exposes LMCache application metrics (`lmcache_*`) for scraping.
Collector internals (`:8888`)	The Collector’s own metrics (`otelcol_*`), used to confirm data is flowing.
Tempo	A trace storage backend. The chart can deploy a `TempoMonolithic`; you view traces in your own Grafana via a Tempo data source.
`remoteWrite`	Pushing metrics to an external Prometheus-compatible endpoint (Grafana Cloud, Mimir, Thanos) instead of (or alongside) in-cluster scraping.

⌘I