CLI

tmo-cli is the Tensormesh administrator CLI: a kubectl-familiar tool for platform/SRE engineers to observe and operate LMCacheEngine resources (the KV-cache controllers) and their per-GPU-node server pods. It unifies two surfaces — the Kubernetes CRD (lmcacheengines.lmcache.lmcache.ai) for engine lifecycle and status, and each server pod’s HTTP control API (reached through the Kubernetes API-server proxy) for cache state, quotas, and diagnostics.

tmo-cli is the admin/SRE CLI talking directly to your Kubernetes cluster (the data plane).

Install

tmo-cli is distributed privately as part of the gated Tensormesh product — not through public Homebrew or krew. It installs from the Tensormesh container registry (GHCR) using the same access token as the operator chart and images: a token scoped to artifact pull only, which grants no access to Tensormesh source code. If you don’t have one, request it from the Tensormesh team — it’s the same token used in the operator install guide.

One-line install (recommended)

curl -sSfL https://operator.tensormesh.ai/install-cli.sh | TMO_TOKEN=<TOKEN_FROM_TENSORMESH> sh

The script detects your OS and architecture, pulls the matching binary from GHCR, verifies its SHA-256 against the registry digest, and installs it to /usr/local/bin (falling back to ~/.local/bin if that isn’t writable — no sudo). To pin a version or change the target, pass flags after -s --:

curl -sSfL https://operator.tensormesh.ai/install-cli.sh | sh -s -- \
  --token <TOKEN_FROM_TENSORMESH> --version <version> --install-dir ~/bin

Manual install (air-gapped / CI)

Prefer to pull the artifact yourself? Authenticate to GHCR with your token and use oras:

echo '<TOKEN_FROM_TENSORMESH>' | oras login ghcr.io -u tensormesh --password-stdin

oras pull ghcr.io/tensormesh-production/cli/tmo-cli:<version>   # or :latest
tar -xzf tmo-cli_*_linux_amd64.tar.gz tmo-cli                   # match your OS/arch
install -m 0755 tmo-cli ~/.local/bin/tmo-cli

Verify

tmo-cli version
tmo-cli diag preflight       # confirm your kube RBAC is sufficient

How it connects

tmo-cli uses your existing kubeconfig — if kubectl works, tmo-cli works. Resolution precedence: --kubeconfig → $KUBECONFIG → the tmo-cli config default → ~/.kube/config.

tmo-cli config use-context prod-us-east
tmo-cli config set-default --output table
tmo-cli config view

HTTP commands (node, cache, diag, quota) reach engine pods through the Kubernetes API-server proxy by default — no extra network path, and RBAC on pods/proxy gates them. Pass --direct to reach pod host IPs directly when running inside the cluster.

Global flags

Flag	Meaning
`-n, --namespace`	namespace (default `default`)
`-A, --all-namespaces`	across all namespaces (list / HTTP commands)
`-e, --engine`	restrict HTTP commands to one engine
`-o, --output table\|json\|yaml`	output format; JSON is stable and versioned (`tmoOutputVersion: v1`)
`--context`, `--kubeconfig`	kube target overrides
`--no-color`	disable ANSI color (honors `NO_COLOR`; auto-off when not a TTY)
`--direct`, `--workers`, `--pod-timeout`	HTTP fan-out controls

Exit codes

Stable and scriptable:

Code	Meaning
`0`	success
`1`	generic / unexpected error
`2`	usage error (e.g. a removed `engine` write-verb → use Helm)
`3`	not found (engine, server, salt)
`4`	validation / guardrail rejection
`5`	timed out (e.g. `engine wait`)
`6`	confirmation declined / `--yes` required

Common tasks

See the fleet

tmo-cli engine list -A                # all engines, all namespaces
tmo-cli engine list -A -w             # watch for changes (table only)
tmo-cli engine describe my-engine     # spec + status + conditions + endpoints + events

Deploying or changing engines (owned by Helm)

Engine fleet membership and spec are owned by the Helm release — tmo-cli does not create, apply, edit, or delete engines. To add, change, or remove an engine, edit the chart values and helm upgrade.

The CLI’s role in a rollout is the readiness gate after the upgrade:

helm upgrade tensormesh-operator oci://ghcr.io/tensormesh-production/charts/tensormesh-operator -f values.yaml
tmo-cli engine wait my-cache --for=AllInstancesReady --timeout=5m   # exit 5 on timeout
tmo-cli engine describe my-cache

Inspect servers and cache state

tmo-cli node list -e my-cache                       # per-node servers + live health
tmo-cli cache status -e my-cache                    # aggregated /status + per-node + TOTAL
tmo-cli cache status -e my-cache --node gpu-node-07 # raw /status for one server
tmo-cli engine logs my-cache --node gpu-node-07 -f  # stream logs (--all to multiplex)

Operate (destructive — gated)

tmo-cli cache clear -e my-cache --node gpu-node-07  # one server; prompts
tmo-cli cache clear -e my-cache --all --yes         # whole fleet, no prompt

tmo-cli quota list -e my-cache                      # per-cache_salt budgets
tmo-cli quota set acme-prod --limit 48 -e my-cache
tmo-cli quota rm bench-harness -e my-cache --yes    # data evicts next cycle

Destructive runtime operations (cache clear, quota rm) print a blast-radius preamble and require confirmation; pass -y/--yes to bypass. In a non-TTY without --yes they refuse (exit 6) rather than guess.

Diagnose and file a ticket

tmo-cli diag health -e my-cache       # healthcheck + periodic-thread sweep
tmo-cli diag periodic -e my-cache     # per-server periodic-thread detail
tmo-cli diag threads --node gpu-node-07 -e my-cache
tmo-cli diag loglevel set lmcache DEBUG --node gpu-node-07 -e my-cache
tmo-cli diag bundle -e my-cache       # redacted support tarball (.tgz)

diag bundle collects the CR, recent events, and each server’s /conf, /status, /threads, and /periodic-threads-health into a gzip tarball. Secrets are redacted by default; /env is included only with --include-env (and is still redacted). Attach the tarball to a support ticket.

Output and scripting

Tables are for humans; -o json (and -o yaml) is the automation contract, wrapped in an envelope with tmoOutputVersion: v1:

tmo-cli node list -e my-cache -o json | jq '.items[] | select(.healthy==false)'
tmo-cli engine get my-cache -o json | jq '.item.status.readyInstances'

Color is off in pipes and non-terminals automatically, and can be forced off with --no-color or NO_COLOR=1.

Troubleshooting

Symptom	Cause / fix
`... forbidden ... ask for the 'tensormesh-admin' role`	your RBAC lacks a needed verb; run `tmo-cli diag preflight` to see which
`node list` / `cache status` empty	the engine has no ready endpoints — is a GPU node up and the engine `Running`?
`unreachable` in a health table	that server’s HTTP API didn’t answer; check `tmo-cli engine describe` + pod status
`engine wait` exits `5`	the condition didn’t become true within `--timeout`
HTTP commands hang on a big fleet	tune `--workers` / `--pod-timeout`; partial results still render

tmo-cli does not phone home — there is no telemetry in the default build.

Installation

Feature List

Integrations

Management

Operator UI

Reference

Install

One-line install (recommended)

Manual install (air-gapped / CI)

Verify

How it connects

Global flags

Exit codes

Common tasks

See the fleet

Deploying or changing engines (owned by Helm)

Inspect servers and cache state

Operate (destructive — gated)

Diagnose and file a ticket

Output and scripting

Troubleshooting

​Install

​One-line install (recommended)

​Manual install (air-gapped / CI)

​Verify

​How it connects

​Global flags

​Exit codes

​Common tasks

​See the fleet

​Deploying or changing engines (owned by Helm)

​Inspect servers and cache state

​Operate (destructive — gated)

​Diagnose and file a ticket

​Output and scripting

​Troubleshooting

Install

One-line install (recommended)

Manual install (air-gapped / CI)

Verify

How it connects

Global flags

Exit codes

Common tasks

See the fleet

Deploying or changing engines (owned by Helm)

Inspect servers and cache state

Operate (destructive — gated)

Diagnose and file a ticket

Output and scripting

Troubleshooting