Skip to main content
Peer-to-peer (P2P) KV transfer lets engine pods read cached KV data directly from each other’s memory over a transfer channel. When a pod receives a request whose prefix was already computed by another pod, it can fetch the KV tensors from that peer instead of recomputing them. This supplements the L1 (local memory) and L2 (Redis, filesystem, etc.) tiers with a third path: any peer in the fleet can serve as a warm cache source. P2P is most useful when:
  • your fleet serves many models or tenants and prompt reuse spans nodes
  • you want warm-hit latency close to L1 without provisioning a large shared L2
  • your cluster has high-bandwidth interconnect (InfiniBand, RoCE, or fast TCP)
For background on P2P in LMCache, see the upstream P2P documentation.

​
Prerequisites

P2P requires the following minimum versions:
ComponentMinimum version
Tensormesh Operator helm chart0.5.0
LMCache Operator (lmcache/lmcache-operator)v0.5.0
LMCache vLLM (lmcache/vllm-openai)v0.5.0

​
Configuration

Enable P2P by setting engine.p2p.enabled: true in your values file. The chart handles the rest — no manual CLI flags needed. 
engine:
  p2p:
    enabled: true
    port: 48500                  # transfer channel port peers connect to
    l1AlignBytesOverwrite: 65536 # 64 KiB alignment, recommended for P2P
P2P also requires that every engine pod can reach every other engine pod on the transfer channel port. How you achieve that depends on your cluster’s network fabric — see the networking options below.

​
Networking options

​
Option 1: Host networking (easiest to get started)

The simplest approach is hostNetwork: true. The engine pod binds directly to the host’s network namespace, so peers can reach each other without any CNI overlay or NAT. Because the pod shares the host’s port space, move the default server ports to high values to avoid conflicts with other services on the node:
my-values.yaml
engine:
  enabled: true
  name: tensormesh-engine
  p2p:
    enabled: true
    port: 48500                  # P2P transfer channel
    l1AlignBytesOverwrite: 65536 # 64 KiB, recommended for P2P
  spec:
    hostNetwork: true
    server:
      port: 45555                # gRPC (default 5555)
      httpPort: 48080            # health/admin (default 8080)
    l1:
      sizeGB: 60
    image:
      repository: lmcache/vllm-openai
      tag: v0.5.0
Apply it with: 
helm upgrade --install tensormesh-operator \
  oci://ghcr.io/tensormesh-production/charts/tensormesh-operator \
  -n tensormesh-operator --create-namespace \
  -f my-values.yaml --wait
When hostNetwork is true, the operator automatically sets dnsPolicy: ClusterFirstWithHostNet so cluster DNS still works inside the engine pod.

​
Option 2: HCA passthrough (coming soon)

Coming soon — recommended for InfiniBand environments.
HCA (Host Channel Adapter) passthrough exposes the InfiniBand device directly to the engine pod. This gives the transfer channel native RDMA verbs without any overlay or kernel bypass emulation. The operator will support scheduling HCA resources via the k8s-rdma-shared-dev-plugin so that each engine pod gets a dedicated HCA device.

​
Option 3: SR-IOV (coming soon)

Coming soon — recommended for RoCE environments.
SR-IOV (Single Root I/O Virtualization) creates lightweight virtual functions (VFs) from a physical NIC, each assigned to a pod. This gives near-hardware RDMA performance while allowing multiple pods to share the same physical NIC. The operator will integrate with the SR-IOV Network Operator to request VFs via a NetworkAttachmentDefinition.

​
Option 4: Macvlan (coming soon)

Coming soon — recommended for RoCE environments.
Macvlan creates a virtual interface directly on the physical NIC’s L2 segment. Each pod gets its own MAC address and IP on the fabric network, bypassing the CNI overlay. This is simpler to set up than SR-IOV (no VF provisioning) but still gives pods direct L2 reachability on the RDMA fabric. The operator will integrate with Multus CNI to attach a secondary macvlan interface to engine pods.

​
Option 5: TCP (non-RDMA environments)

If your cluster does not have InfiniBand or RoCE hardware, P2P still works over TCP. Performance is lower than RDMA but still faster than recomputing prefills from scratch. Use the same host-networking configuration from Option 1 and set UCX_TLS to TCP-only transports: 
engine:
  spec:
    env:
      - name: UCX_TLS
        value: "tcp,sm,self"
No special NIC passthrough or CNI plugin is needed. This is the right choice for:
  • development and testing clusters
  • cloud environments without RDMA-capable instances
  • clusters where the CNI overlay provides sufficient bandwidth

​
Helm values reference

ValueTypeDefaultDescription
engine.p2p.enabledboolfalseEnable P2P KV transfer between engine pods.
engine.p2p.portint8500Port the transfer channel advertises to peers.
engine.p2p.l1AlignBytesOverwriteint65536L1 buffer alignment in bytes. 64 KiB recommended for P2P.
engine.spec.hostNetworkboolfalseRun the pod in the host’s network namespace.
engine.spec.server.portint5555gRPC server port. Move to a high port when using hostNetwork.
engine.spec.server.httpPortint8080HTTP health/admin port. Move to a high port when using hostNetwork.

​
Verification

After deploying with P2P enabled, verify that the engine pods started correctly and the P2P transfer channel is active.

​
Check the engine pod is running

kubectl get pods -n tensormesh-operator -l app.kubernetes.io/component=cache-engine

​
Query the engine’s L2 backends

Curl the engine’s HTTP endpoint to list the active L2 backends. When P2P is enabled, a P2P backend should appear alongside any configured L2 (Redis, filesystem, etc.): 
ENGINE_POD=$(kubectl get pods -n tensormesh-operator \
  -l app.kubernetes.io/component=cache-engine -o name | head -1)

# Port-forward the HTTP port (use your configured httpPort)
kubectl port-forward -n tensormesh-operator "$ENGINE_POD" 48080:48080 &

curl -s http://localhost:48080/l2/backends | jq .
The L2 backends endpoint and its response format may change in future releases. This section will be updated as the API stabilizes.

​
Common mistakes

  • Forgetting to move server.port and server.httpPort to high ports when using hostNetwork, causing bind failures on ports already in use by the host
  • Not setting engine.p2p.enabled: true — hostNetwork alone does not turn on P2P
  • Using the default P2P port (8500) on hostNetwork when another service already binds to it
  • Setting UCX_TLS to an RDMA transport (e.g. rc_verbs) without the corresponding device passthrough — UCX will fail to open the transport and the engine will not start