tm infer chat

​

When To Use This

​

Usage

tm infer chat [OPTIONS]

​

Examples

​

Send a non-streaming serverless request with an explicit inference API key.

tm infer chat --api-key YOUR_INFERENCE_API_KEY --model YOUR_SERVERLESS_MODEL_NAME --json '[{"role":"user","content":"Say hello."}]'

​

Stream tokens over SSE and fail a stalled stream after 30 seconds of upstream silence.

echo '[{"role":"user","content":"Stream tokens."}]' | tm infer chat --api-key YOUR_INFERENCE_API_KEY --model YOUR_SERVERLESS_MODEL_NAME --stream --stream-idle-timeout 30

​

Options

​

Inherited Global Options

​

Auth Scope

• inference-api-key

​

Prerequisites

• Provide --model explicitly or include model in the JSON request body.
• If you have Control Plane access for the same Tensormesh environment, discover published serverless model names with tm billing pricing serverless list, then pass the returned pricing[].model value with --model.
• If you only have inference credentials, or you are targeting a different serverless host override, get the model name from your Tensormesh environment before sending the request.

​

Caveats

• Reuses gateway_api_key from config.toml as the inference API key when --api-key is omitted.
• gateway_api_key is the stored inference API key used by the SDK as inference_api_key.
• tm billing pricing serverless list helps discover published serverless model names for the current Tensormesh Control Plane environment. If you are targeting a different serverless host override, confirm the model name for that host separately.
• --stream currently supports only --output text.

​

Related Commands

• tm billing pricing serverless list
• tm models list
• tm doctor

​

Parent Command

• tm infer