Routing preferences

Each model is served through one or more routes with automatic failover. A request can narrow how a route is chosen — prefer cost or latency, forbid fallbacks, require zero data retention or a region — never widen it.

The tf.routing object

json

{
  "model": "zai-org/GLM-5.3",
  "messages": [{ "role": "user", "content": "…" }],
  "tf": { "routing": { "prefer": "latency", "allow_fallbacks": false, "require_zdr": false, "region": "us-east-1" } }
}

prefer: cost picks the cheapest healthy route, latency the fastest by first-token p50. allow_fallbacks: false stops the gateway from trying another route before the first byte. require_zdr and region tighten the constraints; a request no route can satisfy returns 400 naming the parameter.

Enterprise contracts can name dedicated routes. Those names are given to you with the contract and can be listed in providers, in order of preference; any other name returns 400.

What was applied

http

x-tf-routing: prefer=latency;fallbacks=off
x-tf-model-version: 2026-04-24
x-tf-fallback: true     # present only when a fallback was used

The tf block in the response body carries the same information and the number of attempts. The request log shows every attempt with its outcome and latency.

Protocol conversion

You can call any model with the OpenAI, Anthropic or Gemini request format; the gateway converts it for the route that serves the model and lists what it had to change in the billing detail of the request and in the x-tf-conversion-warnings header.

Gemini-format requests go to /v1beta/models/{model}:generateContent (or streamGenerateContent, countTokens, embedContent) with the catalog model name in place of {model}. Responses & Messages

The project tool_loss_policy defaults to allow. safe rejects dropped tools or tool choices; strict rejects any lossy or dropped field. A key policy can override the project. When fallbacks are allowed the gateway first tries a route that can take the request unchanged; otherwise it returns 400 with the offending param.

Session affinity

Repeated user IDs, a selected x-tf-tags value, or the first system-message prefix prefer the same healthy route, which keeps prompt caches warm. Bindings expire after ten idle minutes and are isolated by organization, project and model. Health, region and privacy constraints still apply, and an unavailable route falls back as usual.

A prompt cache belongs to a route: after a failover the first request on the new route starts with a cold cache.