Smart Routing configuration

Smart Routing selects a harness and model from the session's first message. See Smart Routing for the user workflow and behavior.

Built-in routing

Configure a server-level llm: block to use the built-in judge. No routing: block is required. With neither an llm: block nor an external routing: block, Smart Routing is disabled and the Auto picker option is hidden.

llm:
  model: openai/gpt-5.5
  connection:
    api_key: ${OPENAI_API_KEY}

Field	Required	Description
`model`	Yes	Provider-prefixed model id used by the judge, such as `openai/gpt-5.5`.
`connection`	No	Provider-specific connection overrides, such as `api_key` and `base_url` for an OpenAI-compatible endpoint. `${ENV}` references are expanded. Omit it to use the provider's usual environment variables.
`profile`	No	Databricks CLI profile from `~/.databrickscfg`. `connection` takes precedence when both are set.
`fallback_models`	No	Ordered provider-prefixed models to try if the primary model fails. The same connection is shared across the list, so use same-provider fallbacks or environment-based credentials.

The built-in judge caps the entire routing call at 9 seconds, including fallback attempts. The server-level llm.request_timeout setting does not extend this routing-specific limit.

External routing

Add a top-level routing: block to delegate selection to an external service:

routing:
  provider: external
  base_url: https://gateway.example.com/ai-gateway/routing/v1
  router_name: task_v0
  model_prefix: databricks-
  api_key: ${ROUTING_API_KEY}

Field	Required	Description
`provider`	Yes	Must be `external`. Any other value, or an omitted block, falls back to the built-in judge when `llm:` is configured.
`base_url`	Yes	Routing service base URL. The server appends `/routes:select`.
`router_name`	Yes	Routing strategy, such as `task_v0`. Sent as `route_selector.router_name`.
`model_prefix`	No	Prefix or list of prefixes removed from catalog model ids before the request and restored on the response, such as `databricks-`.
`api_key`	No	Static bearer token. `${ENV}` references are expanded. Takes precedence over `profile`.
`profile`	No	Databricks CLI profile. The server resolves a fresh OAuth token for every request.

If base_url or router_name is missing, the provider is skipped and routing stays off rather than preventing the server from starting.

Authentication precedence

The first configured option wins:

api_key: sends Authorization: Bearer <token> after expanding ${ENV}.
profile: uses the Databricks SDK to resolve a fresh token for every call, so long-running servers do not retain an expired token.
Neither: sends requests without authentication.

The request and response contract is documented in the external routing API reference.