Model Routing

Overview

Instead of hardcoding a single model, agents can use routing strategies to dynamically select the best model for each request.

Routing Modes

Static (default)

Uses the configured modelId for every request.

Cost Optimized

Selects the cheapest model from candidates that meets the required capabilities.

Quality First

Selects the highest-quality model from candidates, falling back to cheaper options only if budget constraints apply.

Balanced

Balances cost and quality using a scoring function across candidates.

Custom

Evaluates rules in order. Each rule has a condition expression and a modelId. The first matching rule wins; if none match, uses fallbackModelId.

Configuration

{
  "routingStrategy": {
    "mode": "cost_optimized",
    "candidates": [
      "openai/gpt-4o",
      "openai/gpt-4o-mini",
      "anthropic/claude-sonnet-4-20250514"
    ],
    "maxCostPerRequest": 0.05,
    "requiredCapabilities": ["reasoning"],
    "fallbackModelId": "openai/gpt-4o-mini"
  }
}

Model Capabilities

Capability	Description
coding	Code generation and review
reasoning	Complex logic and analysis
creative	Creative writing and ideation
vision	Image understanding
long_context	Large context windows (100k+)

Custom Rules Example

{
  "routingStrategy": {
    "mode": "custom",
    "rules": [
      {
        "condition": "input.length > 10000",
        "modelId": "anthropic/claude-sonnet-4-20250514"
      },
      {
        "condition": "task === 'code_review'",
        "modelId": "openai/gpt-4o"
      }
    ],
    "fallbackModelId": "openai/gpt-4o-mini"
  }
}