Octo Router logoOctoRouter
Routing

Routing

Understand how Octo Router routes requests.

Routing is the core function of Octo Router. It determines which provider and model fulfills a request based on your configured strategy and policies.

Available Strategies

Octo Router supports several routing strategies to optimize for different goals:

  • Weighted: Distribute traffic across providers based on percentage weights.
  • Round Robin: Distribute traffic evenly across all available providers.
  • Cost-Based: Automatically select the cheapest provider that meets the requirements.
  • Latency-Based: Route to the provider with the lowest response time.
  • Semantic: Route based on the intent or "meaning" of the user's prompt.

Models

Learn how Octo Router uses local embeddings to route requests based on intent.

Weighted

Distribute traffic across providers using percentage-based weights.

On this page

Available Strategies