> For the complete documentation index, see [llms.txt](https://docs.bito.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.bito.ai/governor/auto-ai-model-routing.md). # Auto AI model routing Send simple requests to cost-efficient models and complex work to capable ones, using an Auto-router. Auto-routing sends each request to the model that fits the work. Simple requests go to a cost-efficient, faster model, and complex requests go to a more capable one. Without it, every request reaches the same model. Renaming a variable costs the same as planning a migration across four services, because both go to whichever model the coding tool asked for. [Bito Governor](/governor/overview.md) sorts each request into one of four tiers, Simple, Medium, Complex, and Reasoning, then sends it to the model you assigned to that tier. You write the description that defines each tier, so requests are sorted by your own words rather than by a fixed rule. Your engineers change nothing. Their tools keep asking for the same model name, and Governor answers from whichever model it selected. You turn auto-routing on one model name at a time. That name is the alias your coding tools ask for, such as `claude-opus-5` or `claude-*`. Each alias appears as a group on the **Routes** page, with a **Direct** and an **Auto** control. **Direct** answers from the targets you configured. **Auto** hands the alias to the router. ## Before you start Set prices on the **Prices** page for every model a tier will point at. Routing direction, the price ceiling, and the shadow cost comparison all read from those prices, and a model with no price cannot be selected. Add a route for the model you want the classifier to run on. The **classifier model** field takes a route alias in this workspace rather than a raw vendor model ID.

## Open the router In the Bito Governor, open the **Routes** tab. Either select **auto-route this alias** in the **Add route** form, which adds the route and then opens the router settings, or click **Auto** on an existing group in the routes table.

The router step is optional. Nothing about auto-routing is saved until you click **Save**, and the route itself stays either way. **Skip for now** closes the router and keeps the route. ## Acknowledge the response header The form opens on a notice you have to accept before it will save. Clients asking for the alias will receive answers from other models. The `x-bito-routed-model` response header always names the model that answered. Do not read the `model` field inside the response instead, because some providers return the model that was asked for and others return the model that answered. Anything you run that compares the two will see a different name. Tick **I understand.** to enable **Save**.

## Settings

Setting	Description
fallback	Read only. Names the route that answers while the router is Off or in Shadow, and whenever a tier cannot be used.
mode	Off disables the router. Shadow is a dry run that writes down what it would have done while requests still go to the alias's own route, exactly as they do on Off. On routes traffic. A new router starts in Shadow.
routing direction	Down only never sends a request to a model that costs more than the one your agent asked for. It needs a price for both models, and leaves the request alone and records why when either is missing. Both ways also allows an upgrade, for example sending a planning request from a small model to a larger one. Down only is the default.
price ceiling	A cap in dollars per million output tokens. Governor never sends a request to a model above it, whatever the classifier decides. Leave it blank for no ceiling.
rubric preset	Fills in the four tier descriptions below, and retunes the built-in scorer under Advanced to match that kind of traffic. Pick the one closest to your traffic and edit from there. Once you change the wording, the field reads User-defined.
classifier	LLM is the default and the most accurate. Heuristic uses the built-in scorer, which is free and instant, and on coding-tool traffic calls almost everything simple, so it is kept as the backup for when the LLM is too slow to answer. Keywords only matches on raw keywords. See How requests are classified.
classifier model	The route alias the classifier runs on. Pick a cheap, fast model, because it runs on every request that no rule has already decided.
timeout (ms)	How long the classifier may take before Governor falls back to the built-in scorer. Default `2000`.
context turns	How many recent user prompts the classifier reads, rather than the current one alone. Default `3`. Reading the last few turns catches a request that continues earlier work, so a short follow-up inside a planning session is still classified as planning.

#### Tier mapping The four tier descriptions are the rule. They are the text the classifier reads to decide which tier a request belongs in, so write them the way you would explain the distinction to a colleague. A preset seeds them, and you can edit any of them. Each tier takes: | Column | What to enter | | :----------------------------: | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | When should this tier be used? | The description the classifier reads. | | Routes to | A route alias in this workspace, named exactly, and the model. Listing a route here does not make it reachable by your apps, and it keeps that route's own account, load balancing, failover, and cooldown. | | Thinking | The reasoning effort for that tier. **as asked** passes through whatever the caller requested. | **+ model** gives a tier a second choice. Use it only for models you would be equally happy to receive, because the tier is the whole of what the classifier decided. **+ new target** adds a provider, account, and model without leaving the router. **Suggest from my routes** proposes tier assignments from the routes you already have. Governor seeds the preset descriptions from its own model research, which scored around 30 models across roughly 2,000 engineering tasks. The four tiers sort by risk and clarity rather than by how many files a request touches: | Tier | What reaches it | | :-------: | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Simple | A stated change with a clear target: build the described feature, fix the described bug, refactor, rename, add a test. Work spanning many files still counts as simple when the request says what done looks like. | | Medium | Explaining and checking: how a piece of code works, why a job failed, whether a plan is missing anything, and requests from non-engineers that name what they want. | | Complex | Judgment and risk: writing an implementation plan, a design, or a migration plan, choosing between approaches, reviewing a diff, deciding whether something is safe to ship, and anything touching production, data, secrets, protected branches, or destructive commands. | | Reasoning | Requests that do not say what done looks like: a symptom with no named cause, a request that could mean several different jobs, and open architecture questions. Working out what is actually being asked is the first task, so these reach the most capable model. |

#### How requests are classified The LLM classifier reads the request and returns a tier. When it cannot answer inside the timeout, the built-in scorer decides instead, so every router has a scorer behind it whether you tune one or not. The scorer turns each request into a score from signals such as context size, code markers, and reasoning markers, then places it in a tier using three boundaries. Defaults are shown in grey on the form, and leaving a box empty keeps the default.

Advanced setting	Default
simple / medium boundary	0.15
medium / complex boundary	0.35
complex / reasoning boundary	0.6
weight: context size	0.35
weight: code markers	0.15
weight: reasoning markers	0.15
weight: technical vocabulary	0.12
weight: tool count	0.05
weight: declared effort	0.05
weight: conversation depth	0.03
weight: multi-step phrasing	0.02
context ramp starts at (tokens)	20000
context ramp saturates at (tokens)	150000
tool count saturates at	8
turn depth saturates at	8

The three boundaries have to ascend. If the simple to medium line sat above the medium to complex line, nothing could ever be medium, and Governor refuses to save a set that does not ascend. Most workspaces never open **Advanced**. Picking the rubric preset closest to your traffic retunes the scorer for you.

## Backing groups Select **backing group (internal)** on the routes holding the models a router picks between. A backing group is hidden from your apps, so it cannot be requested by name and does not appear in the model list, and only the router can send traffic to it. This stops anyone reaching an internal model directly and bypassing the router.

## When the router does not route The alias's own route answers whenever the router is off or in shadow, and whenever a tier cannot be used. A tier is unusable when its model has no price, has no deployment on the account, or would break your routing direction, and the tier shows **will not route** in that case. The routes table also carries a **when not routing** row naming the route that answers. If the router selects a model that turns out to be unavailable, Governor falls back to the model your tool originally requested, so the request is still answered. Governor also protects prompt caching. Moving an ongoing conversation to a different model discards its cache and makes the next turn slower and more expensive, so Governor keeps that conversation where it is when staying put costs less. One-off requests carry no cache and are routed freely. ## Test before you turn it on | Action | What it does | | --------------------- | --------------------------------------------------------------------------------------------------------------------- | | Classify a prompt | Enter a sample prompt and see which tier it lands in and which model would answer. Nothing is sent. | | Test, on a tier row | Sends a real request to that tier's model to confirm it works on that account. | | Review shadow results | Reports what the router would have done across recorded traffic, with an estimated cost comparison. Export it as CSV. | Leave a new router in **Shadow** across a day of real traffic, then review the results before switching to **On**. Shadow results state plainly when traffic was not moved, so a report never reads as a saving that did not happen. The routes table reports what the router actually did, splitting recent traffic into requests routed, requests that needed a fallback, requests that failed, and requests deliberately not routed with the reason. {% hint style="info" %} A request routed to a model on a different API is marked **converted** in the routes table. Your messages, tools, images, and reasoning effort carry over where the target model supports them. Earlier thinking, attached documents, images returned by tools, and tools the provider runs itself, such as web search, do not carry over. {% endhint %} {% hint style="info" %} Router changes reach live traffic within 30 seconds. {% endhint %} --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.bito.ai/governor/auto-ai-model-routing.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.