Auto AI model routing
Send simple requests to cost-efficient models and complex work to capable ones, using an Auto-router.
Auto-routing sends each request to the model that fits the work. Simple requests go to a cost-efficient, faster model, and complex requests go to a more capable one.
Without it, every request reaches the same model. Renaming a variable costs the same as planning a migration across four services, because both go to whichever model the coding tool asked for.
Bito Governor sorts each request into one of four tiers, Simple, Medium, Complex, and Reasoning, then sends it to the model you assigned to that tier. You write the description that defines each tier, so requests are sorted by your own words rather than by a fixed rule.
Your engineers change nothing. Their tools keep asking for the same model name, and Governor answers from whichever model it selected.
You turn auto-routing on one model name at a time. That name is the alias your coding tools ask for, such as claude-opus-5 or claude-*. Each alias appears as a group on the Routes page, with a Direct and an Auto control. Direct answers from the targets you configured. Auto hands the alias to the router.
Before you start
Set prices on the Prices page for every model a tier will point at. Routing direction, the price ceiling, and the shadow cost comparison all read from those prices, and a model with no price cannot be selected.
Add a route for the model you want the classifier to run on. The classifier model field takes a route alias in this workspace rather than a raw vendor model ID.
Open the router
In the Bito Governor, open the Routes tab.
Either select auto-route this alias in the Add route form, which adds the route and then opens the router settings, or click Auto on an existing group in the routes table.
The router step is optional. Nothing about auto-routing is saved until you click Save, and the route itself stays either way. Skip for now closes the router and keeps the route.
Acknowledge the response header
The form opens on a notice you have to accept before it will save.
Clients asking for the alias will receive answers from other models. The x-bito-routed-model response header always names the model that answered. Do not read the model field inside the response instead, because some providers return the model that was asked for and others return the model that answered. Anything you run that compares the two will see a different name.
Tick I understand. to enable Save.
Settings
fallback
Read only. Names the route that answers while the router is Off or in Shadow, and whenever a tier cannot be used.
mode
Off disables the router. Shadow is a dry run that writes down what it would have done while requests still go to the alias's own route, exactly as they do on Off. On routes traffic. A new router starts in Shadow.
routing direction
Down only never sends a request to a model that costs more than the one your agent asked for. It needs a price for both models, and leaves the request alone and records why when either is missing. Both ways also allows an upgrade, for example sending a planning request from a small model to a larger one. Down only is the default.
price ceiling
A cap in dollars per million output tokens. Governor never sends a request to a model above it, whatever the classifier decides. Leave it blank for no ceiling.
rubric preset
Fills in the four tier descriptions below, and retunes the built-in scorer under Advanced to match that kind of traffic. Pick the one closest to your traffic and edit from there. Once you change the wording, the field reads User-defined.
classifier
LLM is the default and the most accurate.
Heuristic uses the built-in scorer, which is free and instant, and on coding-tool traffic calls almost everything simple, so it is kept as the backup for when the LLM is too slow to answer.
Keywords only matches on raw keywords.
classifier model
The route alias the classifier runs on. Pick a cheap, fast model, because it runs on every request that no rule has already decided.
timeout (ms)
How long the classifier may take before Governor falls back to the built-in scorer. Default 2000.
context turns
How many recent user prompts the classifier reads, rather than the current one alone. Default 3. Reading the last few turns catches a request that continues earlier work, so a short follow-up inside a planning session is still classified as planning.
Tier mapping
The four tier descriptions are the rule. They are the text the classifier reads to decide which tier a request belongs in, so write them the way you would explain the distinction to a colleague. A preset seeds them, and you can edit any of them.
Each tier takes:
When should this tier be used?
The description the classifier reads.
Routes to
A route alias in this workspace, named exactly, and the model. Listing a route here does not make it reachable by your apps, and it keeps that route's own account, load balancing, failover, and cooldown.
Thinking
The reasoning effort for that tier. as asked passes through whatever the caller requested.
+ model gives a tier a second choice. Use it only for models you would be equally happy to receive, because the tier is the whole of what the classifier decided.
+ new target adds a provider, account, and model without leaving the router. Suggest from my routes proposes tier assignments from the routes you already have.
Governor seeds the preset descriptions from its own model research, which scored around 30 models across roughly 2,000 engineering tasks. The four tiers sort by risk and clarity rather than by how many files a request touches:
Simple
A stated change with a clear target: build the described feature, fix the described bug, refactor, rename, add a test. Work spanning many files still counts as simple when the request says what done looks like.
Medium
Explaining and checking: how a piece of code works, why a job failed, whether a plan is missing anything, and requests from non-engineers that name what they want.
Complex
Judgment and risk: writing an implementation plan, a design, or a migration plan, choosing between approaches, reviewing a diff, deciding whether something is safe to ship, and anything touching production, data, secrets, protected branches, or destructive commands.
Reasoning
Requests that do not say what done looks like: a symptom with no named cause, a request that could mean several different jobs, and open architecture questions. Working out what is actually being asked is the first task, so these reach the most capable model.
How requests are classified
The LLM classifier reads the request and returns a tier. When it cannot answer inside the timeout, the built-in scorer decides instead, so every router has a scorer behind it whether you tune one or not.
The scorer turns each request into a score from signals such as context size, code markers, and reasoning markers, then places it in a tier using three boundaries. Defaults are shown in grey on the form, and leaving a box empty keeps the default.
simple / medium boundary
0.15
medium / complex boundary
0.35
complex / reasoning boundary
0.6
weight: context size
0.35
weight: code markers
0.15
weight: reasoning markers
0.15
weight: technical vocabulary
0.12
weight: tool count
0.05
weight: declared effort
0.05
weight: conversation depth
0.03
weight: multi-step phrasing
0.02
context ramp starts at (tokens)
20000
context ramp saturates at (tokens)
150000
tool count saturates at
8
turn depth saturates at
8
The three boundaries have to ascend. If the simple to medium line sat above the medium to complex line, nothing could ever be medium, and Governor refuses to save a set that does not ascend.
Most workspaces never open Advanced. Picking the rubric preset closest to your traffic retunes the scorer for you.
Backing groups
Select backing group (internal) on the routes holding the models a router picks between. A backing group is hidden from your apps, so it cannot be requested by name and does not appear in the model list, and only the router can send traffic to it. This stops anyone reaching an internal model directly and bypassing the router.
When the router does not route
The alias's own route answers whenever the router is off or in shadow, and whenever a tier cannot be used. A tier is unusable when its model has no price, has no deployment on the account, or would break your routing direction, and the tier shows will not route in that case. The routes table also carries a when not routing row naming the route that answers.
If the router selects a model that turns out to be unavailable, Governor falls back to the model your tool originally requested, so the request is still answered.
Governor also protects prompt caching. Moving an ongoing conversation to a different model discards its cache and makes the next turn slower and more expensive, so Governor keeps that conversation where it is when staying put costs less. One-off requests carry no cache and are routed freely.
Test before you turn it on
Classify a prompt
Enter a sample prompt and see which tier it lands in and which model would answer. Nothing is sent.
Test, on a tier row
Sends a real request to that tier's model to confirm it works on that account.
Review shadow results
Reports what the router would have done across recorded traffic, with an estimated cost comparison. Export it as CSV.
Leave a new router in Shadow across a day of real traffic, then review the results before switching to On. Shadow results state plainly when traffic was not moved, so a report never reads as a saving that did not happen.
The routes table reports what the router actually did, splitting recent traffic into requests routed, requests that needed a fallback, requests that failed, and requests deliberately not routed with the reason.
Last updated