Set up Bito Governor (self-hosted with Kubernetes)
Run Bito Governor in your own Kubernetes cluster. Install it with the Bito Gateway Helm chart, expose it through your ingress controller, and point your coding tools at it.
Governor runs either on Bito's infrastructure or inside your own network. To have Bito run it for you, see Set up Bito Governor (Bito-hosted). To run it on a single host with Docker rather than Kubernetes, see Set up Bito Governor (self-hosted with Docker).
With Kubernetes, you install Governor from the Bito Gateway Helm chart. The chart defaults bring up the gateway, a MySQL database, and a valkey counter store, all on persistent volumes, so nothing outside your cluster is required.
Setup has four parts:
Create a Secret. Four credentials that the gateway reads at startup, including the key that encrypts your provider credentials.
Install the chart. One Helm command renders and applies every Kubernetes resource.
Expose the gateway. The install gives you a running gateway rather than a reachable one.
Configure your workspace. Add a provider account, a route, a gateway key, and any features you want, then connect your coding tools.
The steps below follow that order.
Prerequisites
Kubernetes cluster
Version 1.23 or later, running in your own environment
kubectl
Installed, and pointed at the cluster you intend to install into.
Helm
Version 3.14 or later. Helm 4 is also supported.
Cluster context
Run kubectl config current-context and confirm it names the target cluster. Every command below applies to whichever cluster your context selects.
Outbound access
Your cluster needs to reach two destinations:
registry-1.docker.io, to pull the Helm chart. Needed when you install and when you upgrade.The endpoint of every LLM provider you configure. Needed whenever Governor serves traffic.
Docker is not required on your own machine. Kubernetes runs the container images for you.
Step 1: Create the namespace and Secret
The chart reads its credentials from a Kubernetes Secret that must already exist in the release namespace. The chart does not create it, so this comes first.
Run the command as written. The openssl calls generate each value for you, so nothing in it needs replacing.
DB_PASSWORD
The password the gateway uses to reach its database.
CRYPTO_ENV_KEK_KEY
The key encryption key. It encrypts every provider credential Governor stores, and must be base64 of exactly 32 bytes.
MYSQL_ROOT_PASSWORD
The root password for the in-cluster MySQL database.
ADMIN_TOKEN
The bootstrap admin credential, used to sign in to the admin UI the first time.
Step 2: Install the chart
gw is the release name. Choose any name you like. It prefixes every resource the chart creates, so a release named gw produces a deployment named gw-bito-gateway.
The command installs the newest published chart version. To pin a version for reproducible deployments, add --version X.Y.Z. Available versions are listed under the Tags tab. To see what a version contains before installing it, run helm show chart oci://registry-1.docker.io/bitoai/bito-gateway-helm.
The install renders three Deployments, three Services, two PersistentVolumeClaims, and a provisioning Job.
Choose a values file
The chart ships example values files, each a complete starting point you copy and edit. Installing with no values file gives you the same result as values-selfhosted.yaml.
values-selfhosted.yaml
Start here. The defaults written out and explained, with the gateway, MySQL, and valkey all in-cluster on persistent volumes.
values-external-db.yaml
You have a managed database such as RDS, Cloud SQL, or Azure Database, with TLS and credentials from a Secret you already own.
values-production-ha.yaml
You want multiple replicas with a shared counter store.
values-single-node.yaml
Small or non-production, with your database already in the cluster.
values-demo.yaml
Evaluating — in-cluster stores with no persistent volume; every restart loses all data
Unpack the chart to get them:
Pass your file with -f:
Put your settings in a file of your own rather than editing values.yaml, which ships inside the chart so that you can read what each setting does.
Step 3: Verify the deployment
Every pod reaches Running, apart from the provisioning Job, which reaches Completed.
Output:
Step 4: Expose the gateway
The install gives you a running gateway rather than a reachable one. Port forwarding is enough for a first look, and an Ingress is what your team connects to.
Port forward, for a first look
The gateway is then available at http://localhost:8788, and the admin UI at http://localhost:8788/admin/. This reaches only your own machine, and only while the command runs.
Ingress, for your team
values-selfhosted.yaml carries four ready-to-uncomment Ingress presets, for nginx, AWS ALB, GKE, and Azure AGIC. Uncomment the one matching your ingress controller, set your host, and apply it with helm upgrade.
Point a DNS record at the load balancer address your controller provisions, and your team reaches Governor at that hostname.
Step 5: Sign in to the admin UI
Read your admin token back from the Secret. Kubernetes stores Secret values base64 encoded, so the
base64 -dstep decodes it.
Open the admin UI at your Ingress hostname, or at
http://localhost:8788/admin/while port forwarding, and sign in with that token.Create a workspace and open it — your tenant (like a team/project).
The workspace opens on the Dashboard. The left sidebar contains the following pages: Dashboard, Documentation, Reports, Keys, Members, Accounts, Routes, Features, Limits, Prices, Test, and Audit.
Most configuration pages contain a form at the top and a table of existing entries below it. Complete Steps 6 to 11 in the admin UI.
Configure from the command line instead
gwctl command ships inside the gateway image and reads the database settings the pod already has. Substitute the workspace ID that the first command prints.
Use a database-backed admin instead of ADMIN_TOKEN
ADMIN_TOKEN exists as a bootstrap credential for the case where the database has no rows yet. To avoid storing an admin credential in a Kubernetes Secret, leave it out of the Secret and mint an admin against the database instead.
The command prints the token once. Governor stores only a SHA-256 hash of it, so the raw value never reaches disk or cluster state.
Credential at rest
In your Secret
A hash in the database
Takes effect
After a pod restart
Immediately
Revoking
Edit the Secret and restart
gwctl admin revoke <id>
Audit trail
One shared identity
One row per admin
Scope
Always global
Global, or scoped to one workspace
Leaving ADMIN_TOKEN unset is a supported configuration. The gateway boots normally and serves traffic, and only the bootstrap admin is disabled.
Step 6: Add a provider account
A provider account stores one LLM API key. Add an account for every LLM provider you use, such as Anthropic, OpenAI, Groq, Fireworks, OpenRouter, Together, Google, etc. Add more than one account for the same provider when you hold several keys, for example one per team or environment.
In the left sidebar, click Accounts.
In the Add account form, complete the following fields:
provider
The provider you are connecting.
name
A name for this account, for example prod or my-vllm. The name appears in routes, reports, and prices.
key
Your provider API key. Governor stores it envelope encrypted under the key encryption key in your cluster.
base url
The provider endpoint. The default is filled in for each provider. Override it for a proxy, a regional endpoint, or a self-hosted model server.
Click Add.
The account appears in the Accounts table with the actions edit, test, reveal, and remove, and a toggle to enable or disable it.
Click test on the new account.
test checks the connection to the provider to verify the key and endpoint.
Examples
Step 7: Add a route
A route maps a model alias your coding tool requests to a provider account and an upstream model.
In the left sidebar, click Routes.
In the Add route form, complete the following fields:
alias
The model name your coding tool requests, for example claude-opus-4-8 or claude-*. Enter * to match any model.
provider
The provider that serves the request.
account
The provider account used.
model (or *)
The upstream model sent to the provider. Enter * to forward the model name the tool requested.
api
The API dialect. Leave it on auto to follow the caller.
priority
The failover tier. Lower numbers are used first. Default 0.
weight
The share of traffic within a tier. Default 1.
retries
Retry attempts for this target. Leave blank to use the gateway default.
Click Add route.
Routes are grouped by alias in the table below the form. Each group lists its targets with provider, model, account, priority, weight, retries, an enable toggle, and a health state. Priority, weight, and retries are editable inline.
A target marked cooling is in a cooldown period after repeated failures. Governor sends its traffic to the next available target until it recovers.
Wildcard and exact aliases
An alias can be an exact model name such as claude-opus-4-8, or a wildcard such as claude-sonnet-* or *.
A route with * as both the alias and the upstream model passes every request through to the selected account unchanged.
Exact aliases take precedence over wildcards. A route for claude-opus-4-8 is used instead of a claude-* route, and a claude-* route is used instead of *. This lets you run one catch-all route so that every model reaches a provider, then override selectively for the models you care about.
Failover and load balancing
Add the same alias again with a different provider account.
Governor uses the lowest priority tier first.
Within a tier, traffic is distributed by weight.
Targets with the same priority receive requests in turn.
If a provider returns errors, Governor sends the request to the next available target.
Examples
Step 8: Create a gateway key
A gateway key authenticates a coding tool to Governor. Provider keys stay in the Bito UI.
In the left sidebar, click Keys.
In the Create key form, enter a name that identifies the team or tool that will use the key.
Click Create.
Copy the key.
The key starts with gw_sk_ and is displayed once. Governor stores keys hashed and cannot display them again.
The Keys table lists each key by ID, prefix, and name, with an enable toggle and a revoke action. Create one key per team or tool so that you can revoke one without affecting the others.
Step 9: Enable features
Features are server-side capabilities that Governor runs inside a request. Two features are available, and both are optional.
The Features table lists each enabled feature with its alias, state, whether a token is stored, and its MCP URL, with edit, test connection, and disable actions. Enabled features also appear as badges against each alias on the Routes page.
AI Architect
AI Architect serves system context from a live knowledge graph of your engineering system, covering code, business context, and tribal knowledge. Governor applies it inside each request, so your coding tools receive that context as they work.
In the left sidebar, click Features.
In the Configure feature form, select
ai_architectfrom the feature list.Complete the following fields:
alias
Leave blank to enable the feature across the workspace, or enter a route alias such as claude-* to scope it to that alias.
MCP URL
Your AI Architect MCP endpoint. If left empty, Governor falls back to a built-in stub.
Steering text
Overrides the default instructions Governor sends with AI Architect. Leave blank to use the default.
Tool allowlist
Restricts which AI Architect tools the model may call, one tool name per line. Leave empty to allow all.
Max hops
The hop budget for a single AI Architect lookup. Default 16. See Max hops below.
Max hops per request
The combined hop budget across every AI Architect lookup in one request. Leave blank to use the gateway default.
Prompt-cache injection
Caches what Governor sends with AI Architect on Anthropic, so repeat hops bill at the cache-read rate. Leave on Use default to follow the gateway-wide setting.
Architect model (sub-agent)
Runs the AI Architect lookups on a cost-efficient sub-agent while the route model writes the answer, for example GPT-5.6 Luna or Gemini Flash Lite 3.5. Leave blank to run them on the request's own route model. You must configure a sub-agent to capture most of the cost savings.
Run Architect in-loop (legacy)
Runs the AI Architect tools inline on the route model instead of the default sub-agent mode. Ignored when an Architect model is set.
Delegation pressure
How hard the assistant is pushed to consult AI Architect.
Balanced consults it when the assistant judges research worthwhile, which suits coding tools whose own file search competes for the same job.
Aggressive tells the assistant to consult AI Architect first, before searching files itself, which suits chat-style products.
Left unset it follows the Architect model setting above — Balanced with none pinned, Aggressive with one, because a pinned model makes research cheap enough to lean on; the unset option names whichever applies right now. Choosing a value explicitly overrides that link and holds even if the Architect model changes later.
Evidence format
What a finished research call hands back.
Prose returns a written answer with citations.
Structured returns the same research as separate findings, each carrying a verbatim code excerpt with its file and line.
Prose is recommended, and matched or beat Structured on accuracy in five test setups out of six while costing less in five out of six, because excerpts add to the size of every request. Choose Structured when you parse the findings yourself.
Default: Prose
Quality mode
Sets how deeply AI Architect researches a question.
Choose one of the following:
Use default: follows the gateway-wide setting.
Normal: uses the standard prompts and hop budget. This is the default.
High quality: researches deeper and more thoroughly, at roughly twice the AI Architect cost.
Codebase context
Codebase context tells the assistant about your own codebase before it starts work.
This is the highest-impact setting on this form, and it is off by default.
Conventions sends a short summary of the repository the person is working in, covering how you handle errors and logging, how you name things, how you test, and your security and module boundaries. The summary is the same for every request, so it is inexpensive to send repeatedly, and in testing it roughly halved the cost of a coding session.
Conventions + task research also reads the request, works out what kind of work it is, and looks that up before answering, covering where the change belongs, what it would break, which pattern to follow, and what is already underway.
Default: Off
Include risk areas and in-flight work
Also tells the assistant about known risk areas, technical debt, and work currently underway in the part of the codebase the caller is working in. Carries no contributor names.
Available only when Codebase context is on.
MCP token
Your AI Architect access token. Leave blank to keep the token already stored.
Task guidance
Task guidance adds what only your indexed codebase can supply to three kinds of request. Each setting is off by default and applies only to the request type named.
Guide what plans cover
When someone asks the assistant to plan or scope a change, it also covers what else the change would affect across your repositories, the patterns your code already follows, the business rules that constrain it, and the parts of the code that are risky to touch.
This makes plans more complete rather than more accurate, so it ensures the important ground is covered without making individual details more likely to be right.
In testing, plans that addressed those areas rose from about half to roughly four in five, with no slowdown.
Applies to planning, design, and implementation requests. Questions and troubleshooting are unaffected.
Guide what reviews cover
When someone asks what the codebase requires of a change they are about to make, the assistant also establishes which other repositories and services call the code being changed and would have to move with it, the convention the affected code is meant to follow, and the invariants the change could break.
The cross-repository part is the point, because call sites and conventions inside one repository are already searchable while a consumer in another service is not.
Applies to review requests only.
Guide what triage covers
When someone brings a bug or an outage, the assistant first checks what its local view cannot show, including whether an incident or alert is already firing for the services involved, what recently deployed in that area, and any issue already recorded against it.
Tracing the code is something the assistant can already do from the repository, while knowing that an alert has been firing since this morning is not.
Applies to bug reports and troubleshooting only.
Click Enable.
Changes apply to the next request. Users take no action.
Max hops
Some questions require several passes to answer. A question about how your repositories connect requires Governor to retrieve the repository list, then look up the dependencies of each repository. Each pass is a hop.
Two settings cap this work, and both apply at the same time.
Max hops
One AI Architect lookup.
Max hops per request
Every AI Architect lookup in one request, combined.
A single request can trigger more than one lookup, so Max hops per request is what stops a complex request from running up cost through repeated lookups that each stay within their own limit.
Governor stops as soon as it has an answer, so both values are ceilings rather than fixed costs.
16
Default. Suitable for most workspaces.
30 or higher
Workspaces with several hundred repositories, or teams that ask broad cross-repository questions.
Raise Max hops if answers come back incomplete. Leave Max hops per request blank to use the gateway default, and set it when you want a firm ceiling on how much AI Architect work a single request can do. Each hop consumes tokens.
Reasoning downgrade
reasoning_downgrade lowers the reasoning effort of a request by exactly one level. Reasoning tokens bill at the output rate, so a lower level reduces the cost of the request.
In the left sidebar, click Features.
In the Configure feature form, select
reasoning_downgradefrom the feature list.Leave alias blank to apply the feature across the workspace, or enter a route alias such as
claude-*to scope it to that alias.Click Enable.
Apart from alias, the feature has no settings.
Governor leaves a request unchanged when it already uses the lowest or second-lowest reasoning level, or when it sends no reasoning at all.
Step 10: Connect a coding tool
In the left sidebar, click Documentation. Your own base URL is displayed at the top of the page, and the links below it jump to the four sections on the page.
Get started
Your base URL, a field for your gateway key, and how to authenticate.
Use it
Ready-made curl, Python, and JavaScript requests, and Try it for sending a live request.
Connect a tool
Copy-paste setup for each supported coding tool.
Reference
Endpoints, your route aliases, and how usage and cost are reported.
To connect a coding tool:
In the Get started section, paste a gateway key into the key field. Every example on the page fills in with your base URL and key. To generate a key here, click + Create test key.
In the Connect a tool section, select your tool.
Setup is provided for Claude Code, Cursor, Cline, Continue, Aider, Codex CLI, GitHub Copilot CLI, Windsurf, Zed, and any OpenAI-compatible or Anthropic-compatible tool.
To print the same setup from a terminal, run gwctl connect inside the gateway pod. The command outputs configuration values and changes no state.
The output contains your data-plane URL and the required environment variables, with a <YOUR_GATEWAY_KEY> placeholder. Replace it with the key you created in Step 8.
For Claude Code, set two environment variables and start the tool:
Replace GATEWAY_HOST with your Ingress hostname, or use http://localhost:8788 while port forwarding. To configure a team, distribute these two variables using your existing developer environment tooling. If your traffic already passes through a central gateway, set them there instead.
Governor accepts requests on three endpoints:
/v1/messages
Anthropic Messages
Claude Code, Anthropic SDKs
/v1/chat/completions
OpenAI Chat Completions
Codex, most OpenAI-compatible tools
/v1/responses
OpenAI Responses
OpenAI Responses API clients
Authenticate with Authorization: Bearer <GATEWAY_KEY> or X-Api-Key: <GATEWAY_KEY>. The same gateway key works for all three dialects.
Operate the Bito Governor
Verify the configuration
Check route selection. In the left sidebar, click Test. Enter a model alias and click Test. Governor returns the route, provider, account, and lane it would use. No request is sent, so this costs nothing.
Send a request. In the left sidebar, click Documentation. In the Use it section, go to Try it. Paste a gateway key, select a model, type a prompt in the message box, and click Run. This sends a real, billable request to your provider.
Query your own system. From your coding tool, ask a question that requires knowledge of your repositories. A response naming your own services confirms AI Architect is active.
Check the report. In the left sidebar, click Reports and confirm the requests appear.
Set prices
Token prices produce the cost figures on the Dashboard and in Reports. Prices are expressed in $/Mtok, meaning US dollars per million tokens.
Global defaults are maintained by your gateway operator. Set a price here to override the default for your workspace, or to record a negotiated rate for one account. Governor resolves prices in this order: account, then workspace, then global.
In the left sidebar, click Prices.
In the Set price form, select the vendor.
Select an account to apply the price to that account only, or leave it on all accounts to apply the vendor rate across your workspace.
Enter the model name.
Enter in, out, and optionally cache-read and cache-write prices, all in
$/Mtok.Click Set.
The Prices table lists each price with its scope, vendor, model, rates, and source.
A model with no price reports token counts and a cost of zero. The Dashboard shows the number of unpriced requests per model in the UNPRICED column.
Set limits
Limits are applied per gateway key.
In the left sidebar, click Limits.
In the Set limits form, select a key.
Complete any of the following fields:
rpm
Maximum requests per minute.
tpm
Maximum tokens per minute.
budget ($/month)
Monthly spend cap. Requires prices to be set.
max_concurrency
Maximum concurrent requests.
Click Set.
Leave a field blank to leave it unchanged. Enter 0 to remove a cap, which makes that limit unlimited.
Monitor usage and cost
Dashboard
The Dashboard shows spend and request volume for the last 30 days, with totals for requests, input tokens, output tokens, and cost. Use Group by to break the numbers down by model, alias, key, provider, account, detail, or feature.
The table below the chart lists requests, token counts, cost, and unpriced request count per model.
Reports
Reports is the raw event log, with one row per request, updated in near real time. Filter the log and export it with Download CSV.
Token counts are split into four buckets:
in
Fresh prompt tokens, excluding anything served from cache.
cached
Prompt tokens read from cache, billed at a lower rate.
cache_w
Tokens written to the cache. On Anthropic this carries a small premium.
out
All generated tokens, including reasoning tokens.
The full prompt your tool sent is in + cached + cache_w. The buckets do not overlap, so nothing is counted twice.
Features make hidden calls inside a request and are reported separately. The base columns show the answer the caller received, and the feature columns show the feature's own usage. A request's total is base plus feature. Expand a row to see the split.
Measure the effect of AI Architect
Governor does not currently report savings against a baseline. To measure the effect:
Select a set of tasks your team runs regularly.
Disable AI Architect, run the tasks, and record cost per task from Reports.
Enable AI Architect and run the same tasks with the same tool and model.
Compare cost per task and confirm the tasks still complete correctly.
Add members
In the left sidebar, click Members.
In the Add team member form, enter a name and select a role.
Click Create.
Each member signs in to the admin UI with the token generated for them.
Member
Manages their own gateway keys and views their own usage.
Workspace admin
Full configuration access to the workspace, including routes, accounts, features, and limits.
You can disable a member from the Team table.
View the audit log
Audit records every configuration change and every secret reveal in your workspace, with the admin who performed it and a timestamp. The log is read only.
Give each person their own member token, so that the log identifies who made each change.
Use your own MySQL or Redis
The chart brings up both by default. Turn off either one independently and point the gateway at your own.
MySQL
mysql.enabled: false
config.DB_HOST, config.DB_PORT, config.DB_NAME, config.DB_USER, and config.DB_TLS
Redis or valkey
redis.enabled: false
config.REDIS_ADDR, plus REDIS_TLS and REDIS_PASSWORD
Both
Both false
Use values-external-db.yaml
For an external database, create the database and a user that may run DDL on the first install. The gateway creates its own tables.
Deliver REDIS_PASSWORD through your Secret rather than through config, in the same way as every other credential.
Run more than one replica
Governor holds one piece of shared mutable state, the counter store behind your rate limits, budgets, and route health. In-memory counters are per-pod, so running several replicas without a shared store multiplies every limit by the number of pods.
Before raising replicaCount above 1 or enabling autoscaling, set config.STORE_BACKEND to redis, valkey, or aerospike, and set config.REDIS_ADDR to a single host:port for the Redis and valkey backends. The chart refuses to install a multi-replica release without one.
Sentinel and Cluster are unsupported. Use a single endpoint.
Upgrade Governor
Pass no new secret values on upgrade. The Secret is yours rather than the chart's, and both DB_PASSWORD and CRYPTO_ENV_KEK_KEY are fixed after the first install.
A change to the chart's configuration rolls the pods for you. A change to a Secret you own does not, because the chart cannot read it, so roll the pods yourself after rotating one:
Roll back
A rollback returns the pods to the previous image while the database keeps the newer schema. Bito keeps migrations backward compatible with the previous release for this reason. A schema rollback is a separate, deliberate operation.
Uninstall
Your data survives. The PersistentVolumeClaims are annotated to be kept, so helm uninstall leaves them behind and a reinstall under the same release name adopts them.
Remove them deliberately when you want the data gone:
Helm also leaves the migration objects behind:
Troubleshooting
Start with the pod status, which names most problems on its own:
ImagePullBackOff on any pod
Docker Hub is rate limiting anonymous pulls, or your registry needs credentials.
Set imagePullSecrets, which every pod in the chart renders.
The gateway pod crash loops with an access-denied error
config.DB_USER does not match the database user.
Set config.DB_USER. The binary defaults to root, and the in-chart MySQL root password is random.
The database pod exits during startup
An init step failed.
Read its logs with kubectl -n bito-gateway logs deploy/gw-bito-gateway-mysql.
CreateContainerConfigError on the database pod
The Secret named by secrets.existingSecret is missing, or has no DB_PASSWORD key.
Create the Secret in the release namespace, then roll the pods.
Every pod is Ready, /healthz returns 200, every call returns 401
The database has no workspace or gateway key yet.
Create them, as in Step 5.
/admin/api returns 401 with the token you read back
ADMIN_TOKEN was absent from the Secret when the pod started.
Add it, then run kubectl -n bito-gateway rollout restart deploy/gw-bito-gateway. An existing Secret carries no checksum, so nothing rolls the pods for you.
The gateway is Ready but every request fails
The database pod restarted and re-ran its init, so the schema is fresh and holds no tenants.
Recreate your workspace and keys, and check that persistence is enabled.
Answers are truncated mid-sentence
Your ingress controller timed the response out.
Raise the response timeout, as in Step 4.
401 from Governor with a gateway key
The key is invalid or revoked.
Create a new key on the Keys page and update the tool configuration.
404 for a model
No route matches the requested alias.
Add a route for the exact model name, or add a * route.
Requests reach an unexpected model
An exact alias takes precedence over the * route.
Check the Routes page. Exact aliases override wildcards.
A route shows cooling
The target failed repeatedly and is in a cooldown.
Check the provider account with test on the Accounts page. Traffic uses the next target until it recovers.
Cost column shows zero, or UNPRICED is high
The model has no price set.
Add prices for that model on the Prices page.
AI Architect responses lack system context
The feature is disabled, scoped to a different alias, or the MCP URL is empty.
Check the alias, MCP URL, and token on the Features page, then click test connection. An empty MCP URL falls back to a built-in stub.
Broad questions return incomplete answers
Governor reached a hop budget.
Increase Max hops, and check Max hops per request if the request makes several AI Architect lookups.
What's next
Set up Bito Governor (self-hosted with docker) to run Governor on a single host with Docker
Contact support@bito.ai for deployment assistance
Last updated