> For the complete documentation index, see [llms.txt](https://docs.bito.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.bito.ai/governor/set-up-bito-governor-self-hosted-with-kubernetes.md). # Set up Bito Governor (self-hosted with Kubernetes) Run Bito Governor in your own Kubernetes cluster. Install it with the Bito Gateway Helm chart, expose it through your ingress controller, and point your coding tools at it. Governor runs either on Bito's infrastructure or inside your own network. To have Bito run it for you, see [Set up Bito Governor (Bito-hosted)](/governor/set-up-bito-governor-bito-hosted.md). To run it on a single host with Docker rather than Kubernetes, see [Set up Bito Governor (self-hosted with Docker)](/governor/set-up-bito-governor-self-hosted-with-docker.md). With Kubernetes, you install Governor from the Bito Gateway Helm chart. The chart defaults bring up the gateway, a MySQL database, and a valkey counter store, all on persistent volumes, so nothing outside your cluster is required. Setup has four parts: 1. **Create a Secret.** Four credentials that the gateway reads at startup, including the key that encrypts your provider credentials. 2. **Install the chart.** One Helm command renders and applies every Kubernetes resource. 3. **Expose the gateway.** The install gives you a running gateway rather than a reachable one. 4. **Configure your workspace.** Add a provider account, a route, a gateway key, and any features you want, then connect your coding tools. The steps below follow that order. ## Prerequisites | Requirement | Details | | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Kubernetes cluster | Version 1.23 or later, running in your own environment | | kubectl | Installed, and pointed at the cluster you intend to install into. | | Helm | Version 3.14 or later. Helm 4 is also supported. | | Cluster context | Run `kubectl config current-context` and confirm it names the target cluster. Every command below applies to whichever cluster your context selects. | | Outbound access |

Your cluster needs to reach two destinations:

| Docker is not required on your own machine. Kubernetes runs the container images for you. ## Step 1: Create the namespace and Secret The chart reads its credentials from a Kubernetes Secret that must already exist in the release namespace. The chart does not create it, so this comes first. ```shellscript kubectl create namespace bito-gateway ``` ```bash kubectl create secret generic bito-gateway-secrets \ --namespace bito-gateway \ --from-literal=DB_PASSWORD="$(openssl rand -base64 24 | tr -d '/+=')" \ --from-literal=CRYPTO_ENV_KEK_KEY="$(openssl rand -base64 32)" \ --from-literal=MYSQL_ROOT_PASSWORD="$(openssl rand -base64 24 | tr -d '/+=')" \ --from-literal=ADMIN_TOKEN="$(openssl rand -base64 32 | tr -d '/+=')" ``` Run the command as written. The `openssl` calls generate each value for you, so nothing in it needs replacing. | Key | What it is | | --------------------- | ---------------------------------------------------------------------------------------------------------------------- | | `DB_PASSWORD` | The password the gateway uses to reach its database. | | `CRYPTO_ENV_KEK_KEY` | The key encryption key. It encrypts every provider credential Governor stores, and must be base64 of exactly 32 bytes. | | `MYSQL_ROOT_PASSWORD` | The root password for the in-cluster MySQL database. | | `ADMIN_TOKEN` | The bootstrap admin credential, used to sign in to the admin UI the first time. | {% hint style="info" %} **Back up `CRYPTO_ENV_KEK_KEY` outside this cluster.** It encrypts every provider credential in your database. If you lose it, those credentials cannot be decrypted, including credentials in a database backup. Never re-roll it against a database that already holds provider accounts. {% endhint %} {% hint style="info" %} `DB_PASSWORD` reaches MySQL only once, when the database initializes on first boot. Changing it in the Secret afterwards leaves MySQL on the old password and the gateway on the new one, which produces an access-denied crash loop that a restart does not clear. Rotate the password inside MySQL first. {% endhint %} ## Step 2: Install the chart ```bash helm install gw oci://registry-1.docker.io/bitoai/bito-gateway-helm \ --namespace bito-gateway \ --set secrets.existingSecret=bito-gateway-secrets ``` `gw` is the release name. Choose any name you like. It prefixes every resource the chart creates, so a release named `gw` produces a deployment named `gw-bito-gateway`. The command installs the newest published chart version. To pin a version for reproducible deployments, add `--version X.Y.Z`. Available versions are listed under the [Tags tab](https://hub.docker.com/r/bitoai/bito-gateway-helm/tags). To see what a version contains before installing it, run `helm show chart oci://registry-1.docker.io/bitoai/bito-gateway-helm`. The install renders three Deployments, three Services, two PersistentVolumeClaims, and a provisioning Job. {% hint style="info" %} The Helm chart is published at [hub.docker.com/r/bitoai/bito-gateway-helm](https://hub.docker.com/r/bitoai/bito-gateway-helm). {% endhint %} {% hint style="info" %} Helm reports `STATUS: deployed` before a single image is pulled, so a successful install command is not proof that Governor is running. Verify it yourself in Step 3. {% endhint %} #### Choose a values file The chart ships example values files, each a complete starting point you copy and edit. Installing with no values file gives you the same result as `values-selfhosted.yaml`. | File | Use it when | | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | | `values-selfhosted.yaml` | Start here. The defaults written out and explained, with the gateway, MySQL, and valkey all in-cluster on persistent volumes. | | `values-external-db.yaml` | You have a managed database such as RDS, Cloud SQL, or Azure Database, with TLS and credentials from a Secret you already own. | | `values-production-ha.yaml` | You want multiple replicas with a shared counter store. | | `values-single-node.yaml` | Small or non-production, with your database already in the cluster. | | `values-demo.yaml` | Evaluating — in-cluster stores with **no persistent volume**; every restart loses all data | Unpack the chart to get them: ```bash helm pull oci://registry-1.docker.io/bitoai/bito-gateway-helm --untar ``` ```shellscript ls bito-gateway-helm/values-*.yaml ``` Pass your file with `-f`: ```bash helm install gw oci://registry-1.docker.io/bitoai/bito-gateway-helm \ --namespace bito-gateway -f my-values.yaml ``` Put your settings in a file of your own rather than editing `values.yaml`, which ships inside the chart so that you can read what each setting does. ## Step 3: Verify the deployment ```bash kubectl -n bito-gateway rollout status deploy/gw-bito-gateway --timeout=5m ``` ```shellscript kubectl -n bito-gateway get pods ``` Every pod reaches `Running`, apart from the provisioning Job, which reaches `Completed`. Output: ``` NAME READY STATUS RESTARTS AGE gw-bito-gateway-75f659b99c-chksx 1/1 Running 0 6m51s gw-bito-gateway-mysql-586665659d-gvjhr 1/1 Running 0 6m51s gw-bito-gateway-mysql-provision-vl7vg 0/1 Completed 0 6m51s gw-bito-gateway-redis-6df545479b-8zpff 1/1 Running 0 6m51s ``` ## Step 4: Expose the gateway The install gives you a running gateway rather than a reachable one. Port forwarding is enough for a first look, and an Ingress is what your team connects to. #### Port forward, for a first look ```bash kubectl -n bito-gateway port-forward svc/gw-bito-gateway 8788:8788 ``` The gateway is then available at `http://localhost:8788`, and the admin UI at `http://localhost:8788/admin/`. This reaches only your own machine, and only while the command runs. #### Ingress, for your team `values-selfhosted.yaml` carries four ready-to-uncomment Ingress presets, for **nginx**, **AWS ALB**, **GKE**, and **Azure AGIC**. Uncomment the one matching your ingress controller, set your host, and apply it with `helm upgrade`. Point a DNS record at the load balancer address your controller provisions, and your team reaches Governor at that hostname. {% hint style="info" %} Raise the response timeout on your ingress controller. LLM responses stream for minutes, and a default timeout truncates answers mid-sentence. Each controller spells this differently, which is why the presets exist. {% endhint %} {% hint style="info" %} A `/` path rule publishes the admin UI and admin API alongside the data plane. Scope the path to `/v1` to publish the data plane alone. {% endhint %} ## Step 5: Sign in to the admin UI 1. Read your admin token back from the Secret. Kubernetes stores Secret values base64 encoded, so the `base64 -d` step decodes it. ```bash kubectl -n bito-gateway get secret bito-gateway-secrets \ -o jsonpath='{.data.ADMIN_TOKEN}' | base64 -d ``` 2. Open the admin UI at your Ingress hostname, or at `http://localhost:8788/admin/` while port forwarding, and sign in with that token. 3. Create a workspace and open it — your tenant (like a team/project).
The workspace opens on the **Dashboard**. The left sidebar contains the following pages: Dashboard, Documentation, Reports, Keys, Members, Accounts, Routes, Features, Limits, Prices, Test, and Audit.
Most configuration pages contain a form at the top and a table of existing entries below it. Complete Steps 6 to 11 in the admin UI. #### Configure from the command line instead `gwctl` command ships inside the gateway image and reads the database settings the pod already has. Substitute the workspace ID that the first command prints. ```bash kubectl -n bito-gateway exec deploy/gw-bito-gateway -- gwctl workspace create --name demo ``` ```shellscript kubectl -n bito-gateway exec deploy/gw-bito-gateway -- gwctl key create --workspace --name demo ``` ```shellscript kubectl -n bito-gateway exec deploy/gw-bito-gateway -- gwctl account add --workspace --provider anthropic --key ``` ```shellscript kubectl -n bito-gateway exec deploy/gw-bito-gateway -- gwctl route add --workspace --alias '*' --provider anthropic ``` {% hint style="info" %} Note: `key create` prints the `gw_sk_` key once. {% endhint %} {% hint style="info" %} Every pod can be `Ready`, `/healthz` can return `200`, and every request can still return `401`. That is an empty database rather than a failure. Create a workspace and a gateway key, and the `401` stops. {% endhint %} #### Use a database-backed admin instead of ADMIN\_TOKEN `ADMIN_TOKEN` exists as a bootstrap credential for the case where the database has no rows yet. To avoid storing an admin credential in a Kubernetes Secret, leave it out of the Secret and mint an admin against the database instead. ```bash kubectl -n bito-gateway exec deploy/gw-bito-gateway -- gwctl admin create --role global ``` The command prints the token once. Governor stores only a SHA-256 hash of it, so the raw value never reaches disk or cluster state. | | ADMIN\_TOKEN in a Secret | gwctl admin create | | ------------------ | --------------------------- | ---------------------------------- | | Credential at rest | In your Secret | A hash in the database | | Takes effect | After a pod restart | Immediately | | Revoking | Edit the Secret and restart | `gwctl admin revoke ` | | Audit trail | One shared identity | One row per admin | | Scope | Always global | Global, or scoped to one workspace | Leaving `ADMIN_TOKEN` unset is a supported configuration. The gateway boots normally and serves traffic, and only the bootstrap admin is disabled. ## Step 6: Add a provider account A provider account stores one LLM API key. Add an account for every LLM provider you use, such as Anthropic, OpenAI, Groq, Fireworks, OpenRouter, Together, Google, etc. Add more than one account for the same provider when you hold several keys, for example one per team or environment. 1. In the left sidebar, click **Accounts**. 2. In the **Add account** form, complete the following fields: | Field | Description | | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | | provider | The provider you are connecting. | | name | A name for this account, for example `prod` or `my-vllm`. The name appears in routes, reports, and prices. | | key | Your provider API key. Governor stores it envelope encrypted under the key encryption key in your cluster. | | base url | The provider endpoint. The default is filled in for each provider. Override it for a proxy, a regional endpoint, or a self-hosted model server. | 3. Click **Add**. The account appears in the **Accounts** table with the actions **edit**, **test**, **reveal**, and **remove**, and a toggle to enable or disable it. 4. Click **test** on the new account. **test** checks the connection to the provider to verify the key and endpoint. {% hint style="info" %} To forward the credential supplied by the caller, leave the **key** field blank. This is passthrough mode. Azure accounts can authenticate with an Entra ID service principal instead of an API key. {% endhint %} {% hint style="info" %} **reveal** displays a stored provider key. Every use is recorded in the audit log. {% endhint %} #### Examples
Expand to view examples Each example states a goal, then the values to enter in the **Add account** form. #### Connect a provider Add your organization's Anthropic key. | provider | name | key | base url | | --------- | ---------------- | ---------------------- | ------------------------- | | anthropic | `prod-anthropic` | your Anthropic API key | leave the prefilled value | Governor fills in **base url** when you select a provider. Change it only for a proxy, a regional endpoint, or a server you host yourself. #### Separate production and staging spend Bill staging traffic to a different key from production. | provider | name | key | base url | | --------- | ------------------- | ------------------- | ------------------------- | | anthropic | `prod-anthropic` | your production key | leave the prefilled value | | anthropic | `staging-anthropic` | your staging key | leave the prefilled value | Add the provider twice under different names. Routes select an account by name, and reports show which account served each request, so spend separates cleanly. #### Connect a model you host yourself Send requests to a model running on your own inference server. | provider | name | key | base url | | -------- | --------- | -------------------------------------------------- | ----------------------------- | | openai | `my-vllm` | your server's key, or leave empty if it needs none | your inference server address | Any server that implements the OpenAI API works here. #### Forward the caller's own key Apply routing and reporting to traffic without storing provider keys centrally. | provider | name | key | base url | | -------- | -------------------- | ----------- | ------------------------- | | openai | `passthrough-openai` | leave empty | leave the prefilled value | An account with no key runs in passthrough mode, where Governor forwards the credential the caller supplied.
## Step 7: Add a route A route maps a model alias your coding tool requests to a provider account and an upstream model. 1. In the left sidebar, click **Routes**. 2. In the **Add route** form, complete the following fields:
FieldDescription
aliasThe model name your coding tool requests, for example claude-opus-4-8 or claude-*. Enter * to match any model.
providerThe provider that serves the request.
accountThe provider account used.
model (or *)The upstream model sent to the provider. Enter * to forward the model name the tool requested.
apiThe API dialect. Leave it on auto to follow the caller.
priorityThe failover tier. Lower numbers are used first. Default 0.
weightThe share of traffic within a tier. Default 1.
retriesRetry attempts for this target. Leave blank to use the gateway default.
auto-route this aliasAdds the route, then opens auto-routing so you can say which models answer which kind of request. It starts as a dry run: every request is classified and recorded, and this route keeps answering until you turn routing on.

See Auto AI model routing for more details.
backing group (internal)Hides this route from your apps. They cannot ask for it by name and it does not appear in the model list, so only auto-routing can send traffic here. Use it for the models a router picks between, so nobody skips the router by asking for one directly.
3. Click **Add route**. Routes are grouped by alias in the table below the form. Each group lists its targets with provider, model, account, priority, weight, retries, an enable toggle, and a health state. Priority, weight, and retries are editable inline. Each group also carries a **Direct** and an **Auto** control. **Direct** sends every request for that alias to the targets you configured. **Auto** hands the alias to the auto-router, which picks a model for each request. A target marked **cooling** is in a cooldown period after repeated failures. Governor sends its traffic to the next available target until it recovers. #### Wildcard and exact aliases An alias can be an exact model name such as `claude-opus-4-8`, or a wildcard such as `claude-sonnet-*` or `*`. A route with `*` as both the alias and the upstream model passes every request through to the selected account unchanged. Exact aliases take precedence over wildcards. A route for `claude-opus-4-8` is used instead of a `claude-*` route, and a `claude-*` route is used instead of `*`. This lets you run one catch-all route so that every model reaches a provider, then override selectively for the models you care about. #### Failover and load balancing Add the same alias again with a different provider account. * Governor uses the lowest priority tier first. * Within a tier, traffic is distributed by weight. * Targets with the same priority receive requests in turn. * If a provider returns errors, Governor sends the request to the next available target. #### Auto-routing An alias can answer from the targets you configured, or you can hand it to the auto-router, which classifies each request and picks a model for it. Select **auto-route this alias** when you add the route, or click **Auto** on the group in the routes table. See [Auto AI model routing](/governor/auto-ai-model-routing.md) for more details. #### Examples
Expand to view examples Each example states a goal, then the values to enter in the **Add route** form. #### Route every model to one provider Send all traffic to a single account, without naming each model. | alias | provider | account | model | priority | | ----- | --------- | ---------------- | ----- | -------- | | `*` | anthropic | `prod-anthropic` | `*` | 0 | Type the asterisk in both fields. `*` in the alias matches any model, and `*` in the model field forwards the model name your tool sent. Add this route first, so that every request reaches a provider while you configure the rest. #### Route a model family to one account Send every version of Sonnet to an Azure account. | alias | provider | account | model | priority | | ----------------- | -------- | ------------ | ----- | -------- | | `claude-sonnet-*` | azure | `azure-prod` | `*` | 0 | The wildcard matches `claude-sonnet-5`, `claude-sonnet-4-5`, and later versions, so new releases need no new route. This alias is more specific than `*`, so it overrides the catch-all. #### Fail over to a second provider Serve Opus 5 from Anthropic, and switch to Azure while Anthropic is unavailable. | alias | provider | account | model | priority | | --------------- | --------- | ---------------- | --------------- | -------- | | `claude-opus-5` | anthropic | `prod-anthropic` | `claude-opus-5` | 0 | | `claude-opus-5` | azure | `azure-prod` | `claude-opus-5` | 1 | Add the alias once per account. Priority `0` takes all traffic. When those requests fail, Governor moves them to priority `1` until the primary account recovers. #### Split traffic across two accounts Spread Opus 5 traffic so that neither account reaches its rate limit. | alias | provider | account | model | priority | weight | | --------------- | --------- | ---------------- | --------------- | -------- | ------ | | `claude-opus-5` | anthropic | `prod-anthropic` | `claude-opus-5` | 0 | 3 | | `claude-opus-5` | azure | `azure-prod` | `claude-opus-5` | 0 | 1 | Equal priorities put both targets in the same tier, and the weights send three requests to Anthropic for every one to Azure. Set both weights to `1` for an even split. #### Serve a request with a different model Serve Opus 5 requests with a lower-cost model, with no change on any developer machine. | alias | provider | account | model | priority | | --------------- | --------- | ---------------- | ----------------- | -------- | | `claude-opus-5` | anthropic | `prod-anthropic` | `claude-sonnet-5` | 0 | The alias is the model your tool requests. The model is what serves the request. Your tools continue to request Opus 5, and Governor serves those requests with Sonnet 5. Edit the route to reverse it.
{% hint style="info" %} Before applying a model substitution across your workspace, run it against a representative set of your own tasks and compare results as well as cost. {% endhint %} ## Step 8: Create a gateway key A gateway key authenticates a coding tool to Governor. Provider keys stay in the Bito UI. 1. In the left sidebar, click **Keys**. 2. In the **Create key** form, enter a **name** that identifies the team or tool that will use the key. 3. Click **Create**. 4. Copy the key. The key starts with `gw_sk_` and is displayed once. Governor stores keys hashed and cannot display them again. The **Keys** table lists each key by ID, prefix, and name, with an enable toggle and a **revoke** action. Create one key per team or tool so that you can revoke one without affecting the others. ## Step 9: Enable features Features are server-side capabilities that Governor runs inside a request. Two features are available, and both are optional. The **Features** table lists each enabled feature with its alias, state, whether a token is stored, and its MCP URL, with **edit**, **test connection**, and **disable** actions. Enabled features also appear as badges against each alias on the **Routes** page. #### AI Architect AI Architect serves system context from a live knowledge graph of your engineering system, covering code, business context, and tribal knowledge. Governor applies it inside each request, so your coding tools receive that context as they work. 1. In the left sidebar, click **Features**. 2. In the **Configure feature** form, select `ai_architect` from the **feature** list. 3. Complete the following fields:
FieldDescription
aliasLeave blank to enable the feature across the workspace, or enter a route alias such as claude-* to scope it to that alias.
MCP URLYour AI Architect MCP endpoint. If left empty, Governor falls back to a built-in stub.
Steering textOverrides the default instructions Governor sends with AI Architect. Leave blank to use the default.
Tool allowlistRestricts which AI Architect tools the model may call, one tool name per line. Leave empty to allow all.
Max hopsThe hop budget for a single AI Architect lookup. Default 16. See Max hops below.
Max hops per requestThe combined hop budget across every AI Architect lookup in one request. Leave blank to use the gateway default.
Prompt-cache injectionCaches what Governor sends with AI Architect on Anthropic, so repeat hops bill at the cache-read rate. Leave on Use default to follow the gateway-wide setting.
Architect model (sub-agent)Runs the AI Architect lookups on a cost-efficient sub-agent while the route model writes the answer, for example GPT-5.6 Luna or Gemini Flash Lite 3.5. Leave blank to run them on the request's own route model.

You must configure a sub-agent to capture most of the cost savings.
Run Architect in-loop (legacy)Runs the AI Architect tools inline on the route model instead of the default sub-agent mode. Ignored when an Architect model is set.
Delegation pressure

How hard the assistant is pushed to consult AI Architect.

  • Balanced consults it when the assistant judges research worthwhile, which suits coding tools whose own file search competes for the same job.
  • Aggressive tells the assistant to consult AI Architect first, before searching files itself, which suits chat-style products.

Left unset it follows the Architect model setting above — Balanced with none pinned, Aggressive with one, because a pinned model makes research cheap enough to lean on; the unset option names whichever applies right now. Choosing a value explicitly overrides that link and holds even if the Architect model changes later.

Evidence format

What a finished research call hands back.

  • Prose returns a written answer with citations.
  • Structured returns the same research as separate findings, each carrying a verbatim code excerpt with its file and line.

Prose is recommended, and matched or beat Structured on accuracy in five test setups out of six while costing less in five out of six, because excerpts add to the size of every request. Choose Structured when you parse the findings yourself.

Default: Prose

Quality mode

Sets how deeply AI Architect researches a question.

Choose one of the following:

  • Use default: follows the gateway-wide setting.
  • Normal: uses the standard prompts and hop budget. This is the default.
  • High quality: researches deeper and more thoroughly, at roughly twice the AI Architect cost.
Codebase context

Codebase context tells the assistant about your own codebase before it starts work.

This is the highest-impact setting on this form, and it is off by default.

  • Conventions sends a short summary of the repository the person is working in, covering how you handle errors and logging, how you name things, how you test, and your security and module boundaries. The summary is the same for every request, so it is inexpensive to send repeatedly, and in testing it roughly halved the cost of a coding session.
  • Conventions + task research also reads the request, works out what kind of work it is, and looks that up before answering, covering where the change belongs, what it would break, which pattern to follow, and what is already underway.

Default: Off

Include risk areas and in-flight work

Also tells the assistant about known risk areas, technical debt, and work currently underway in the part of the codebase the caller is working in. Carries no contributor names.

Available only when Codebase context is on.

MCP tokenYour AI Architect access token. Leave blank to keep the token already stored.
**Task guidance** Task guidance adds what only your indexed codebase can supply to three kinds of request. Each setting is off by default and applies only to the request type named. | Field | What it adds | | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Guide what plans cover |

When someone asks the assistant to plan or scope a change, it also covers what else the change would affect across your repositories, the patterns your code already follows, the business rules that constrain it, and the parts of the code that are risky to touch.

This makes plans more complete rather than more accurate, so it ensures the important ground is covered without making individual details more likely to be right.

In testing, plans that addressed those areas rose from about half to roughly four in five, with no slowdown.

Applies to planning, design, and implementation requests. Questions and troubleshooting are unaffected.

| | Guide what reviews cover |

When someone asks what the codebase requires of a change they are about to make, the assistant also establishes which other repositories and services call the code being changed and would have to move with it, the convention the affected code is meant to follow, and the invariants the change could break.

The cross-repository part is the point, because call sites and conventions inside one repository are already searchable while a consumer in another service is not.

Applies to review requests only.

| | Guide what triage covers |

When someone brings a bug or an outage, the assistant first checks what its local view cannot show, including whether an incident or alert is already firing for the services involved, what recently deployed in that area, and any issue already recorded against it.

Tracing the code is something the assistant can already do from the repository, while knowing that an alert has been firing since this morning is not.

Applies to bug reports and troubleshooting only.

| 4. Click **Enable**. Changes apply to the next request. Users take no action. {% hint style="info" %} Setting **Architect model (sub-agent)** to a cost-efficient alias moves the AI Architect lookups off your main model while the route model still writes the answer. This reduces the cost of a request that takes several hops. {% endhint %} #### Max hops Some questions require several passes to answer. A question about how your repositories connect requires Governor to retrieve the repository list, then look up the dependencies of each repository. Each pass is a hop. Two settings cap this work, and both apply at the same time. | Setting | Scope | | -------------------- | --------------------------------------------------- | | Max hops | One AI Architect lookup. | | Max hops per request | Every AI Architect lookup in one request, combined. | A single request can trigger more than one lookup, so **Max hops per request** is what stops a complex request from running up cost through repeated lookups that each stay within their own limit. Governor stops as soon as it has an answer, so both values are ceilings rather than fixed costs. | Max hops | Use | | ------------ | ------------------------------------------------------------------------------------------------- | | 16 | Default. Suitable for most workspaces. | | 30 or higher | Workspaces with several hundred repositories, or teams that ask broad cross-repository questions. | Raise **Max hops** if answers come back incomplete. Leave **Max hops per request** blank to use the gateway default, and set it when you want a firm ceiling on how much AI Architect work a single request can do. Each hop consumes tokens. #### Reasoning downgrade `reasoning_downgrade` lowers the reasoning effort of a request by exactly one level. Reasoning tokens bill at the output rate, so a lower level reduces the cost of the request. 1. In the left sidebar, click **Features**. 2. In the **Configure feature** form, select `reasoning_downgrade` from the **feature** list. 3. Leave **alias** blank to apply the feature across the workspace, or enter a route alias such as `claude-*` to scope it to that alias. 4. Click **Enable**. Apart from **alias**, the feature has no settings. Governor leaves a request unchanged when it already uses the lowest or second-lowest reasoning level, or when it sends no reasoning at all. ## Step 10: Connect a coding tool In the left sidebar, click **Documentation**. Your own base URL is displayed at the top of the page, and the links below it jump to the four sections on the page. | Section | Contents | | -------------- | -------------------------------------------------------------------------------------------- | | Get started | Your base URL, a field for your gateway key, and how to authenticate. | | Use it | Ready-made curl, Python, and JavaScript requests, and **Try it** for sending a live request. | | Connect a tool | Copy-paste setup for each supported coding tool. | | Reference | Endpoints, your route aliases, and how usage and cost are reported. | To connect a coding tool: 1. In the **Get started** section, paste a gateway key into the key field. Every example on the page fills in with your base URL and key. To generate a key here, click **+ Create test key**. 2. In the **Connect a tool** section, select your tool. Setup is provided for Claude Code, Cursor, Cline, Continue, Aider, Codex CLI, GitHub Copilot CLI, Windsurf, Zed, and any OpenAI-compatible or Anthropic-compatible tool. To print the same setup from a terminal, run `gwctl connect` inside the gateway pod. The command outputs configuration values and changes no state. ```bash kubectl -n bito-gateway exec deploy/gw-bito-gateway -- gwctl connect claude # Supported values: claude | cursor | codex | copilot | cline | continue | aider ``` The output contains your data-plane URL and the required environment variables, with a `` placeholder. Replace it with the key you created in Step 8. For Claude Code, set two environment variables and start the tool: ```bash export ANTHROPIC_BASE_URL="https://GATEWAY_HOST" export ANTHROPIC_AUTH_TOKEN="gw_sk_..." claude ``` Replace `GATEWAY_HOST` with your Ingress hostname, or use `http://localhost:8788` while port forwarding. To configure a team, distribute these two variables using your existing developer environment tooling. If your traffic already passes through a central gateway, set them there instead. Governor accepts requests on three endpoints: | Path | Dialect | Used by | | ---------------------- | ----------------------- | ----------------------------------- | | `/v1/messages` | Anthropic Messages | Claude Code, Anthropic SDKs | | `/v1/chat/completions` | OpenAI Chat Completions | Codex, most OpenAI-compatible tools | | `/v1/responses` | OpenAI Responses | OpenAI Responses API clients | Authenticate with `Authorization: Bearer ` or `X-Api-Key: `. The same gateway key works for all three dialects. Every response carries an `x-bito-routed-model` header naming the model that answered. Read that header rather than the `model` field inside the response body, because some providers return the model that was requested and others return the model that answered. The difference matters once auto-routing is on, since the model that answers may not be the one your tool asked for. {% hint style="info" %} Claude Code reports that its connectors are disabled when it runs through Governor, because the session authenticates against Governor rather than a Claude account. This message is expected. AI Architect continues to work, because Governor serves it from the server side. {% endhint %} ## Operate the Bito Governor ### Verify the configuration 1. **Check route selection.** In the left sidebar, click **Test**. Enter a model alias and click **Test**. Governor returns the route, provider, account, and lane it would use. No request is sent, so this costs nothing. 2. **Check auto-routing, if you set it up.** In the left sidebar, click **Routes**. On a routed alias, click **Classify a prompt**, enter a sample request, and confirm it lands in the tier you expect. No request is sent, so this costs nothing. 3. **Send a request.** In the left sidebar, click **Documentation**. In the **Use it** section, go to **Try it**. Paste a gateway key, select a model, type a prompt in the message box, and click **Run**. This sends a real, billable request to your provider. 4. **Query your own system.** From your coding tool, ask a question that requires knowledge of your repositories. A response naming your own services confirms AI Architect is active. 5. **Check the report.** In the left sidebar, click **Reports** and confirm the requests appear. {% hint style="info" %} For a wildcard alias such as `claude-*`, enter a concrete model that matches it, for example `claude-opus-4-8`. {% endhint %} ### Set prices Token prices produce the cost figures on the **Dashboard** and in **Reports**. Prices are expressed in `$/Mtok`, meaning US dollars per million tokens. Global defaults are maintained by your gateway operator. Set a price here to override the default for your workspace, or to record a negotiated rate for one account. Governor resolves prices in this order: account, then workspace, then global. 1. In the left sidebar, click **Prices**. 2. In the **Set price** form, select the **vendor**. 3. Select an **account** to apply the price to that account only, or leave it on **all accounts** to apply the vendor rate across your workspace. 4. Enter the **model** name. 5. Enter **in**, **out**, and optionally **cache-read** and **cache-write** prices, all in `$/Mtok`. 6. Click **Set**. The **Prices** table lists each price with its scope, vendor, model, rates, and source. A model with no price reports token counts and a cost of zero. The **Dashboard** shows the number of unpriced requests per model in the **UNPRICED** column. Auto-routing also depends on these prices. A model with no price cannot be selected by a router, and both the routing direction and the price ceiling are evaluated against them. ### Set limits Limits are applied per gateway key. 1. In the left sidebar, click **Limits**. 2. In the **Set limits** form, select a **key**. 3. Complete any of the following fields: | Field | Description | | ---------------- | --------------------------------------------- | | rpm | Maximum requests per minute. | | tpm | Maximum tokens per minute. | | budget ($/month) | Monthly spend cap. Requires prices to be set. | | max\_concurrency | Maximum concurrent requests. | 4. Click **Set**. Leave a field blank to leave it unchanged. Enter `0` to remove a cap, which makes that limit unlimited. {% hint style="info" %} Setting a limit to `0` removes the cap rather than blocking the key. To stop a key entirely, disable it on the **Keys** page. {% endhint %} ### Monitor usage and cost #### Dashboard The **Dashboard** shows spend and request volume for the last 30 days, with totals for requests, input tokens, output tokens, and cost. Use **Group by** to break the numbers down by model, alias, key, provider, account, detail, or feature. The table below the chart lists requests, token counts, cost, and unpriced request count per model. #### Reports **Reports** is the raw event log, with one row per request, updated in near real time. Filter the log and export it with **Download CSV**. Token counts are split into four buckets: | Bucket | Description | | -------- | ----------------------------------------------------------------------- | | in | Fresh prompt tokens, excluding anything served from cache. | | cached | Prompt tokens read from cache, billed at a lower rate. | | cache\_w | Tokens written to the cache. On Anthropic this carries a small premium. | | out | All generated tokens, including reasoning tokens. | The full prompt your tool sent is `in + cached + cache_w`. The buckets do not overlap, so nothing is counted twice. Features make hidden calls inside a request and are reported separately. The base columns show the answer the caller received, and the feature columns show the feature's own usage. A request's total is base plus feature. Expand a row to see the split. #### Measure the effect of AI Architect Governor does not currently report savings against a baseline. To measure the effect: 1. Select a set of tasks your team runs regularly. 2. Disable AI Architect, run the tasks, and record cost per task from **Reports**. 3. Enable AI Architect and run the same tasks with the same tool and model. 4. Compare cost per task and confirm the tasks still complete correctly. ### Add members 1. In the left sidebar, click **Members**. 2. In the **Add team member** form, enter a **name** and select a **role**. 3. Click **Create**. Each member signs in to the admin UI with the token generated for them. | Role | Permissions | | --------------- | --------------------------------------------------------------------------------------------- | | Member | Manages their own gateway keys and views their own usage. | | Workspace admin | Full configuration access to the workspace, including routes, accounts, features, and limits. | You can disable a member from the **Team** table. ### View the audit log **Audit** records every configuration change and every secret reveal in your workspace, with the admin who performed it and a timestamp. The log is read only. Give each person their own member token, so that the log identifies who made each change. ### Use your own MySQL or Redis The chart brings up both by default. Turn off either one independently and point the gateway at your own. | You already run | Set | Point the gateway at yours with | | --------------- | ---------------------- | ------------------------------------------------------------------------------------------- | | MySQL | `mysql.enabled: false` | `config.DB_HOST`, `config.DB_PORT`, `config.DB_NAME`, `config.DB_USER`, and `config.DB_TLS` | | Redis or valkey | `redis.enabled: false` | `config.REDIS_ADDR`, plus `REDIS_TLS` and `REDIS_PASSWORD` | | Both | Both `false` | Use `values-external-db.yaml` | For an external database, create the database and a user that may run DDL on the first install. The gateway creates its own tables. Deliver `REDIS_PASSWORD` through your Secret rather than through `config`, in the same way as every other credential. ### Run more than one replica Governor holds one piece of shared mutable state, the counter store behind your rate limits, budgets, and route health. In-memory counters are per-pod, so running several replicas without a shared store multiplies every limit by the number of pods. Before raising `replicaCount` above 1 or enabling autoscaling, set `config.STORE_BACKEND` to `redis`, `valkey`, or `aerospike`, and set `config.REDIS_ADDR` to a single `host:port` for the Redis and valkey backends. The chart refuses to install a multi-replica release without one. {% hint style="info" %} Give the counter store a dedicated Redis database or instance, configured with `maxmemory 0` or `maxmemory-policy noeviction`. Every counter carries a TTL, so an eviction policy removes them, and an evicted budget counter reads back as zero spend with no error. Two separate installs must never share a Redis database and key prefix. Their counters are shaped identically and would sum silently, so limits bind early and spend over-counts. {% endhint %} Sentinel and Cluster are unsupported. Use a single endpoint. ### Upgrade Governor ```bash helm upgrade gw oci://registry-1.docker.io/bitoai/bito-gateway-helm \ --namespace bito-gateway -f my-values.yaml --timeout 1800s ``` Pass no new secret values on upgrade. The Secret is yours rather than the chart's, and both `DB_PASSWORD` and `CRYPTO_ENV_KEK_KEY` are fixed after the first install. A change to the chart's configuration rolls the pods for you. A change to a Secret you own does not, because the chart cannot read it, so roll the pods yourself after rotating one: ```bash kubectl -n bito-gateway rollout restart deploy/gw-bito-gateway ``` ### Roll back ```bash helm rollback gw -n bito-gateway ``` A rollback returns the pods to the previous image while the database keeps the newer schema. Bito keeps migrations backward compatible with the previous release for this reason. A schema rollback is a separate, deliberate operation. ### Uninstall ```bash helm uninstall gw -n bito-gateway ``` Your data survives. The PersistentVolumeClaims are annotated to be kept, so `helm uninstall` leaves them behind and a reinstall under the same release name adopts them. Remove them deliberately when you want the data gone: ```bash kubectl -n bito-gateway delete pvc gw-bito-gateway-mysql kubectl -n bito-gateway delete pvc gw-bito-gateway-redis ``` Helm also leaves the migration objects behind: ```bash kubectl -n bito-gateway delete job,configmap,serviceaccount,secret \ gw-bito-gateway-migrate --ignore-not-found ``` ## Troubleshooting Start with the pod status, which names most problems on its own: ```bash kubectl -n bito-gateway get pods kubectl -n bito-gateway logs deploy/gw-bito-gateway ```
SymptomCauseResolution
ImagePullBackOff on any podDocker Hub is rate limiting anonymous pulls, or your registry needs credentials.Set imagePullSecrets, which every pod in the chart renders.
The gateway pod crash loops with an access-denied errorconfig.DB_USER does not match the database user.Set config.DB_USER. The binary defaults to root, and the in-chart MySQL root password is random.
The database pod exits during startupAn init step failed.Read its logs with kubectl -n bito-gateway logs deploy/gw-bito-gateway-mysql.
CreateContainerConfigError on the database podThe Secret named by secrets.existingSecret is missing, or has no DB_PASSWORD key.Create the Secret in the release namespace, then roll the pods.
Every pod is Ready, /healthz returns 200, every call returns 401The database has no workspace or gateway key yet.Create them, as in Step 5.
/admin/api returns 401 with the token you read backADMIN_TOKEN was absent from the Secret when the pod started.Add it, then run kubectl -n bito-gateway rollout restart deploy/gw-bito-gateway. An existing Secret carries no checksum, so nothing rolls the pods for you.
The gateway is Ready but every request failsThe database pod restarted and re-ran its init, so the schema is fresh and holds no tenants.Recreate your workspace and keys, and check that persistence is enabled.
Answers are truncated mid-sentenceYour ingress controller timed the response out.Raise the response timeout, as in Step 4.
401 from Governor with a gateway keyThe gateway key is invalid or revoked.Create a new key on the Keys page and update the tool configuration.
404 for a modelNo route matches the requested alias.Add a route for the exact model name, or add a * route.
Requests reach an unexpected modelAn exact alias takes precedence over the * route.Check the Routes page. Exact aliases override wildcards.
A route shows coolingThe target failed repeatedly and is in a cooldown.Check the provider account with test on the Accounts page. Traffic uses the next target until it recovers.
Cost column shows zero, or UNPRICED is highThe model has no price set.Add prices for that model on the Prices page.
Budget limit has no effectThe model has no price set, or the limit is 0.Set prices, then set a positive budget. 0 means unlimited.
A key still works after setting limits to 00 removes the cap rather than blocking the key.Disable the key on the Keys page.
AI Architect responses lack system contextThe feature is disabled, scoped to a different alias, or the MCP URL is empty.Open the Features page and check the alias, MCP URL, and token. An empty MCP URL falls back to a built-in stub.
Broad questions return incomplete answersGovernor reached the max hops budget.Increase Max hops, and check Max hops per request if the request makes several AI Architect lookups.
Repeated 429 or 5xxOne provider account is rate limited or unavailable.Add a second target on the same alias to enable failover.
Requests reach a model you did not configure for that aliasAuto-routing is on for the alias.Open Routes and check the tier assignments, or click Direct to stop routing that alias.
A check that compares the requested model to the returned model failsThe model field in the response names the model that answered on some providers.Read the x-bito-routed-model response header instead, which always names the model that answered.
Auto-routing is on but nothing is routedThe router is in shadow, or every tier is unusable.Check the mode on Routes. A tier marked will not route has a model with no price, no deployment on the account, or one blocked by your routing direction.
Every request is classified simpleThe LLM classifier is timing out, so the built-in scorer is deciding instead.Raise the classifier timeout, or point classifier model at a faster route alias.
## What's next * [Governor overview](/governor/overview.md) * [Set up Bito Governor (self-hosted with Docker)](/governor/set-up-bito-governor-self-hosted-with-docker.md) to run Governor on a single host with Docker * [Set up Bito Governor (Bito-hosted)](/governor/set-up-bito-governor-bito-hosted.md) * Contact for deployment assistance --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.bito.ai/governor/set-up-bito-governor-self-hosted-with-kubernetes.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.