Setting Up the Multi-Vendor Router, Step by Step

OpenTechnologyApp's chatbot doesn't run on a single AI vendor. An org admin configures three roles — primary, simple, and fallback — each pointing at a vendor and a model. This guide walks the actual Admin Panel screen that sets it up.

Where this lives

Org admin → the "Org AI — Routing Config" card. Three vendor pickers (primary, simple, fallback), each a row of buttons currently limited to Anthropic, OpenAI, and Hugging Face — no other vendor appears in this UI today (see Wiring Ollama for local-only mode for why Ollama isn't one of the three buttons).

The three roles

• Primary vendor + model. What most requests use. The placeholder text changes per vendor — claude-sonnet-4-6 for Anthropic, gpt-4o for OpenAI, and a Hugging Face model id like Qwen/Qwen2.5-72B-Instruct for HF — but the model field is free text, so you can point it at any model name that vendor's API accepts.
• Simple vendor + model. Used for requests the router classifies as low-complexity — labeled "low-complexity queries — saves cost" in the UI. Defaults to Hugging Face with a smaller model (Qwen/Qwen2.5-7B-Instruct by default), because most simple questions don't need a frontier model's reasoning.
• Fallback vendor + model. Used only if the primary vendor's call fails after retries. This is not a manual switch — the router picks it automatically.

Routing thresholds

Two number inputs, labeled "simple below low; tool-heavy at/above high." These are the two cut points the router's complexity score is compared against — see Choosing Complexity Thresholds for what the score actually measures and how to tune these two numbers instead of guessing.

Before you save: the test-model check

The Admin Panel calls a test-model check against your primary vendor and model before persisting the config. This confirms the vendor accepts your API key and model name — it does not confirm every tool works or that your data-access boundaries are correct (see Per-Vendor API Key Security for what a passing test-model check does and doesn't prove).

What happens after you save

Your new config doesn't apply to already-in-flight requests, but it also doesn't need a redeploy — the router caches resolved config per org for 30 seconds and clears that cache automatically when settings are saved. In practice: save, wait a few seconds, and your next chat request uses the new routing. If you're testing and don't see the change immediately, that short cache window is almost always why — it is not a sign that the save failed.

A minimal working setup

If you're starting from scratch and just want routing to work at all:

1. Set primary to Anthropic with a real model name and a valid key.
2. Set simple to Hugging Face — cheapest realistic setup for low-complexity traffic, and HF's free-tier credits are enough to confirm this path works before you commit to real spend (see the hybrid-AI post for what "enough" means in practice — a handful of turns, not a full day of testing).
3. Set fallback to whichever vendor you have a second working key for. If you only have one vendor's key, set fallback to the same vendor as primary — a same-vendor retry is still better than an unhandled failure.
4. Leave the two thresholds at their defaults (0.6 / 0.7) until you have real traffic to tune against.

Send one test message before relying on it in production. A response that comes back fast and reads like it came from the wrong vendor's tone is worth double-checking — see the next guide for how the response itself tells you which vendor actually answered.