You are here: Building with agents › The kits › LlmRouter
Summary#
Osysharp.LlmRouter is a routing policy for an app with more than one model: filters by what a model can do, a quality floor for each kind of work, a cost ceiling, then the cheapest model that clears them — in an order that is also the order they are tried in when one fails.
Without a router an app uses its default model for everything. With this one, no call site names a model: classifying an inbox goes to a small model and drafting a contract clause to a careful one, because the policy says so in one place.
| Package | Osysharp.LlmRouter, version 0.3.0 |
| You add it with | use Osysharp.LlmRouter; in the app's manifest. |
| It sits on | [[agent-kit-llm|Llm]] |
| Its own test app | A print shop with three models and a policy over them — kits/llm-router/tests |
Description#
What do you get without writing anything?#
- Every call is routed, including the calls agents make.
- The reason a model was chosen is written on the meter beside the call.
- When money for a piece of work runs short, the router steps down to a cheaper model that still clears the floor.
When do you need it?#
When your app lists more than one model and you do not want each call to choose. Any shape.
What does it expose?#
The tables below are generated from the kit itself, not written by hand: osy kits Osysharp.LlmRouter --surface lists each of
them with the code of the kit's own test app that uses it, and the tests that show it working.
Types you use
| Type | What it is |
|---|---|
RoutedModel | One model this policy may serve with, and what it can do — declared by the app that names the router, because only the app knows which models it has credentials for. |
QualityFloor | The lowest Quality that may serve one kind of work. |
StepFloor | The lowest Quality that may serve one STEP of a larger job — "plan with rank 3, fill with anything". |
CostTieredRouter | A cost-tiered router — the policy most apps actually want, in the order the decisions have to happen. |
What to read next#
- Osysharp.Llm — every call to a model: which one, what it may cost, what it did cost — every call to a model: which one, what it may cost, what it did cost
- Osysharp.Models.Anthropic — Anthropic's models as a provider — Anthropic's models as a provider
- The map of the kits — which sits on which, and what each one is for — the map of the kits — which sits on which, and what each one is for
Go deeper#
- [[agent-llm-calls#routing]] — how the model for a call is chosen
- Model spend — budgets, the meter and the price book (Llm kit) — what each call cost