# Model spend — budgets, the meter and the price book (Llm kit)

> The Llm kit keeps three things about what model calls cost. The METER: one `ModelCall` row for every attempt, with the model, the router's reason, the provider's token counts and the cost in millionths of a dollar. The PRICE BOOK: what a million tokens cost on each model from a moment on — a call keeps the price it was made at. BUDGETS: a hard limit per scope (the app, each person, a model, any scope a call names) per day, month or in total, reserved before a call leaves, so calls arriving together cannot all take the last of it. Only the kit writes any of them.

<!-- id: agent-llm-spend · area: agent · stability: preview · html: https://osysharp.com/reference/agent/llm-spend/ -->

## Summary        {#summary}
Money in the Llm kit is counted in **millionths of a US dollar**, from the token counts the provider itself reports:

```osy syntax
SetBudget("requester", "", LlmBudgetWindow.Day, 5m);      // each person: five dollars a day
SetModelPrice("Fast", 0.10m, 0.50m);                      // dollars per million tokens, from now on
decimal spent = SpentOn("task:" + task.Id.ToString());    // what one piece of work cost, read from the meter
```

## Signature      {#signature}
```osy syntax
void    SetBudget(string scope, string subject, LlmBudgetWindow window, decimal dollars,
                  LlmOnLimit onLimit = LlmOnLimit.Stop, decimal warnAt = 0.8m)       // needs ManagesModelSpend
bool    RemoveBudget(string scope, string subject, LlmBudgetWindow window)           // needs ManagesModelSpend
void    SetModelPrice(string model, decimal input, decimal output,
                      decimal? cacheRead = null, decimal? cacheWrite = null, DateTime? from = null)   // needs ManagesModelSpend
decimal SpentIn(string scope, string subject, LlmBudgetWindow window)   // counted against a budget, this period
decimal SpentOn(string scope)                                           // "kind:subject", totalled from the meter
decimal SpentAgainst(string scope, LlmBudgetWindow window = Total)      // counted against a limit the calls carried

new LlmCall { Limits = [ new LlmLimit { Scope = "task:" + id, LimitMicros = 500000 } ] }   // a limit code puts on its own calls
int     ForgetModelCallsBefore(DateTime before)                         // needs ManagesModelSpend
int     ForgetModelCallRequester(Guid requester, DateTime? before = null)   // needs ManagesModelSpend
bool    ModelCallsInFlightFor(Guid requester)

enum LlmBudgetWindow { Day, Month, Total }       enum LlmOnLimit { Stop, StepDown }
```

## Description    {#description}

### What is a scope?   {#scopes}
What a call is charged to: a **kind** and a **subject**. Every call is charged to four without being asked —
`app` (no subject), `requester` (the signed-in person, or the actor the code runs as), `model` (the model's name) —
and to whatever its caller names in `LlmCall.Scopes`, written `kind:subject`: `task:8f1c…`, `agent:Screener`,
`lot:A1`. A budget is set on a scope; a total is read for one.

### What is a call made in a workflow charged to?   {#runs}
To the workflow and to that run of it, with nothing said at the call: `workflow:Onboarding` and
`run.Onboarding:8f1c…` ([Workflow.Current (which run is this code in?)](https://osysharp.com/reference/workflow/run-facts/)). So what a process costs is counted as it happens, a budget can hold
every run of a workflow together or each run on its own, and what one run cost is read back:

```osy syntax
SetBudget("workflow", "Onboarding", LlmBudgetWindow.Month, 200);       // every run of it together, a month
SetBudget(RunKind("Onboarding"), "", LlmBudgetWindow.Total, 2);        // each run of it, two dollars in all
var cost = SpentOn(RunScope("Onboarding", run));                       // what one run has cost
```

`RunScopes()` answers the two scopes where it is called (none outside a run); `WorkflowScope(workflow)`,
`RunScope(workflow, run)` and `RunKind(workflow)` spell them. A call that names what it is charged to itself says
`ChargedToItsRun = false` — code that goes on, in a run of its own, with work that was started for another: the
Agents kit does, and names the run the work was given in ([Agents in business processes — an agent's work as a step of a workflow, told on the run (Agents.Processes kit)](https://osysharp.com/reference/agent/processes/)).

### How do I set a limit?   {#budgets}
`SetBudget(scope, subject, window, dollars)`. An empty subject is a budget for **every** subject of that kind, each
with an allowance of its own — one row gives each person five dollars a day. A budget naming a subject replaces that
one for it. Windows are a UTC day, a UTC month, or `Total`, which never starts over.

A call's room is **reserved before it leaves**: the most it could cost — its prompt, and the longest answer it may
give, at the model's price — against every budget it falls under, all or nothing. When the provider answers, the
reservation is settled at what it actually used; when it fails, the room comes back. Calls that arrive together pass
the reservation one at a time, so they cannot both take the last of a budget.

### What happens at the limit?   {#on-limit}
The call is refused before anything is sent: `LlmBudgetExceeded`, carrying the budget's `Scope`, `Subject`, `Window`
and `LeftMicros`. That is `LlmOnLimit.Stop`, the default.

With `LlmOnLimit.StepDown` — on the budget, or on the call (`LlmCall.OnLimit`) — the router is asked again with what
is left as the call's ceiling (`LlmTaskFacts.MaxCostMicros`), and a cheaper model it admits serves. When it admits
none, the call stops as above. Stepping down never goes below a router's own quality floor: that is the router's to
decide.

A single call can carry a ceiling of its own: `LlmCall.MaxCostMicros`.

### Can code hold its own calls to a limit?   {#limits}
Yes — without anybody setting a budget. A call carries the limit itself:

```osy syntax
await Llm.Complete(request, new LlmCall {
  Limits = [ new LlmLimit { Scope = "job:" + job.Id, LimitMicros = 500000 } ],     // half a dollar for this job, in all
});
```

Every call that carries a limit for the same scope is counted against it, and each one's room is reserved before it
leaves, exactly as against a budget — so twelve at once cannot overrun it together. A call that does not fit is
refused with the same `LlmBudgetExceeded` (its `Scope` and `Subject` are the scope's two halves), or steps down when
the limit says `OnLimit = LlmOnLimit.StepDown`. The scope is one the call is charged to, like any in `Scopes`.

A limit binds **only the calls that carry it**: code holds its own work to a limit, and can never hold anybody
else's — a call that names no limit for the scope is not counted and not refused. Budgets set with `SetBudget` apply
on top, to every call. `SpentAgainst("job:" + job.Id)` reads what the calls have used of it — the figure as it is
stored now, so asked before a call and again after it, the second answer includes the call.

### What is on the meter?   {#meter}
One `ModelCall` row per attempt, written once and never changed:

| | |
|---|---|
| `Call`, `Attempt` | the call the attempt belongs to, and which try it was |
| `Model`, `ProviderModel`, `FallbackFrom` | the app's name, the provider's id, and the model that failed before it |
| `Outcome` | `Served`, `Failed` (the provider), `OverBudget` or `Busy` (nothing was sent) |
| `RouteRule`, `Facts`, `Rejected` | the router's rule, the facts it decided from, and each model it dropped with the reason |
| `Requester`, `Scopes`, `Purpose` | who asked, what it is charged to, what it was for |
| `InputTokens`, `OutputTokens`, `CacheReadTokens`, `CacheWriteTokens` | the provider's own counts |
| `CostMicros`, `Price` | the cost, and the price-book row it was priced from (null: the model's declared price) |
| `DurationMs`, `Problem` | how long it took, and what the provider or the budget said when it did not serve |

The meter never holds a prompt or an answer. Who reads it is `ReadsModelUsage` — one answer for the whole app — and,
a call at a time, `ReadsModelUsageOf(scopes)`, asked of that call's own `Scopes`: how an app of many organisations
lets each one's auditor read its own calls and no other's ([[agent-roles#tenants]]).

### How is a call priced?   {#prices}
`tokens × dollars-per-million` for each of the four counts, rounded up to a millionth of a dollar. The price is the
latest `ModelPrice` row for the model that has started (`SetModelPrice`), else the `LlmPrice` declared on the model in
`app.Llm`. A call keeps pointing at the row that priced it, so a new price never rewrites what was spent. Cached reads
and writes cost what input does unless the price says otherwise.

### Who can change any of this?   {#who-writes}
Only the kit's own functions. `ModelCall`, `ModelPrice`, `LlmBudget` and what each budget has counted are guarded with
[`deny … except via`](https://osysharp.com/reference/security/except-via/): app code, a page, an import and another package can read what the app's
policies let them, and write none of it. `SetBudget`, `RemoveBudget`, `SetModelPrice`, `ForgetModelCallsBefore`
and `ForgetModelCallRequester` run only for `ManagesModelSpend`, who reads the meter's rows as well.

### Can a person be taken off the meter?   {#forget-a-person}
Yes. A row says who a call was made for (`Requester`, and a `requester:<id>` line among what it was charged to) and
keeps no text of the call. `ForgetModelCallRequester(person)` takes them off every row — the row, the model, the
tokens and the cost stay, charged to the same app, model and named scopes — and deletes what was counted against a
budget of their own and a budget that names them. With `before`, only what is older than that moment. It is refused
while a call for them is still on its way (`ModelCallsInFlightFor`). An app that uses `Osysharp.Privacy` writes none
of this: [Privacy for the agent family — agents, the model meter, assignments and work repositories under a person's request](https://osysharp.com/reference/agent/privacy/) does it when the person asks, and when the period ends.

Anyone who may call a model can read the budgets and what each has counted — the call path reads them as the caller.

### Can my app hear about a budget?   {#events}
Two events ([[Event] — what happens to a record, declared once](https://osysharp.com/reference/function/event/), heard with [[function-on|`[On]`]]): `LlmBudgetNearlySpent` once per period when a
budget passes its `warnAt` share, and `LlmBudgetSpent` each time a call is refused for it.

## Examples       {#examples}
```osy title="a daily budget, a price change and a scope's total" test app=agent-llm-spend
app Studio {
  use Osysharp.Llm;
  model "model/**/*.osy";
}

using Osysharp.Llm;

app.Llm = new LlmSetup {
  Models = [ new LlmModel { Name = "Writer", Client = new ScriptedModel { Name = "Writer" }, Model = "scripted-writer",
                            Price = new LlmPrice { Input = 1, Output = 5 }, MaxOutputTokens = 1000 } ],
};

// A small studio: everybody may read the meter and manage the spend.
policy ReadsModelUsage   => true;
policy ManagesModelSpend => true;

entity Heard {
  [MaxLength(200)] string What = "";
  security { allow read, create when IsAuthenticated || IsAnonymous; }
}
[On] void BudgetRanShort(LlmBudgetSpent e) { new Heard { What = $"{e.Scope} budget spent" }; }

/// A caption for one shoot — charged to the shoot.
string Caption(string shoot, string about) {
  return (await Llm.Complete(RequestOf("Caption: " + about), new LlmCall { Scopes = ["shoot:" + shoot] })).Text();
}
```

```osy title="reserved before the call leaves, refused when it does not fit, priced at the price in force" run app=agent-llm-spend
using Osysharp.Llm;

[Test]
void A_call_that_does_not_fit_is_refused_before_anything_is_sent() {
  // Each call reserves about 5000 millionths (up to 1000 tokens of answer at $5 a million) and then costs 2000.
  ScriptedAnswer.Stub((model, request) => Scripted.Text("Golden hour on the pier.", 1000, 200));
  SetBudget("shoot", "pier", LlmBudgetWindow.Total, 0.008m);          // 8000 millionths for this shoot
  UnitOfWork.Commit();

  Caption("pier", "two people at sunset");                           // 2000 spent
  Caption("pier", "the same, wider");                                // 2000 + ~5000 still fits: 4000 spent
  var refused = Assert.Throws<LlmBudgetExceeded>(() => Caption("pier", "one more"));
  Assert.Equal("shoot", refused.Scope);
  Assert.Equal("pier", refused.Subject);
  Assert.Equal(4000, refused.LeftMicros);

  Assert.Equal(0.004m, SpentIn("shoot", "pier", LlmBudgetWindow.Total));
  Assert.Equal(2, ModelCall.Count(c => c.Outcome == ModelCallOutcome.Served));
  Assert.Equal(1, ModelCall.Count(c => c.Outcome == ModelCallOutcome.OverBudget));
  // Another shoot has its own room.
  Caption("harbour", "boats at dawn");

  Background.Settle();
  Assert.Equal("shoot budget spent", Heard.Single().What);
}

[Test]
void A_new_price_applies_to_the_calls_after_it() {
  ScriptedAnswer.Stub((model, request) => Scripted.Text("Golden hour on the pier.", 1000, 200));
  Caption("pier", "two people at sunset");
  SetModelPrice("Writer", 2, 10);
  UnitOfWork.Commit();
  Caption("pier", "the same, wider");

  var costs = ModelCall.OrderBy(c => c.At).ToList().Select(c => c.CostMicros).ToList();
  Assert.Equal(2000, costs[0]);
  Assert.Equal(4000, costs[1]);
  Assert.Equal(0.006m, SpentOn("shoot:pier"));
}
```

```osy title="a limit the code puts on its own calls" run app=agent-llm-spend
using Osysharp.Llm;

/// A caption for one shoot, with at most `limitMicros` spent on that shoot's captions in all.
string CaptionWithin(string shoot, long limitMicros, string about) {
  return (await Llm.Complete(RequestOf("Caption: " + about), new LlmCall {
    Limits = [ new LlmLimit { Scope = "shoot:" + shoot, LimitMicros = limitMicros } ],
  })).Text();
}

[Test]
void A_limit_a_call_carries_is_reserved_against_like_a_budget() {
  ScriptedAnswer.Stub((model, request) => Scripted.Text("Golden hour on the pier.", 1000, 200));

  CaptionWithin("pier", 8000, "two people at sunset");               // 2000 spent
  CaptionWithin("pier", 8000, "the same, wider");                    // 2000 + ~5000 still fits: 4000 spent
  var refused = Assert.Throws<LlmBudgetExceeded>(() => CaptionWithin("pier", 8000, "one more"));
  Assert.Equal("shoot", refused.Scope);
  Assert.Equal("pier", refused.Subject);
  Assert.Equal(0.004m, SpentAgainst("shoot:pier"));

  // It is nobody else's limit: a call that carries none is served.
  Caption("pier", "a fourth, with no limit of its own");
}
```

## See also       {#see-also}
- [The Llm kit — call a model, and know what each call cost](https://osysharp.com/reference/agent/llm-calls/) — the call path these rows are written by
- [Testing model calls without a model — ScriptedModel (Llm kit)](https://osysharp.com/reference/agent/llm-scripted-model/) — the scripted model the examples use
- [deny … except via](https://osysharp.com/reference/security/except-via/) — a row only one function may write
- [[CommitsAlone] — a function whose writes land by themselves](https://osysharp.com/reference/function/commits-alone/) — how a meter row outlives the failure of the work around it