LLM budgets (hard daily limits per organisation, app and user)

Summary#

Beside the runtime switch sits the dial. The platform meters every LLM and embedding call an application makes and enforces hard daily budgets before the call leaves: a call that would exceed the day's limit is refused, not sent. Nobody wakes up to a bill because an agent looped overnight — the eleventh call over the cap is refused, and the application carries on without it.

Three scopes, checked in order, any one of which can refuse:

The budget day starts at midnight in the budget's own time zone (the organisation's, when set), not the server's.

Signature#

admin → org → Budgets
   Daily LLM tokens        ______   (blank = platform default · 0 = blocked)
   Daily embedding tokens  ______
   Daily USD cap           ______   (blank = no cost cap; the token limit is the floor)
   Default per-user tokens ______   (application rows only)

using Osysharp.Llm.Budget;        // and `use Osysharp.Llm.Budget;` in app.osy

entity UserLlmBudget {          // provided by the capability — one row per user, optional
  principal UserId;             // the app-local user this override applies to
  long DailyLlmTokenLimit;      // input + output tokens per budget day; 0 = blocked
}

An application never declares its own organisation or application cap in source: those are the operator's, set outside the app's reach for the same reason the switch is. What an application can do is narrow the per-user limit below the app default, row by row.

Description#

How a call is metered#

Before a model call is sent, the gate reserves the estimated tokens against each scope in one atomic statement — check and increment together, so two concurrent calls cannot both squeeze under a limit. After the response it settles the reservation to the actual tokens. Usage is recorded per scope, per day, per model, so the admin's budgets page shows today's spend against the cap, and the audit trail's LlmCallRecord carries the same numbers per call.

What a refusal looks like#

A refused call throws. The message names the scope, the limit and the usage, and says when the day resets:

Daily LLM token budget for this application (2,000,000 tokens) is exhausted (2,000,113 used); resets at 00:00 Europe/Stockholm.

Over HTTP the platform answers 429; in a workflow or a background handler the refusal is terminal for that call — it is logged and not retried, because a retry storm on an exhausted budget is exactly what a budget exists to prevent. In Osy# it is an ordinary exception: catch it where the application has a sensible way to go on without the model — the same degraded path the switch uses.

try {
  var reply = Screener.Ask(turns);
} catch (Exception e) {
  // budget exhausted, or AI switched off: the report is screened by the rules alone
  report.ScreeningSkipped = true;
}

Fail-closed#

If a budget cannot be verified — the admin row cannot be read, the accounting statement fails — the call is refused, with a message that says so rather than one that claims exhaustion. A budget that fails open is not a budget; measured once on this platform (an incident on 2026-06-12) and the gate has refused on doubt since.

Per-user overrides#

The application's default per-user limit comes from the admin. An app that wants to give one person more, or block one, writes a UserLlmBudget row — an ordinary entity under the app's own security { }, so who may set it is the app's rule:

using Osysharp.Llm.Budget;

[Role] enum Role { Authenticator, Admin }
entity RoleGrant { [Required] User User; [Required] Role Role;
  security { allow read when IsAuthenticated; allow create when IsAuthenticator; } }
policy IsAuthenticator => RoleGrant.Any(g => g.User == user && g.Role == Role.Authenticator);
policy IsAdmin         => RoleGrant.Any(g => g.User == user && g.Role == Role.Admin);

[Principal] entity User {
  [Required, Unique, MaxLength(200)] string Email;
  [MaxLength(200)] string PasswordHash;
  security {
    allow read when IsAuthenticated;
    allow read, create when IsAuthenticator;
    deny read PasswordHash when !IsAuthenticator;
  }
}

// The capability's entity, reopened only to say who may write it: the app's Admin.
partial entity UserLlmBudget { security { allow read, create, update when IsAdmin; } }

void CapUser(User u, long tokensPerDay) {
  var b = UserLlmBudget.Where(x => x.UserId == u).FirstOrDefault();
  if (b == null) { new UserLlmBudget { UserId = u, DailyLlmTokenLimit = tokensPerDay }; }
  else { b.DailyLlmTokenLimit = tokensPerDay; }
}

[AuthMethod] string Login(string email, string password) {
  var u = User.Where(x => x.Email == email).FirstOrDefault();
  if (u == null) { Security.VerifyPassword(password); return ""; }
  if (Security.VerifyPassword(password, u.PasswordHash)) { return Security.IssueJwt(u.Id, u.Email); }
  return "";
}
app.Auth = new PasswordAuth { LoginField = Email, PasswordField = PasswordHash };
app.AuthBootstrap = new AuthBootstrap { Login = Login, Role = Role.Authenticator };

A user with no row gets the application's default; a row with 0 blocks that user's model calls entirely while everyone else's continue.

What it answers for procurement#

The switch answers can it be turned off. The budget answers what is the most it can ever cost us in a day — per organisation, per application, per person, enforced by the platform rather than promised by the application, and visible on one admin page against today's usage.

Examples#

An organisation caps itself at a dollar a day and one application at a fifth of that:

admin → org "Acme" → Budgets           daily USD cap 1.00 · zone Europe/Stockholm
admin → org "Acme" → app "Desk" → Budgets   daily USD cap 0.20 · default per-user tokens 50,000

The twentieth cent Desk would spend is refused; Acme's other apps keep their share; the person the app's admin capped at 0 with CapUser(u, 0) is refused before the app's cap is ever consulted.

See also#

• turning AI off (the runtime switch) — the switch this dial sits beside: off in seconds, from the same control plane
• what a task cost, and what it did (task.Calls) — where a task's own calls and their cost are read back in the app
• audit read access (app.Audit) — LlmCallRecord, the per-call trail with the same numbers and its own retention
• Agents (calling a model like anything else you declared) — the rest of the agent and model surface