# Testing model calls without a model — ScriptedModel (Llm kit)

> The Llm kit ships a provider whose answers a test writes. Name a `ScriptedModel` where a real provider would go, and answer for it with `ScriptedAnswer.Stub`: the script sees the model's name and the whole request, and returns a response with the token counts it says were used. Everything around the provider is real — routing, budgets, the meter — so a test asserts what a call cost the way the app reads it, and no test ever needs a key.

<!-- id: agent-llm-scripted-model · area: agent · stability: preview · html: https://osysharp.com/reference/agent/llm-scripted-model/ -->

## Summary        {#summary}
```osy syntax
// where the app's models are named — in a test app, or in a kit's own test project
new LlmModel { Name = "Fast", Client = new ScriptedModel { Name = "Fast" }, Model = "scripted-fast" }

// in a test
ScriptedAnswer.Stub((model, request) => Scripted.Text("A short summary.", 1000, 200));
```

## Signature      {#signature}
```osy syntax
class ScriptedModel : ILlmClient, IDecisionModel { string Name; }

LlmResponse ScriptedAnswer(string model, LlmRequest request)            // what a scripted model answers — stub it
LlmDecision ScriptedDecision(string model, LlmDecisionRequest request)  // what a scripted decision model decides — stub it

LlmResponse Scripted.Text(string text, int inputTokens = 100, int outputTokens = 20)
LlmResponse Scripted.ToolUse(string tool, string inputJson, string id = "call-1", int inputTokens = 100, int outputTokens = 20)
LlmDecision Scripted.Decision(string valueJson, decimal confidence, int inputTokens = 100)
string      Scripted.Asked(LlmRequest request)                           // the text of the last user message

bool GenerationIsOn()                                                    // the operator's AI switch — stub it
bool EveryModelIsScripted()                                              // false — stub it to script a real app's models
```

## Description    {#description}

### What is real, and what is scripted?   {#what-is-real}
Only the provider. A call still passes the operator's switch, is routed by the app's router, reserves against its
budgets, falls back when the script throws, and writes its meter rows at the app's prices. So the thing a test
asserts — who served, what it cost, what was refused — is the thing the app does.

### How does a script answer?   {#script}
[Stub](https://osysharp.com/reference/testing/function-stubs/) `ScriptedAnswer` with a lambda taking the model's `Name` and the `LlmRequest`. It
returns an `LlmResponse` — `Scripted.Text` and `Scripted.ToolUse` build one — whose token counts are whatever the
script says were used, which is what the meter then prices. A script can answer differently per model
(`model == "Small"`), per turn (`request.Messages.Count`), or from what was asked (`Scripted.Asked(request)`).

Unscripted, a `ScriptedModel` fails the way a provider does, with a message saying which stub is missing — so a test
that forgot to answer is told so.

### My app names a real provider. How do its tests use the script?   {#real-app}
Say so in the test: `EveryModelIsScripted.Stub(() => true);`. Every model of the app — whatever its `Client` is, an
`AnthropicClient` with a key the test does not have — is then answered by `ScriptedAnswer` under the model's own
`Name` (and `ScriptedDecision`, for a `Decider`). Nothing goes out; the router, the budgets and the meter at the
app's prices are as real as ever.

```osy syntax
// app.osy — what the app runs on
app.Llm = new LlmSetup { Models = [ new LlmModel { Name = "Reader", Client = new AnthropicClient { ApiKey = Secret.Anthropic },
                                                   Model = "claude-haiku-4-5" } ] };

// a test
EveryModelIsScripted.Stub(() => true);
ScriptedAnswer.Stub((model, request) => Scripted.Text("Read by " + model));   // model == "Reader"
```

Outside a test nothing answers it but `false`: a stub is a test's alone.

### How do I play a provider that is down?   {#outage}
Throw `ExternalServiceException` from the script. The call moves to the router's next model, exactly as it would in
production, and the failed attempt is on the meter.

### And the operator's switch?   {#switch}
`GenerationIsOn.Stub(() => false)` — every call is then refused with `LlmSwitchedOff` before anything is sent.

### A streamed call?   {#stream}
`Llm.Stream` over a `ScriptedModel` plays the scripted answer as events: its text as `TextDelta`, a tool call as its
three events, and the closing `MessageComplete` with the script's counts.

## Examples       {#examples}
```osy title="a script per model, an outage, and the switch" run app=agent-llm-scripted-model
app Bakery {
  use Osysharp.Llm;
  model "model/**/*.osy";
}

using Osysharp.Llm;

app.Llm = new LlmSetup {
  Models = [
    new LlmModel { Name = "Small", Client = new ScriptedModel { Name = "Small" }, Model = "scripted-small",
                   Price = new LlmPrice { Input = 1, Output = 5 } },
    new LlmModel { Name = "Large", Client = new ScriptedModel { Name = "Large" }, Model = "scripted-large",
                   Price = new LlmPrice { Input = 10, Output = 50 } },
  ],
  Router = new SmallThenLarge(),
};

public class SmallThenLarge : ILlmRouter {
  public LlmRoutingDecision Route(LlmTaskFacts facts) {
    return new LlmRoutingDecision { RuleId = "bakery:1", Ordered = ["Small", "Large"] };
  }
}

policy ReadsModelUsage => true;

string Describe(string loaf) { return await Llm.Complete("Describe this loaf for the shop window: " + loaf); }

[Test]
void The_script_answers_and_its_counts_are_what_the_meter_prices() {
  ScriptedAnswer.Stub((model, request) => Scripted.Text("Dark rye, baked this morning.", 400, 100));
  Assert.Equal("Dark rye, baked this morning.", Describe("rye"));
  Assert.Equal(900, ModelCall.Single().CostMicros);         // 400 at $1 a million and 100 at $5
}

[Test]
void A_script_that_throws_is_a_provider_that_is_down() {
  ScriptedAnswer.Stub((model, request) => {
    if (model == "Small") { throw new ExternalServiceException("scripted-small is down"); }
    return Scripted.Text("from " + model + ": " + Scripted.Asked(request));
  });
  Assert.Equal("from Large: Describe this loaf for the shop window: rye", Describe("rye"));
  Assert.Equal(1, ModelCall.Count(c => c.Outcome == ModelCallOutcome.Failed && c.Model == "Small"));
}

[Test]
void An_unscripted_model_says_which_stub_is_missing() {
  var fault = Assert.Throws<ExternalServiceException>(() => Describe("rye"));
  Assert.Contains("ScriptedAnswer.Stub", fault.Message);
}

[Test]
void The_switch_is_a_stub_too() {
  GenerationIsOn.Stub(() => false);
  Assert.Throws<LlmSwitchedOff>(() => Describe("rye"));
}
```

## See also       {#see-also}
- [The Llm kit — call a model, and know what each call cost](https://osysharp.com/reference/agent/llm-calls/) — the call path a scripted model stands in
- [Model spend — budgets, the meter and the price book (Llm kit)](https://osysharp.com/reference/agent/llm-spend/) — asserting budgets and costs
- [Typed answers from a model — Llm.Extract<T> and Llm.Decide<T> (Llm kit)](https://osysharp.com/reference/agent/llm-typed-answers/) — scripting `Llm.Extract<T>` and `Llm.Decide<T>`
- [Stubbing your own functions in a test (fault injection)](https://osysharp.com/reference/testing/function-stubs/) — `F.Stub(…)` itself