Typed answers from a model — Llm.Extract<T> and Llm.Decide<T> (Llm kit)

Summary#

A type is a schema, so a model can be asked for one:

var invoice = await Llm.Extract<Invoice>(scannedText);                       // an Invoice
var d = await Llm.Decide<Urgency>(message, "How urgent is this for them?");  // a Decision<Urgency>
if (d.Value == Urgency.High && d.IsAtLeast(0.8m)) { Escalate(message); }

Both take the road every model call does ([[agent-llm-calls#the-road]]): routed, held to the budgets, metered.

Signature#

Task<T>           Llm.Extract<T>(string text, string instructions = "")
Task<T>           Llm.Extract<T>(string text, string instructions, LlmCall call)
Task<T>           Llm.Extract<T>(LlmContentBlock file, string instructions = "")
Task<T>           Llm.Extract<T>(LlmContentBlock file, string instructions, LlmCall call)
LlmContentBlock   Llm.FileBlock(string mediaType, byte[] bytes)      // a picture or a PDF, for a model to read

Task<Decision<T>> Llm.Decide<T>(string input, string question = "")
Task<Decision<T>> Llm.Decide<T>(string input, string question, LlmCall call)

class Decision<T> { T Value; decimal Confidence; bool Calibrated; List<LlmDecisionChoice> Alternatives;
                    string Model; long CostMicros; bool IsAtLeast(decimal threshold); }

Description#

How does Extract know what to ask for?#

From T itself. The model is sent Json.SchemaOf<T>() — every member, with the /// comment you wrote on it — and told to answer with one JSON document of that schema. The answer is read with JsonSerializer.Deserialize<T>. So the words on your type are the prompt: a member called Total with the comment "what the customer owes, VAT included" is read better than one without.

instructions adds what the type cannot say ("the supplier is the company that SENT it"). The text is marked as material to read, never as instructions to the model.

Can it read a file — a receipt, a scanned invoice?#

Yes: hand it the file instead of text. Llm.FileBlock(mediaType, bytes) makes a picture (image/png, image/jpeg, image/webp, image/gif) or a PDF (application/pdf) into something a model reads, and Extract<T> reads it by T's schema exactly as it reads text:

var fields = await Llm.Extract<ReceiptFields>(
  Llm.FileBlock(line.Receipt.MimeType, File.ReadAllBytes(line.Receipt)),
  "The merchant is the business that issued the receipt.");

Any other kind of file is refused with an ArgumentException before anything is sent. The call is a vision task to the router (LlmTaskFacts.NeedsVision), so a router that keeps a model without vision off it is obeyed. Each provider kit sends the file in its own API's shape — Anthropic as an image or a document block, OpenAI as an image URL or a file part, Gemini as inline data.

What if the model does not answer with the shape?#

A code fence or a sentence around the document is removed. An answer that is still not a JSON document is sent back to the model once, as a step of its own (LlmStep.Fill, Attempt = 1) — which a router may answer with a stronger model. A second miss throws LlmAnswerUnreadable.

What is a typed decision?#

State in, a value of a known shape out, with how sure the model is — and no prose. T is usually an enum (which queue, which category, yes/no/ask), and may be a number or a record. Decision<T> carries:

d.IsAtLeast(0.8m) is the usual gate: act when the model is sure, hand to a person when it is not.

Which model makes a decision?#

The router orders the candidates knowing the call wants a decision (LlmTaskFacts.NeedsDecision). A model declared with a Decider — a class implementing IDecisionModel, for a model built to return typed values with a confidence — is asked directly. A model that only chats is asked the same question through its Client, held to a schema of the value and a confidence: slower, dearer, and Calibrated = false. An app with no decision model at all still decides.

Can I have the JSON instead of the type?#

ExtractJson(text, instructions, schemaJson, call) answers the document as text, for code that holds a schema and no type. DecideWith(request, call) is the decision underneath Llm.Decide<T>.

Examples#

app Lettings {
  use Osysharp.Llm;
  model "model/**/*.osy";
}

using Osysharp.Llm;

app.Llm = new LlmSetup {
  Models = [
    new LlmModel { Name = "Reader", Client = new ScriptedModel { Name = "Reader" }, Model = "scripted-reader",
                   Price = new LlmPrice { Input = 1, Output = 5 } },
    // A model built for decisions: it does not chat.
    new LlmModel { Name = "Sorter", Decider = new ScriptedModel { Name = "Sorter" }, Model = "scripted-sorter",
                   Price = new LlmPrice { Input = 0.05m } },
  ],
  Router = new LettingsRouter(),
};

/// A decision goes to the sorter, with the reader behind it; everything else to the reader.
public class LettingsRouter : ILlmRouter {
  public LlmRoutingDecision Route(LlmTaskFacts facts) {
    var decision = new LlmRoutingDecision { RuleId = "lettings:1" };
    if (facts.NeedsDecision) { decision.Ordered.Add("Sorter"); }
    decision.Ordered.Add("Reader");
    return decision;
  }
}

policy ReadsModelUsage => true;

/// What a lease says.
public class LeaseTerms {
  /// The tenant's full name.
  public string Tenant = "";
  /// The rent for one month, in the lease's currency.
  public decimal MonthlyRent;
  /// The first day of the tenancy.
  public DateOnly? Starts;
}

/// What a tenant's message is about.
public enum MessageKind { Repair, Payment, Notice, Other }

LeaseTerms ReadLease(string text) { return await Llm.Extract<LeaseTerms>(text, "The tenant is the person renting, not the landlord."); }

Decision<MessageKind> Sort(string message) { return await Llm.Decide<MessageKind>(message, "What is this tenant's message about?"); }

using Osysharp.Llm;

[Test]
void A_lease_is_read_as_the_type() {
  ScriptedAnswer.Stub((model, request) =>
    Scripted.Text("{\"Tenant\":\"Ada Lund\",\"MonthlyRent\":9400,\"Starts\":\"2026-11-01\"}"));
  var terms = ReadLease("Lease between Fjord Homes (landlord) and Ada Lund, 9 400 kr a month from 1 November 2026.");
  Assert.Equal("Ada Lund", terms.Tenant);
  Assert.Equal(9400m, terms.MonthlyRent);
  Assert.Equal(new DateOnly(2026, 11, 1), terms.Starts);
}

[Test]
void An_answer_that_is_not_the_shape_is_sent_back_once() {
  ScriptedAnswer.Stub((model, request) => request.Messages.Count == 1
    ? Scripted.Text("The tenant is Ada Lund and she pays 9400.")
    : Scripted.Text("{\"Tenant\":\"Ada Lund\",\"MonthlyRent\":9400,\"Starts\":null}"));
  Assert.Equal("Ada Lund", ReadLease("…").Tenant);
  Assert.Equal(2, ModelCall.Count());
}

[Test]
void A_decision_comes_back_typed_with_a_calibrated_confidence() {
  ScriptedDecision.Stub((model, request) => Scripted.Decision("\"Repair\"", 0.94m));
  var d = Sort("The radiator in the bedroom has been cold since Monday.");
  Assert.Equal(MessageKind.Repair, d.Value);
  Assert.True(d.Calibrated);
  Assert.True(d.IsAtLeast(0.9m));
  Assert.Equal("Sorter", d.Model);
}

[Test]
void With_the_decision_model_down_a_chat_model_decides_and_says_its_confidence_is_its_own() {
  ScriptedDecision.Stub((model, request) => { throw new ExternalServiceException("scripted-sorter is down"); });
  ScriptedAnswer.Stub((model, request) => Scripted.Text("{\"value\":\"Payment\",\"confidence\":0.7}"));
  var d = Sort("I have paid twice this month by mistake.");
  Assert.Equal(MessageKind.Payment, d.Value);
  Assert.Equal(0.7m, d.Confidence);
  Assert.False(d.Calibrated);
  Assert.Equal("Reader", d.Model);
}

See also#

• The Llm kit — call a model, and know what each call cost — the call path, routing and refusals
• Json.SchemaOf<T>() — the JSON Schema of a type — the schema a type is sent as
• JsonSerializer — how the answer is read back
• Testing model calls without a model — ScriptedModel (Llm kit) — the scripted model the examples use