Osysharp.Agents.Testing — testing an agent without a model

`Osysharp.Agents.Testing` scripts what the model answers, turn by turn, and reads back what the agent then did. Only the model is scripted: the loop, the rights, the tools, the budget, the meter and the record are the real ones.

You are here: Building with agents › The kits › Agents.Testing

Summary#

Osysharp.Agents.Testing scripts what the model answers, turn by turn, and reads back what the agent then did. Only the model is scripted: the loop, the rights, the tools, the budget, the meter and the record are the real ones.

A test says: on the first turn the model asks for this tool, on the second it hands back this result. Then it asserts which tools ran and in what order, what was proposed, what the work cost. It costs nothing and gives the same answer on every machine.


Package	`Osysharp.Agents.Testing`, version 1.3.0
You add it with	`tests use Osysharp.Agents.Testing;` in the app's manifest — it is for tests, not for the app people run.
It sits on	[[agent-kit-agents\|Agents]], [[agent-kit-llm\|Llm]]
Its own test app	A lighthouse whose keeper has a mate that reads the day's log and files a report — `kits/agents-testing/tests`

Description#

What do you get without writing anything?#

Trajectory.Of(task): the tools that ran, in order.
Trajectory.Tree(task): the tree of work handed on, with how each piece ended.
AgentCast: a script for each of several agents at once.
ProcessTrail: what the agents did in one run of a process.

When do you need it?#

In the tests of every app with an agent in it. Add it with tests use, so it is not part of the app people run.

What does it expose?#

The tables below are generated from the kit itself, not written by hand: osy kits Osysharp.Agents.Testing --surface lists each of them with the code of the kit's own test app that uses it, and the tests that show it working.

Types you use

Type	What it is
`AgentCast`	Several agents, each with its own script.
`Says`	One thing a scripted model says on a turn.
`AgentScript`	What the models answer, turn by turn.
`Trajectory`	What a piece of work did, read back from its record — for a test to assert.
`ProcessTrail`	What the agents did in one run of a process, read back from their receipts — for a test that walks a whole process with scripted models and asserts where it went, what was proposed on the way and what it cost.

Functions you call

Function	What it does
`Plays(script)`	The script of every piece of work the agent `T` does.
`PlaysAbout(about, script)`	The script of the pieces of `T`'s work whose job or brief says `about` — for an agent with several under way: a few words of the skill's description ("Count a delivery"), or of what the piece was given ("Crate A-2").
`Answer(model, request)`	What `model` answers to `request` — what a test hands `ScriptedAnswer.Stub`.
`AskedAgent(request)`	The agent a request is put to, by its own name: the request's first words to the model are "You are X."
`FirstAsked(request)`	What a piece of work was given: the text of the request's first message.
`Tool(action, argumentsJson, input, output)`	It asks for one of the app's actions, with these arguments as JSON.
`HandBack(result, input, output)`	It hands back its finished result.
`HandBackJson(resultJson, input, output)`	It hands back this JSON as its result — for a result that is NOT of the shape, or breaks the skill's check.
`Words(text, input, output)`	It answers in words.
`AskPerson(question, context, input, output)`	It asks a person — the one the work is for, or the agent's sponsor.
`Consult(question, context, input, output)`	It asks the model the agent consults.
`HandTo(skill, inputJson, call, input, output)`	It hands a piece of the work on to the agent `T`, for `skill`, with this input as JSON of what the skill takes (`{"id":"…"}` for a skill that takes a row).
`Row(id)`	The input that names one row by its id — what a skill that takes a row is handed.
`Together(asks)`	Several things in one turn: everything each of `asks` asks for, asked for together — how a model hands several pieces of work on at once.
`Of(turns)`	A script whose working model answers these, in order.
`Then(turn)`	One more turn.
`Advises(model, advice, input, output)`	What `model` says when it is consulted (a request that offers it no tools).
`IsDown(model)`	`model` cannot be reached.
`Called(task, action)`	Whether the work ran this action.
`Models(task)`	The model that served each model call of the work, in order — a consult among them.

13 more: osy model in the kit's folder.

Go deeper#

Testing an agent without a model — Says, AgentScript and Trajectory (Agents.Testing kit) — Says, AgentScript, Trajectory, AgentCast, ProcessTrail
Testing model calls without a model — ScriptedModel (Llm kit) — the scripted model underneath