Why Osy# · Chapter 06 · Testing
An agent can prove its code works. Not say so.
Code written by an agent has to be proven, not vouched for, and a proof is only as good as what it ran against. In Osy# a test runs the application: your functions over your entities, under the rules you declared, as a person the test names, from a starting state the test chose. What it exercised is the program, so what it proved is the program.
[Test(TwoPeopleThreeReports)] [runas(Anna)] void anna_sees_her_two_reports() { Assert.Equal(2, MyReports().Count); }
01
A test runs the application
The application from the last chapter: a User, an ExpenseReport that each employee may read only their own of, and MyReports(), which returns what its caller may see. A test of it is three declarations in one file.
1[TestFixture] void TwoPeopleThreeReports() { var anna = new User { Name = "Anna" }; var otto = new User { Name = "Otto" }; new ExpenseReport { Title = "Lisbon, March", Employee = anna }; new ExpenseReport { Title = "Berlin, April", Employee = anna }; new ExpenseReport { Title = "Oslo, May", Employee = otto }; } 2principal Anna => User.Single(u => u.Name == "Anna"); principal Otto => User.Single(u => u.Name == "Otto"); [Test(TwoPeopleThreeReports)] 3[runas(Anna)] void anna_sees_her_two_reports() { 4 Assert.Equal(2, MyReports().Count); }
The fixture establishes the world. It creates rows through the application's own entities, with the rules off, because it is scaffolding: setting up a scenario never means loosening a rule.
A principal names somebody in that world. It is a query that finds a row the fixture made. It binds; it never creates.
The test runs the application as her. A test body on its own is an anonymous caller with the rules on; under runas, every read in it is Anna's.
An ordinary call to the application's own function, and the answer Anna gets: two rows, because the rule on ExpenseReport is where Employee == user.
The fixture establishes the world. The principal names somebody in it. The test runs the application as them. Everything on this page is that sentence with more of the application in scope.
02
A fixture is a starting state, and every test gets its own copy
A fixture seeds once. Every test that names it starts from a private copy of that state, so what one test writes, another never reads, in any order and at the same time. A test can create rows, break rules and leave the wreckage where it fell.
$ osy test Starting a local platform for this project… ✓ a_note_is_invisible_to_everyone_but_its_owner (908ms) ✓ an_owner_still_sees_her_own_note (967ms) ✓ a_note_planted_for_somebody_else_is_not_even_EXPRESSIBLE (967ms) ✓ an_ITEM_is_invisible_when_its_list_is (627ms) ✓ an_item_cannot_be_added_to_somebody_elses_list (1103ms) … 16 more ✓ 21 passed. ⚠ the app's test set CHANGED since the last run — 24 declared then, 21 now: gone: shot_recall_lists, shot_recall_note, shot_recall_notes A count that holds still over a set that moved reads as "still green" while coverage moves.
Real output. Each of the 21 ran against its own copy of one fixture's data. The warning at the end is the runner noticing that the set of tests moved while the count did not.
03
A rule is proven by running as the people it governs
Chapter 05 put a rule inside every query. The rule on ExpenseReport is the one the test in §1 ran, and a rule has two sides: the person it admits and the people it excludes. Recall is a personal app, and its rule is that two people using one deployment never see a trace of each other.
[Test(TwoPeople)] 1[runas(Bo)] void a_note_is_invisible_to_everyone_but_its_owner() { 2 Assert.Null(Note.FirstOrDefault(n => n.Title == "Ana's private note")); 3 // …and not by title alone: a listing must not leak it either. Assert.Equal(1, Note.Where(n => n.Title != "").Take(50).ToList().Count); }
Run as the other person. The rule is inside the query, so Ana's note is not hidden from Bo after the fact: it is not selected.
By name: the row is not there for him.
…and by listing, which is the leak a lookup by name would miss.
The positive half. A rule that denied everyone passes every refusal test ever written and leaves the app unusable. The owner's test is what proves the rule is a rule and not a wall.
04
The same person reaches the screen
So far the test has established a state, named a person and run the application as them. Ui.SignInAs signs a browser in as that same person, and the test goes on from the page: Ui.Visit opens a route, Ui.Fill and Ui.Click do what she would do, and the same Assert.* verbs ask the questions.
void a_message_you_send_appears_in_the_transcript_it_arrived_on() { Guid id = Guid.Empty; 1 runas (Ada) { id = CreateChannel("design", "how things look").Id; } 2 Ui.SignInAs(Ada); Ui.Visit("/c/" + id); Assert.Visible("Nothing here yet."); 3 Ui.Fill("Message", "morning — pushing the new rail today"); Ui.Click("Send"); Assert.Visible("morning — pushing the new rail today"); Assert.Visible("ada"); 4 Assert.Hidden("Nothing here yet."); }
The scene is set through the application's own functions, as Ada, so the rows the test starts from were made the way the app makes them.
The same principal, now in the browser. From here the page is hers, and every read behind it is her read.
A control is found by its label, the way a person finds it.
The empty state is gone: the assertion that catches a list which appends without clearing.
The same principal and the same application continue through the screen. The test that checks a rule and the test that fills in a form are one kind of thing, in one file.
05
When appearance matters, the same test runs in a real browser
Every assertion so far is about text or state, and a screen can satisfy all of them and still be unusable. osy test renders the page without a compositor, which is cheap and enough for what a person can read and press. osy test --pixels runs the same file in a real browser, where geometry can be judged.
A checkout page. A promo ribbon ships, positioned over the card. The test is three assertions long, and it is the test you would have written.
void the_confirm_button_is_there_and_a_person_can_press_it() { Ui.Visit("/"); Assert.Visible("Confirm"); // text: on the page Assert.Enabled("Confirm"); // state: not disabled Assert.Clickable("Confirm"); // geometry: a press actually reaches it }
$ osy test │ Assert.Clickable(…) — NOT CHECKED: this run renders in happy-dom, which has no compositor — every box it reports is 0×0, so nothing here can judge what covers what. Run the same tests with `osy test --pixels` to check them in a real browser. ✓ the_confirm_button_is_there_and_a_person_can_press_it (1361ms) ✓ 1 passed.
Green, and honest about it. The claim this renderer could not judge is named in the run as NOT CHECKED, which is a different answer from passed.
$ osy test --pixels ✗ the_confirm_button_is_there_and_a_person_can_press_it (2089ms) — at line 7:3 Assertion failed: Assert.Clickable(…) failed: a click at 'Confirm's own centre (640, 140) lands on <div> "half price today" instead — something is covering it. That is what a person pressing it would hit, so the control is on screen and unusable. ✗ 0 passed, 1 failed. 📷 1 screenshot(s) — one per UI test, with that test's own data — in osy-shots/
Same file, same three assertions, real Chromium. The button renders, is enabled and is correctly labelled, and nobody can press it.
A pixel run also photographs every page it drove, with that test's own data, into
osy-shots/, and prints the path. An agent that wrote the page can open the PNG and look at
what it built, which is a different kind of evidence from a passing assertion: an assertion checks the
thing you thought to check, and a picture shows the thing you did not.
📷 3 screenshot(s) — one per UI test, with that test's own data — in /Users/you/projects/shop/osy-shots
Measured: an agent read this line, listed the directory, opened one of the PNGs and reported what it saw — "labeled fields, the accent color on the primary action, a status badge, and a working switch". Nobody told it to.
Clickability is one geometry question. The browser tier also answers whether a label fits
its box, whether a card stays inside its column, and where things sit relative to each other; the whole
set, and what the tier needs installed, is in the
testing reference. The model is one file: osy
test for behaviour, --pixels when geometry matters, and a screenshot when you want
to look.
06
A test can move time
The workflows of chapters 03 and 04 wait: an approval that somebody claimed has eight hours before it escalates, and its assignee is nudged halfway. A test does not wait with them. TestClock.Advance moves the application's clock, and every place the application reads the time reads the new one: a function body, the instant bound into a query, and the browser. It is test-only; the compiler refuses it in application code.
void a_claimed_slot_left_to_rot_escalates() { var acct = Account.Single(a => a.Owner.Name == "Otto"); var bea = User.Single(u => u.Name == "Bea"); 1 var po = new PurchaseOrder { Title = "Forklift", Total = 80000, Requester = bea, Account = acct, Category = PoCategory.Capex }; Workflow.Settle(po); PoApproval.RaiseSubmit(po); 2 runas (Lena) { PoApproval.For(po).Legal.Claim(); } 3 TestClock.Advance(TimeSpan.FromHours(9)); // past Finished.Within = 8h Workflow.Settle(po); 4 Assert.Equal(PoStatus.Escalated, po.Status); }
A purchase order from Bea. Its workflow gives a claimed approval eight hours (Finished { Within = TimeSpan.FromHours(8); … }) and goes to Escalated when they run out.
Lena claims the legal approval, which starts the eight-hour clock.
Nine hours pass. Nothing waits: the run's clock moves, and with it every clock the application reads. Workflow.Settle then runs whatever the workflow has due, the way the platform's own sweep would.
The application is now in the state it would be in by tonight, and the test asks about it in the ordinary way.
Time is part of the state a test controls, like the rows the fixture made and the person the test runs as.
What happens when you change it
Testing changes, and what they cost
Each of these is a thing you would do on a Tuesday. The verdict is the compiler's, not a convention's.
| You do this | Verdict | Because |
|---|---|---|
| Rename an entity field a test reads | refused | The test is Osy# against your model, so a stale name is a compile error, found before anything runs. |
| Tighten a security rule | compiles | The tests that run as each side of it say which side moved. The owner's test says whether you went too far, which is the failure a refusal test cannot see. |
| Add a layout assertion to an existing test | compiles | It runs in both tiers. Under osy test it reports NOT CHECKED; under --pixels it is judged. |
| Read, in one test, a row another test wrote | compiles | …and the row is not there. Each test starts from its own copy of the fixture's state, so there is nothing another test wrote to read, and no order to get right. |