Osy#betaRun it on Osyrin: osyrin.com

Reference / Testing

[PageBudget]: what a page's first paint and live re-run may cost, held by osy test

[Route("/shelves")] [PageBudget(Statements = 12, Rows = 60, Seed = FullStore, As = Clerk)] component Shelves() { … } osy test · osy test --page-budgets

Declare beside a page's [Route] the most SQL statements and rows read its first paint may cost. osy test opens the page in the fixture's world as the principal you name, measures the visit, and fails it when it costs more. A live page can also budget one re-run after a change (LiveStatements, LiveRows, Change). osy test --page-budgets also fails every routed page that declares no budget.

preview1 example compiled by CItestingcostperformanceui

Summary#

A page reads no more than it shows, and says so. [PageBudget] beside a page's [Route] names the most its first paint may cost — SQL statements sent and rows read — and osy test holds it to that on every run:

[Route("/shelves")]
[PageBudget(Statements = 12, Rows = 60, Seed = FullStore, As = Clerk)]
component Shelves() { … }

Signature#

[PageBudget(Statements = <n>, Rows = <n>)]                       // both numbers, always
[PageBudget(Statements = <n>, Rows = <n>,
            Seed = <a [TestFixture]>,                            // the world it is opened in (default: an empty app)
            As = <a declared principal>,                         // who opens it (required unless [AllowAnonymous])
            Visit = "/the/address?tab=2",                        // the address to open (default: its route)
            LiveStatements = <n>, LiveRows = <n>,                // what one live re-run after Change may cost…
            Change = <a test function with no parameters>)]      // …the change it is measured around
osy test                    # every declared budget is held
osy test --page-budgets     # …and a routed page with no budget fails, saying what to write

Description#

A budget is a test the run writes for you. For each page with a [PageBudget], osy test adds a test called page_<Component>_keeps_its_budget. It forks the Seed fixture, signs in as As, takes a Cost.Mark(), opens the page with Ui.Visit and reads Cost.StatementsSince and Cost.RowsReadSince once the page has settled. Over either number, it fails:

  ✗ page_Shelves_keeps_its_budget
    page '/shelves' (component Shelves, model/shelves.osy:3) is over its budget at first paint: 31 SQL statements
    (budget 12) and 2400 rows read (budget 60). Read what the page shows and no more — …

It is the same counter a hand-written Cost.Mark() and Cost.StatementsSince(mark) test reads and a deployed app's Usage tab shows, and it runs on the same fixtures, branches and browser as your other tests — so --filter, --test and --cost work on it as on any. osy test --filter page_Shelves --cost lists the lines that sent the statements.

Seed it at production size. A page that reads every row to count them costs three rows over a three-row fixture and thousands in production. Name a fixture that seeds the volume the page will meet (layer it on your ordinary one with [TestFixture(Floor)], Fixtures: built once, cloned for every test, and built on each other) and set the budget against that.

Name who opens it. As is a principal Name => …; declared in your tests ([runas(Name)] test attribute and principal selectors); the page is measured as that person sees it, through their security. A page that is not [AllowAnonymous] must name As: opened by nobody, it would measure the sign-in refusal, not the page.

A page that takes a row ([Route("/shelves/{shelf}")] component ShelfPage(Shelf shelf)) is opened with a row of that entity the visitor can read in the seed. A parameter that is not a row (an id, a code) cannot be picked: Visit = "/codes/A7" names the address to open, or take the row itself as the parameter.

Requiring a budget on every page is the run's choice, not the page's: osy test --page-budgets adds a failing test for each routed page that declares none, naming the page, its file and line, and the attribute to write. A team that holds every page to a budget runs it as a check step; a declared budget is held either way. It is a flag and not the default so that an app is never turned red by upgrading osy — whether budgets are required is a quality rule of your workflow, and the check step it adds runs osy test --page-budgets. With --affected, a changed page selects its budget test, and with --page-budgets a changed page with none selects the test that says so.

And one live re-run after a change. A live page re-runs its live queries each time a row they read changes, and that is a cost the first paint does not show: a page that counts with .Count() but also keeps every row live looks fine when opened and re-reads the whole table on every write. Name a change the page shows and what one re-run after it may cost:

[PageBudget(Statements = 12, Rows = 60, LiveStatements = 3, LiveRows = 25, Change = AShelfIsAdded,
            Seed = FullStore, As = Clerk)]

Change is a test function with no parameters, in a test file, that writes a row the page's live query reads (void AShelfIsAdded() { new Shelf { Name = "new" }; }). The test runs it once to warm up and once measured before the page is opened — what the change costs ALONE — and after the first-paint check runs it again with the page open, waits with Ui.AwaitChange(<the entity it writes>) until the page has heard it and re-run, and counts the difference. Over either number, it fails:

  ✗ page_Shelves_keeps_its_budget
    page '/shelves' (component Shelves, model/shelves.osy:3) is over its budget for one live re-run after
    AShelfIsAdded: 4 SQL statements (budget 3) and 2003 rows read (budget 25). …

The entity waited on is the one Change writes (a row it creates, or else one it names) that a live member of the page reads; a Change that is not a test function, takes parameters, or writes nothing the page shows live fails the page's test with that sentence. The live re-run is measured on a live QUERY; a live server call's re-ask is spaced by the client and not waited for by Ui.AwaitChange, so it is not what this measures.

Counts, not time. A budget is statements and rows, never milliseconds: the same page gets the same verdict on a laptop, a loaded CI runner and production, and a failure names what to read less of. A time budget is not offered.

The same number in production. A deployed app meters each page's first paint the way this test measures it. The client marks the requests from a navigation to the moment the page settles, which is the moment the test's Ui.Visit returns: nothing on the wire, no navigation landing, no live query or list render owed, no skeleton showing, all quiet for 250 ms. The server counts those requests a second time on the page's first paint line, one call per navigation. A click ends the window before its first request is sent, and nothing marks a live re-run once the window has closed, so both go on the page's own line only. A request of the paint already on the wire when the click comes stays the paint's: it was sent for the page, not for the click. The zone report carries the first-paint line with the page's [PageBudget] beside it, so statements per call can be read against the budget the test holds.

Checked as you write it. osy validate refuses a [PageBudget] on a component with no [Route], a second one on the same page, one missing a number, an argument it does not take, one with no As on a page that needs a sign-in, a Visit that is not an address of the page's route (a query string is fine; a line break is not), a page whose address takes a value that is not a row ({code} on a string code) with no Visit, LiveStatements/LiveRows with no Change, and a Change without both of them. A Seed or As that names nothing in your app or its tests fails the page's test with that sentence.

Examples#

entity Shelf {
  [Required, MaxLength(60)] string Name;
  security { allow read when IsAuthenticated || IsAnonymous; }
}

[Route("/shelves")]
[Render(CSR)]
[AllowAnonymous]
[PageBudget(Statements = 6, Rows = 25, Seed = FullStore)]
component Shelves() {
  live var count = Shelf.Count();
  live var first = Shelf.OrderBy(s => s.Name).Take(20).ToList();
  render {
    Stack {
      Text($"{count} shelves");
      foreach (var s in first) { Text(s.Name); }
    }
  }
}
[TestFixture]
void FullStore() {
  for (var i = 0; i < 2000; i++) { new Shelf { Name = $"Shelf {i}" }; }
}

See also#

Related

Cost.Mark() and Cost.StatementsSince(mark)

What a test's unit of work sent to the database between two points: take a mark, do the thing, read what it cost — in…

Fixtures: built once, cloned for every test, and built on each other

A [TestFixture] is built once per run, and every test that names it runs on its own copy of what it left — rows, files…

[runas(Name)] test attribute and principal selectors

A test runs deny-all as an anonymous principal, so to read or write real data it must act AS a seeded `[Principal]`…

Ui — drive the app's UI from a test

Drive the real UI from a test: navigate to a route, click what a person would click, and assert on what the screen…

Running tests

Runs your app's tests against a throwaway copy of its database, reporting each test as it finishes. Your local source…