Summary#
The compiler already knows which functions call which: a test calls Price, Price calls Discount, a page at
/cart shows Price. osy test uses that graph two ways.
--affected <base>runs only the tests that can reach a declaration that changed since<base>— a function, a page, a handler, a test. A change the graph cannot place (an entity, the seed, the manifest …) runs every test and names the rule that made it.--coverage [<base>]runs nothing and reports which functions and components the tests reach. Against a base it names the new or changed ones no test reaches, and the ones a test reached on the base and none reaches now.
Signature#
osy test [path] --affected [<base>] [--list] [--json]
osy test [path] --changed <path> [--changed <path> …] [--list] [--json]
osy test [path] --coverage [<base>] [--json]<base> is a git commit (main, HEAD~3: everything since it, uncommitted work included) or a range (a..b). With
no <base>, --affected measures from where your branch left origin/main. Put the project path before the flag:
the word after --affected or --coverage is read as the base.
Description#
What "reaches" means#
A test reaches every declaration its own text names, everything those name, and so on — there is no limit on the distance. A few things count as reaching even though they are not calls by name:
- a page by its route —
Ui.Visit("/cart")reaches the page declared[Route("/cart")]; - a handler by its event — a test that reaches
Ship, which raisesShipped, reaches the[On]handler that hearsShipped; - a method by its name —
order.Total()reaches the class that declaresTotal; - what derives from what is reached — a slot reaches its contributions, a base its derived types;
- the fixtures a test forks from — a change to a
[TestFixture]reaches the tests that name it, and the tests of fixtures built on it.
Names are matched by their simple spelling, so two unrelated functions that share a name both count. That can only select more tests, never fewer. Comment and whitespace edits are not changes.
What runs everything#
Some changes reach code no call graph shows. Any one of these runs every test, and --list names it:
| rule | what changed |
|---|---|
schema | an entity, an enum or an interface — the shape every stored row and every caller has |
seed | a file of the seed role — every branch starts from it |
migration | a migration |
manifest | app.osy, a using or a namespace |
config | osyrin.json, osyrin.lock, .secrets |
setting | an app setting (app.X = …) |
runtime | a policy, a workflow, a principal, a layout, an [OnStart] function, a handler that hears every event |
unplaced | a theme, a store, a translation, an asset — anything else no test reaches by a call — or an edit outside every declaration |
unparsed | a source file that does not parse |
outside | a file outside the project that the project is built from — a kit it uses, or the platform itself |
Prose (.md) and .gitignore reach nothing.
--list — see the selection, run nothing#
$ osy test --affected main --list
--affected main: 3 of 5 test(s) can reach the change.
[reaches] 1 changed declaration(s) reach 3 of 5 test(s)
· tests/shop.test.osy::the_price_is_discounted
← function Discount (changed, model/shop.osy:6)
· tests/shop.test.osy::a_basket_is_priced
← function Discount (changed, model/shop.osy:6)
· tests/shop.test.osy::the_cart_page_shows_the_price
← function Discount (changed, model/shop.osy:6)--list composes with --filter, --file and --test, and works without them too (it then lists what a plain run
would run). It needs no platform: the answer comes from the source alone, in well under a second.
Running the selection#
Without --list, osy test --affected main runs the tests it selected, and the run's last line says it was narrowed
— ran 3 of 5 — FILTERED by --affected main — so a narrowed green is never read as the whole suite passing. A change
that reaches no test is an answer, not an empty pass: the command says so and exits 0 without starting a platform.
--changed <path> is for a tool that already knows the change set. With no base to compare against, every declaration
in a named file counts as changed.
--coverage — what no test reaches#
$ osy test --coverage main
osy test --coverage main: the tests reach 5 of 7 function(s) and component(s) — statically: a declaration counts when a test can reach it.
✗ 1 new or changed, and no test reaches it:
function Refund (added, model/shop.osy:18)
✗ 1 reached on the base, and no test reaches it now:
function Ship (model/shop.osy:8) — was reached by shipping_returns_oneCoverage is static: a function counts as covered when some test can reach it through the graph above, whether or
not that test's path actually executes it. So "no test reaches it" is a firm finding, while "covered" means "a test
could get there". With no base, --coverage lists every function and component no test reaches.
For a gate: --json#
--list --json and --coverage --json each print one JSON document. Fields are only ever added, never renamed.
{
"kind": "affected",
"base": "main, uncommitted work included",
"everything": false,
"declared": 5,
"selected": 3,
"reasons": [ { "rule": "reaches", "what": "1 changed declaration(s) reach 3 of 5 test(s)" } ],
"changes": [ { "kind": "function", "name": "Discount", "file": "model/shop.osy", "line": 6, "change": "changed" } ],
"tests": [ { "id": "tests/shop.test.osy::the_price_is_discounted", "name": "the_price_is_discounted",
"file": "tests/shop.test.osy", "because": [ "function Discount (changed, model/shop.osy:6)" ] } ]
}{
"kind": "coverage",
"base": "main, uncommitted work included",
"method": "static",
"declarations": 7,
"covered": 5,
"uncovered": [ { "kind": "function", "name": "Refund", "file": "model/shop.osy", "line": 18, "change": "added" } ],
"lost": [ { "kind": "function", "name": "Ship", "file": "model/shop.osy", "line": 8, "tests": [ "shipping_returns_one" ] } ],
"unreached": [ { "kind": "function", "name": "Refund", "file": "model/shop.osy", "line": 18 } ],
"changes": [ { "kind": "function", "name": "Refund", "file": "model/shop.osy", "line": 18, "change": "added" } ],
"notes": []
}When everything is true, every test is in tests and reasons names each trigger. A gate that wants "new code
must be tested" fails when uncovered is not empty. A gate that wants "coverage must not shrink" fails when lost is
not empty.
Examples#
osy test --affected main --list # what would run, and why — no platform needed
osy test --affected main # run it
osy test --affected HEAD~1 --json # the last commit's tests, as a stream of events
osy test --changed model/orders.osy # a tool that already knows the changed files
osy test --coverage main --json # what this branch added that no test reachesSee also#
Running tests locally — osy test itself, and the other ways to choose what runs (--filter, --file,
--test).
Fixtures: built once, cloned for every test, and built on each other — fixtures, which a test forks from and which --affected follows.