Summary#
osy drill asks what happens to a person using your app when something goes wrong — the things a test you wrote
rarely stages, because they are about the world rather than your code. It starts a local platform of its own, compiles
your app into it, signs a real browser in, and goes through every page your app declares:
| drill | what it does | what a pass means |
|---|---|---|
double-submit | presses every control twice, fast, on a slow link — and Enter in a field together with its button | one press's effect, or a second press that was answered explicitly |
concurrent-edit | two sessions open the same row's editor, change a field each (the same one when the editor has only one), and both save | the later write of a field wins and every other field keeps its value — or, where [ConcurrencyCheck] guards the field, the stale save is refused and says so |
restart | holds the server just before each write commit a press makes, kills it there, and starts it again | the rows are all of the press or none of it, durable work finishes, the page recovers, and a press that died was said to have failed |
dependency-down | makes every outside call answer 503, then time out; ends every database connection; takes storage down | the person is told; nothing that was saved is lost — and after the database connections end, the press simply goes through: the platform sends its first statement again on a fresh connection |
Signature#
osy drill [path] [--drill <name>]... [--as <login>]... [--setup <verb>]... [--recipe <file>] [--page <route>]... [--max-controls <n>] [--out <dir>] [--keep]Description#
- It runs on its own platform, under the name
drill-<app>, never on the oneosy launchandosy testuse — so your data and your open pages are left alone. That platform is the only kind that can be made to fail: outside calls, storage, the database connections and the moment before a commit are faults it injects itself, on your computer, for the length of the drill. It is deleted afterwards unless you pass--keep. - It needs a browser. The same one
osy test --pixelsuses:osy browser installadds it once for every project (The browser for screenshots). --as <login>signs the browser in as one of your app's accounts, with no password. A second--asis the other person inconcurrent-edit; with one, it is the same person in two tabs.--setup "<verb>"runs one of your app's ownosyverbs against the drill's platform before any page opens — the run book that puts the app in a state worth drilling:--setup "user add [email protected] --role Owner --password pw123456",--setup "import --as [email protected] --password pw123456". Repeat it; they run in order.--recipe <file>is for an app whose way in is more than an account and a run book — a sign-in with a second step, say. It is a JavaScript module whose default export haslogins(the accounts, in place of--as),seed(ctx)(run once the app is compiled) andsignIn(page, login, ctx)(sign a browser page in as that account).--page /routedrills only that page. Without it, every page with a fixed address is drilled, and the first page each address with a parameter (/case/{id}) is linked to from them, except a sign-in page. Pages a kit brings (its/sign-up, say) are drilled with your own.- What it presses. Each page's own buttons first (up to
--max-controls, 12 by default), then your app's shell — its header, navigation and side bar — once for the whole run, not on every page. A button that opens a dialog is followed in: the dialog's empty fields are filled and its own buttons (Create, Save) are drilled, reached through the button that opened it each time. Every empty field gets a fresh value each time — an address, a password long enough for the usual rules, a line of text — so a form that keeps a repeat as the first one (an invitation to an address already invited) is not mistaken for a press that did nothing. Search boxes are left empty. - What a press cost is counted from the requests your pages make, not from everything the platform writes — its own background work, and the drill's own reads, are left out.
- Each drill is evidence, not opinion. Effects are counted by the server itself (write commits per database) and
by reading your rows through
osy queryas the signed-in person; what the page said is read off the page; a failure is photographed. The report isosy-drill/report.jsonin the project (or--out), with a summary on screen. - The exit code is 0 when nothing failed, 1 when a drill failed, and 2 when the drill could not run.
What each verdict means#
- pass — the app did the right thing, and the line says what that was.
- fail — something a person would lose: a second write from one press, a save that put back a field it did not
change, a stale save a
[ConcurrencyCheck]let through, a half-written press, a failure nobody was told about, rows gone. - warn — whole and safe, but worth a look: a press whose own write committed while its outside call failed and nothing was said (fine if that call is retried later), a press that died with the server and was not said to have.
- skip — the drill found nothing to exercise there, and says why: a page with no editor, a press that touches no outside service.
When concurrent-edit fails#
concurrent-edit does two things: two people open the same editor and both save, and — for a button that changes a
row without opening an editor ("Add one") — the first person presses it and the second presses it from a page that has
not yet shown the first press.
It judges the stored rows, field by field, read before, between and after the two writes. The platform's rule is that the last write of a field wins: two people who set the same field have each said what it should be, and the later one is what it is. That is not a loss, and the drill passes it. When the editor has two text fields, the first person changes the second field and the other person the first, so the drill can tell the two failures apart:
- A whole-record overwrite. The later save put a field back to its old value, a field its own page did not change. The first person's change is gone and nobody chose that. The line names the field. Write only the fields the page changed: copy what was edited onto the row, not the whole row as the page last read it.
- A stale save a
[ConcurrencyCheck]accepted. The field is guarded (ConcurrencyCheck, on the entity or on that field), the other person changed it after this page read it, and the save went through anyway.
A guarded field's stale save is refused, and the page should say so. The drill passes it when the page shows anything
new: the platform's conflict surface, an alert of its own, or the app's own sentence. If two people setting the same
field must not simply take the later value, declare [ConcurrencyCheck] on it.
Examples#
osy drill # every drill, every page, signed out
osy drill --drill double-submit --page / # one drill, one page
osy drill --as [email protected] --as [email protected] \
--setup "user add [email protected] --role Owner --password pw123456" \
--setup "user add [email protected] --role Owner --password pw123456"See also#
ConcurrencyCheck — the rule that refuses a stale save, which concurrent-edit holds to.
Checking your app — osy check: validate, lint and your own tests, before you drill.
The browser for screenshots — the browser osy drill drives.