Fixtures: built once, cloned for every test, and built on each other

Summary#

A [TestFixture] is built once per run. Every test that names it gets its own copy of the world the fixture left — its rows, its files and its clock — so a test may write anything without another test seeing it, and the fixture's cost is paid once however many tests use it.

A fixture may build on another: [TestFixture(Floor)] starts from Floor's world. Expensive worlds are built in layers, once each, and every test starts from the layer it needs.

Signature#

[TestFixture]
void <Floor>() { … }                       // built once per run, from an empty app

[TestFixture(<Floor>)]
void <Layer>() { … }                       // built once, on a copy of <Floor>'s world

[TestFixture(PerTest = true)]
void <Fresh>() { … }                       // built again for each test that names it

[Test(<Layer>)]
void <Name>() { … }                        // runs on its own copy of <Layer>'s world

Description#

How often does a fixture run?#

Once per run, however many tests name it. The runner builds the fixture, keeps what it left as a snapshot, and hands each test a fresh copy of the snapshot. Nothing a test does reaches the snapshot, so the tests stay independent and may run in any order, or at the same time.

So a slow world — five hundred tickets seeded through the app's own functions — costs one seeding, not one per test. Put the seeding in a fixture rather than at the top of each test body:

entity Shelf {
  [MaxLength(40)] string Name;
  security { allow read, create when IsAuthenticated || IsAnonymous; }
}

entity Jar {
  [MaxLength(80)] string Contents;
  Shelf? Shelf;
  security { allow read, create, update when IsAuthenticated || IsAnonymous; }
}

void Stock(Shelf shelf, int count) {
  for (var i = 0; i < count; i++) { new Jar { Contents = shelf.Name + " " + i, Shelf = shelf }; }
}

[TestFixture]
void Pantry() {
  Stock(new Shelf { Name = "Jam" }, 200);
}

[Test(Pantry)]
void The_pantry_is_stocked() {
  Assert.Equal(200, Jar.Count());
}

[Test(Pantry)]
void A_test_writes_only_to_its_own_copy() {
  new Jar { Contents = "Mine" };
  Assert.Equal(201, Jar.Count());    // the other test still sees 200
}

How do I build one world on another?#

Name the fixture to build on: [TestFixture(Pantry)] starts from Pantry's world — its rows, its files, its clock — and adds to it. Pantry is still built once, even when several fixtures and tests stand on it.

[TestFixture]
void Shelved() {
  Stock(new Shelf { Name = "Pickle" }, 50);
}

[TestFixture(Shelved)]
void Reshelved() {
  foreach (var b in Jar.Where(b => b.Contents.StartsWith("Pickle 1")).ToList()) { b.Shelf = null; }
}

[Test(Shelved)]
void Every_jar_has_a_shelf() {
  Assert.Equal(0, Jar.Count(b => b.Shelf == null));
}

[Test(Reshelved)]
void The_layer_starts_from_the_floor_beneath_it() {
  Assert.Equal(50, Jar.Count());
  Assert.Equal(11, Jar.Count(b => b.Shelf == null));    // "Pickle 1" and "Pickle 10" … "Pickle 19"
}

A chain must end in a fixture that names no other. One that comes back to itself — A on B, B on A — is a compile error naming the whole chain, and so is a parent that is not a [TestFixture].

What does a test's copy hold?#

Exactly what the fixture left, or the fixture is refused:

A fixture whose clock is unpinned hands its tests the instant it finished, running forward from there — a row it stamped with "now" is never stale by however long the run took to reach a test.

Why was my fixture refused?#

Some things a fixture can do live in its interpreter rather than its database, and no copy can carry them. A fixture that leaves one is refused, naming it, and every test that names it reports the refusal — never runs against a world its fixture did not describe:

• a stub registered anywhere but a top-level line of the fixture — inside a helper, a loop or a branch. Only a fixture's own top-level X.Stub(…) lines are said again for each test; move the line into the fixture's body.
• TestEmbedder.Use(…) anywhere but a top-level line, for the same reason.
• a screen (Ui.*). What a browser holds — who is signed in, what page is open — is not in the database. Build the data in the fixture; drive the screen in the test.
• a prompt or a dialog it stopped at, which leaves no finished world to copy.

The refusal reads fixture 'Leaky' cannot be copied into its tests: it left a stub for Price registered somewhere other than a top-level statement of the fixture ….

What if my fixture throws?#

Every test that names it errors, and each says so the way a test that throws does: fixture 'Broken' failed to seed: …, with the line it threw at (inside a helper the fixture called, if that is where it was), and a recorded trace. osy test prints the values at the throw, and osy inspect <id> --fault opens the whole call stack — the trace is named after the fixture.

Can a fixture act as somebody?#

Yes — through a helper declared in the test file, exactly as a test does: void FillAsIda() { runas (Ida) { … } }, called from the fixture. The fixture still writes past every security { } block; what runas adds is who is acting, so the app's own functions see Session.CurrentUser and answer their [Authorize(…)] policies for that person. A seed that has to go through the app's real functions, as its admin, belongs in a fixture.

How do I prove a rule that sits behind a deny?#

A [Unique] or an invariant on an entity that only a function may write (deny create except via Assign) cannot be broken by the test body directly — the test's person is refused the direct write, so Assert.Throws would catch the DENY instead of the rule. Put the first row in the fixture, which writes past every security { } block, and make the second attempt the way a person would: as them, through the function. The throw is then the rule's.

[Principal] entity Clerk {
  [Unique, MaxLength(80)] string Name;
  security { allow read when IsAuthenticated; }
}

entity Locker {
  [Unique("That locker is already taken."), MaxLength(10)] string Number;
  security {
    allow read, create when IsAuthenticated;
    deny create except via Assign;          // a locker is made only by Assign
  }
}

[Server, Authorize(IsAuthenticated)]
void Assign(string number) { new Locker { Number = number }; }

[TestFixture]
void LockerA1Taken() {
  new Clerk { Name = "Ida" };
  new Locker { Number = "A1" };             // the fixture is the system: the deny does not apply to it
}

principal Ida => Clerk.Single(c => c.Name == "Ida");

[Test(LockerA1Taken)]
[runas(Ida)]
void A_taken_locker_cannot_be_assigned_again() {
  var e = Assert.Throws<ValidationException>(() => Assign("A1"));
  Assert.Contains("already taken", e.Message);
  Assign("B2");                             // …and the way through still works
  Assert.Equal(2, Locker.Count());
}

When should a fixture be built for every test?#

[TestFixture(PerTest = true)] builds the fixture again for each test that names it, on that test's own branch, so its cost is paid once per test. Reach for it when the fixture's world must be made at the moment the test starts, or to rule the shared copy out while chasing a failure. A fixture built on a PerTest one builds on one fresh build of it.

[TestFixture(PerTest = true)]
void FreshShelf() {
  new Shelf { Name = "Returns" };
}

[Test(FreshShelf)]
void Each_test_gets_its_own_build() {
  Assert.Equal(1, Shelf.Count());
}

See also#

• [Test] / [TestFixture] — [Test] and [TestFixture], and why a fixture is unsecured
• Testing (real app, real data, real rules) — the testing guide
• TestClock.Advance and TestClock.Set — moving the test clock
• Stubbing your own functions in a test (fault injection) — stubbing the app's own functions
• Running tests locally — osy test, and --hold to click through a test's world
• deny … except via — the deny a function writes past
• Workflow.Settle and Background.Settle — let due work run, in a test — letting due work run before you assert