Running a function when a version deploys

A migration can run a function of the version it deploys, once, inside the deploy, after the schema change — the data change no schema verb can say. The function must find nothing to do the second time it runs, and the deploy checks that.

Summary#

Some changes to an app are changes to its DATA: a new table that should start out filled from an old one, a count that becomes rows, a value copied into the member that replaces it. rename and rebase say what happened to a member; run says what to do with the rows. run FillSeats; in a migration runs the new version's FillSeats() while the version deploys — after its tables exist, before anyone is served by it.

Signature#

migration "each team's seats as rows" {
  from "ast:<hash>";
  to   "ast:<hash>";

  run FillSeats;            // a function of the version being deployed, with no parameters
  run Billing.Backfill;     // a dotted name, for a function in a namespace
}

Description#

When it runs. Inside the deploy's one transaction: the schema change first, then each run in the order the migration names them, then the switch to the new version. The function runs as the app itself — what a start hook does — so a security {} grant meant for people does not stand in its way. Every way of deploying runs it the same way: a local compile, and osy deploy to an environment, whose zone builds the version and runs the function inside the move's one transaction. The deploy's dry run does not run it — it writes your data, and the dry run writes nothing — so a function that fails is found by the move, which then applies nothing.

It must be safe to run twice. A deploy can be retried, and a release keeps its migration file, so the same function will meet data it has already moved. The platform does not take that on trust: every function runs a second time, straight after its first run, and a second run that would write anything refuses the deploy, naming the function and what it would have written. So write it to find nothing to do the second time: create a row only when it is missing, set a value only when it differs.

The migration function `FillSeats` is not safe to run twice: run again straight after its first run, it would write
4 more row(s) (Seat). A deploy can be retried and a release keeps its migration, so the function must find nothing to
do the second time — create a row only when it is missing, set a value only when it differs. Nothing of the deploy
was applied.

Nothing is half-applied. A function that throws, waits on a person, takes parameters, or is not in the new version refuses the deploy, and the transaction takes back everything — the schema change, the rows the function wrote, the new version. The app goes on being served by the version it had. osy validate already refuses a run naming a function the version does not declare, or one that takes parameters (MIGRATION_RUN_TARGET).

It reads what the new version declares — and nothing it removes. The function is the new version's, so a member the new version drops is not there for it to read. To move data out of a member and then retire it, take two releases: the first keeps the member declared and runs the copy; the next one drops it (drop property).

When the source goes away, expand then contract. Copying data out of a member you mean to retire is the commonest reshaping, and the way to do it is two releases (see the example below): release N expands — it declares the new shape, keeps the old member (no page or function uses it any more) and runs an idempotent copy into the new shape; release N+1 contracts — it removes the old member with drop property. The reason is the rule above: a run function belongs to the version being deployed, so it can only read what that version declares. Between the two releases both shapes exist, the copy has already run, and no one's data is ever in neither place.

It names the function, without parentheses. A migration function takes no arguments, so run FillSeats(); is refused with the spelling it means.

Examples#

The version before keeps a count of seats on each team:

entity Team {
  [Required, MaxLength(80)] string Name;
  int Seats;
}

The next one gives each seat a row of its own, and a function that makes them — only the ones a team does not have yet:

entity Team {
  [Required, MaxLength(80)] string Name;
  int Seats;
}
entity Seat {
  [Required] Team Team;
  int Number;
}

void FillSeats() {
  foreach (var t in Team.ToList()) {
    for (var n = 1; n <= t.Seats; n++) {
      if (!Seat.Any(s => s.Team == t && s.Number == n)) { new Seat { Team = t, Number = n }; }
    }
  }
}

The migration that runs it:

migration "each team's seats as rows" {
  from "ast:<the deployed hash>";
  to   "ast:<the new hash>";

  run FillSeats;
}

Deploy it and every team has its seats as rows. Deploy the next version with the same migration still in place and FillSeats finds every seat already there, so nothing changes. Take out the if and the deploy is refused — the second run would make every seat again.

Expand, then contract: moving data out of a member that goes away#

A contact's single phone number becomes rows, so a contact can have several. The member that holds the number today, Contact.Phone, is to go. That takes two releases.

The version deployed today:

entity Contact {
  [Required, MaxLength(120)] string Name;
  [MaxLength(40)] string? Phone;
}

Release N — expand. The new shape is declared, the old member stays (nothing reads it except the copy), and the migration runs the copy. It writes a row only when the contact does not have that number yet, so a second run finds nothing to do:

entity Contact {
  [Required, MaxLength(120)] string Name;
  [MaxLength(40)] string? Phone;          // kept for one release: only CopyPhones reads it
}
entity PhoneNumber {
  [Required] Contact Contact;
  [Required, MaxLength(40)] string Number;
}

void CopyPhones() {
  foreach (var c in Contact.Where(c => c.Phone != null).ToList()) {
    if (!PhoneNumber.Any(p => p.Contact == c && p.Number == c.Phone)) {
      new PhoneNumber { Contact = c, Number = c.Phone! };
    }
  }
}

migration "phone numbers as rows" {
  from "ast:<today's hash>";
  to   "ast:<release N's hash>";

  run CopyPhones;
}

Release N+1 — contract. Every number is a row now, so the old member and the copy go, and the migration says so:

entity Contact {
  [Required, MaxLength(120)] string Name;
}
entity PhoneNumber {
  [Required] Contact Contact;
  [Required, MaxLength(40)] string Number;
}

migration "retire Contact.Phone" {
  from "ast:<release N's hash>";
  to   "ast:<release N+1's hash>";

  drop property Contact.Phone;
}

Why not one release that adds PhoneNumber, drops Phone and runs CopyPhones? Because CopyPhones would be a function of that release, and that release has no Phone for it to read — it is refused before it deploys. Keeping the member for one release costs a column nobody uses for a while; in exchange, the copy has run and been checked before anything is removed, and an app still on today's version can go on reading Phone while release N rolls out.