# Parallel legs (start several, then wait for them)

> Start several pieces of work at once, each with its own compensation, and wait for them together. Writing `saga.Run("step", ...)` WITHOUT `await` starts a leg and hands back a handle; `await Workflow.WhenAll(a, b)` waits for every leg, `WhenAny` for the first to succeed, `When(n, …)` for enough of them. Each leg is a different kind of work — a flight and a hotel, not ten of the same thing — so each is created by ordinary code and carries the undo that belongs to it.

<!-- id: workflow-parallel-legs · area: workflow · stability: stable · html: https://osysharp.com/reference/workflow/parallel-legs/ -->

## Summary        {#summary}
Some work has to happen **together**. Booking a trip means a flight *and* a hotel; neither is worth having on its own,
both take time, and doing them one after the other means the customer waits twice as long for an answer.

Inside a [Workflow.BeginSaga (a compensating saga scope)](https://osysharp.com/reference/workflow/beginsaga/) scope, **a `saga.Run("step", ...)` written without `await` starts a leg and carries on**. It
hands back a **leg handle**, and `await Workflow.WhenAll(...)` waits for the legs you name. The distinction is the same
one C# draws between `var t = Foo();` and `await Foo();` — start, versus start and wait.

Each leg keeps everything that made a sequential step readable: it is created by ordinary statements above the call,
and its compensation sits on the call that starts it. Nothing has to be squeezed into an argument.

When the legs *are* many of the same kind — one per order line, say — **build the list yourself and hand it to the same
wait**: `await Workflow.WhenAll(legs)`. You still write the loop, the creation and the undo; only the count stops being
something the source knows.

## Signature      {#signature}
```osy syntax
var leg = saga.Run("step", step, () => Undo(step));   // START a leg — no `await`

await Workflow.WhenAll(a, b, c);              // every leg
await Workflow.WhenAny(a, b, c);              // the first to succeed
await Workflow.When(2, a, b, c);              // enough of them

await Workflow.WhenAll(legs);                 // …or a LIST of legs, when the count is only known at run time
await Workflow.When(2, legs);

await Workflow.WhenAny(a, b, losers: Losers.Continue);   // …and leave the rest running
```

`await` is what separates the two forms, and it is not decoration:

| you write | what happens |
|---|---|
| `await saga.Run("step", step, undo)` | run this one step and **wait** for it before the next line |
| `var leg = saga.Run("step", step, undo)` | **start** it and carry on; `leg` is what you wait on later |

## Description    {#description}

### Every leg you start must be joined   {#must-join}
A leg's compensation is registered **when the leg succeeds**, and the wait is what learns that. So a leg you start and
never wait for is work with nothing to undo it — a hotel booked with no way back. That never compiles: the compiler
names the leg and the line.

For the same reason, a wait must be written with `await`. Without it nothing waits, and nothing observes the outcomes.

### Many legs of the same kind: build the list, pass the list   {#list-of-legs}
Sometimes the legs really are the same work repeated — reserve every line of this order, notify every subscriber — and
how many there are is a property of the data, not of the source. Build them in a loop and hand the wait the list:

```osy syntax
var legs = new List<Leg>();
foreach (var line in order.Lines)
  legs.Add(saga.Run("reservation", new Reservation { Line = line }, () => Release(line)));

await Workflow.WhenAll(legs);
```

This is the same overload pair C# gives `Task.WhenAll` — several tasks, or a sequence of them — which is why it is an
argument form rather than a fourth verb. **Everything that made a leg readable stays where it was**: you write the
loop, you create each step, and each carries its own undo. The wait is handed handles; it does not iterate on your
behalf and it does not own any leg's body.

Everything below applies per element: the settle-then-unwind rule, which legs get compensated, and the requirement
that every leg you start is joined — a list built and never waited for is refused by name, exactly as a single leg is.

### Waiting for enough, not for all   {#whenany}
`WhenAll` waits for every leg. `WhenAny` returns as soon as **one succeeds**, and `When(n, …)` as soon as **n do** —
they count successes, not finishes, because the question is "did enough of my work get done", and a leg that failed
did not.

If a quorum becomes impossible — too many legs have failed — the join reports the failure once every leg has settled,
exactly as `WhenAll` does.

### What happens to the legs that lost   {#losers}
By default they are **cancelled and compensated**. A leg you gave an undo to is one you said must not dangle, and work
left standing with nothing to reverse it is the failure that is invisible until it matters.

Cancelling is thorough: the leg stops, **and every level of it unwinds itself on the way out**, so a leg that had its
own compensations registered runs them before it ends. Then this run's own undo for that leg runs too.

That last part is worth being precise about, because it is the difference between correct and nearly correct: a
loser's compensation **runs now**, rather than joining the stack that unwinds if the saga fails. A saga that goes on to
`Complete()` drops that stack — so a loser parked there would be quietly forgotten by exactly the run that won.

Say `losers: Losers.Continue` when the losers really are irrelevant rather than wrong — a "first responder answers"
race where the others doing their work harms nothing. It has to be written down; it is not somewhere you should be
able to arrive by omission.

`WhenAll` has no losers, so passing `losers:` there is an error rather than something quietly ignored.

### What `WhenAll` guarantees   {#whenall}
- **It returns only when every leg has finished.** One leg finishing changes nothing on its own.
- **Every leg that succeeded is compensable.** Their undos go onto the scope's stack in the order the legs were
  started, so a later failure unwinds them last-started-first — the same order a sequence of steps produces.
- **A leg that failed compensates nothing.** Only work that actually committed gets an undo. This is the same rule a
  sequential step follows, and it is why the wait, rather than the start, is where undos are registered.

### When a leg fails: everything settles first   {#settling}
If one leg fails while another is still running, **nothing is cancelled and nothing is compensated yet**. The wait lets
the other legs reach their own conclusion, and only then reports the failure.

This is deliberate, and it is the difference between a compensation that is safe and one that is not. Compensating
against a leg that is *halfway through* means undoing something whose extent nobody knows — the hardest problem in the
category. Letting each leg finish first means every compensation faces a settled fact.

The cost is latency on a path that was already failing, which is the cheapest place to spend it.

### What it costs you when it fails   {#failure}
The failure arrives as an ordinary catchable workflow error naming the legs that failed, so the surrounding
`try`/`catch` — and the scope's unwind — work exactly as they do for a single step. Nothing new to learn.

## Examples       {#examples}
The world the trip is booked in — two kinds of work, each its own entity, its own workflow, and its own way back:

```osy title="the app the trip saga lives in" test app=wf-parallel-legs-example
enum TripStatus { Planning, Booked, Abandoned }
enum LegStatus  { Pending, Confirmed }

entity Trip   { [Required, MaxLength(60)] string Destination; TripStatus Status = TripStatus.Planning; }
entity Flight { [Required] Trip Trip; [Required, MaxLength(20)] string Route; LegStatus Status = LegStatus.Pending; }
entity Hotel  { [Required] Trip Trip; [Required, MaxLength(60)] string City;  LegStatus Status = LegStatus.Pending; }

void CancelFlight(Flight flight) { }
void CancelHotel(Hotel hotel)    { }

workflow FlightFlow { Tracks = Flight.Status; Autostart = false; Initial = Pending;
  event Confirm(); state Pending { subscribe Confirm(); on Confirm { goto Confirmed; } } terminal success Confirmed { } }
workflow HotelFlow { Tracks = Hotel.Status; Autostart = false; Initial = Pending;
  event Confirm(); state Pending { subscribe Confirm(); on Confirm { goto Confirmed; } } terminal success Confirmed { } }
```

Then the booking itself — both legs in flight at once, each with the undo that belongs to it:

```osy title="book the flight and the hotel together" test app=wf-parallel-legs-example
workflow TripFlow {
  Tracks = Trip.Status; Autostart = false; Initial = Planning;
  state Planning {
    enter {
      await using var saga = Workflow.BeginSaga();
      try {
        // Each row is created inside a `Workflow.Once` step: the body is re-entered from the top after a wait, so a
        // bare `new` would create a second row on resume. Complex creation is just statements.
        var flight = Workflow.Once("make-flight", () => new Flight { Trip = this.Item, Route = "LHR-JFK" });
        var hotel  = Workflow.Once("make-hotel",  () => new Hotel  { Trip = this.Item, City  = this.Item.Destination });

        var bookingFlight = saga.Run("flight", flight, () => CancelFlight(flight));  // STARTED, not waited for
        var bookingHotel  = saga.Run("hotel", hotel,  () => CancelHotel(hotel));

        await Workflow.WhenAll(bookingFlight, bookingHotel);               // …now wait for both

        saga.Complete();
        goto Booked;
      } catch (WorkflowError e) { goto Abandoned; }   // whichever leg stood is cancelled on the way out
    }
  }
  terminal success Booked     { }
  terminal error   Abandoned  { Message = "trip abandoned"; }
}
```

If the hotel cannot be had, the flight leg is still allowed to finish — and *then* the flight is cancelled and the trip
routes to `Abandoned`. If both stand, `saga.Complete()` drops the undos and nothing is cancelled.

Change one word and it becomes a race — book whichever is available first, and release the other:

```osy title="whichever confirms first wins" test app=wf-parallel-legs-example
void ReleaseFlight(Flight flight) { }
void ReleaseHotel(Hotel hotel)    { }

workflow RaceFlow {
  Tracks = Trip.Status; Autostart = false; Initial = Planning;
  state Planning {
    enter {
      await using var saga = Workflow.BeginSaga();
      try {
        var flight = Workflow.Once("make-flight", () => new Flight { Trip = this.Item, Route = "LHR-JFK" });
        var hotel  = Workflow.Once("make-hotel",  () => new Hotel  { Trip = this.Item, City  = this.Item.Destination });

        var byAir  = saga.Run("flight", flight, () => ReleaseFlight(flight));
        var byRoom = saga.Run("hotel", hotel,  () => ReleaseHotel(hotel));

        await Workflow.WhenAny(byAir, byRoom);   // the loser is cancelled AND released
        saga.Complete();
        goto Booked;
      } catch (WorkflowError e) { goto Abandoned; }
    }
  }
  terminal success Booked    { }
  terminal error   Abandoned { Message = "trip abandoned"; }
}
```

And when the legs are one-per-element, the loop is yours and only the wait changes:

```osy title="reserve every line of the order, in parallel" test app=wf-parallel-legs-list
enum OrderStatus { Placed, Reserved, Abandoned }
enum LineStatus  { Pending, Held }

entity Order {
  [Required, MaxLength(30)] string Ref;
  OrderStatus Status = OrderStatus.Placed;
  [ForeignKey(Order)] OrderLine[] Lines;
}
entity OrderLine   { [Required] Order Order; [Required, MaxLength(20)] string Sku; int Quantity; }
entity Reservation { [Required] OrderLine Line; LineStatus Status = LineStatus.Pending; }

void ReleaseReservation(Reservation reservation) { }

workflow ReservationFlow { Tracks = Reservation.Status; Autostart = false; Initial = Pending;
  event Confirm(); state Pending { subscribe Confirm(); on Confirm { goto Held; } } terminal success Held { } }

workflow OrderFlow {
  Tracks = Order.Status; Autostart = false; Initial = Placed;
  state Placed {
    enter {
      await using var saga = Workflow.BeginSaga();
      try {
        var legs = new List<Leg>();
        foreach (var line in this.Item.Lines) {
          // The step id carries a per-iteration segment, so one label inside a loop is one step PER LINE.
          var reservation = Workflow.Once("make-reservation", () => new Reservation { Line = line });
          legs.Add(saga.Run("reservation", reservation, () => ReleaseReservation(reservation)));   // one leg per line
        }

        await Workflow.WhenAll(legs);          // however many that turned out to be

        saga.Complete();
        goto Reserved;
      } catch (WorkflowError e) { goto Abandoned; }   // the lines that WERE held are released on the way out
    }
  }
  terminal success Reserved  { }
  terminal error   Abandoned { Message = "order abandoned"; }
}
```

## Notes          {#notes}
- **The legs are named by your variables.** A failure says which leg failed, because you gave it a name — not "leg 2".
- **A leg is a workflow of its own**, so it can wait on people, timers and events exactly as any workflow does. That is
  the case the wait exists for; legs that finish instantly work too, and simply never park anything.
- **Starting a leg does not wait for anything**, so the statements between the starts run immediately. The single pause
  in the whole shape is the wait itself.
- **The count in `When(n, …)` is a plain number**, not an expression. A join that asks for more legs than it was given
  can never be satisfied — a compile error when you passed the legs one by one, and a fault at the wait itself when
  they came from a list, which is the first moment the count exists.
- **A list of legs is still your legs.** The wait never iterates anything and never creates anything; it is handed
  handles you made. If you want the platform to fan work out over a collection *for* you, that is a different
  question — see [dynamic fan-out (foreach over a runtime collection)](https://osysharp.com/reference/workflow/fan-out-dynamic/).

## See also       {#see-also}
- [Workflow.BeginSaga (a compensating saga scope)](https://osysharp.com/reference/workflow/beginsaga/) — the scope these legs live in, and how compensation unwinds.
- [Workflow.Run (start a workflow)](https://osysharp.com/reference/workflow/run/) — start one workflow, and optionally wait for it.
- [dynamic fan-out (foreach over a runtime collection)](https://osysharp.com/reference/workflow/fan-out-dynamic/) — many instances of the *same* work, one per element.