Receiving mail — watched mailboxes (Gmail, Microsoft 365, IMAP)

Summary#

Three lines connect a support inbox:

app.Mail = new MailSetup {
  Mailboxes = [new GmailMailbox { Address = "[email protected]", ClientId = "…", ClientSecret = Secret.GoogleSecret,
                                  RefreshToken = Secret.SupportMailbox }],
  Receiver  = new MailIntoConversations(),     // Osysharp.Conversations.Mail files each mail into its conversation
};

WatchMailbox("[email protected]");              // once — an admin's "Connect", or the app's setup

From then on what arrives is kept once and handed to the receiver — pushed the moment it lands where the provider can push (Pushed mail — Gmail through Pub/Sub, Microsoft 365 through Graph notifications), and asked for on an interval otherwise.

Signature#

use Osysharp.Mail@0;                    // IMailbox, IMailReceiver, MailSetup.Mailboxes / Receiver
use Osysharp.Mail.Receiving@0;          // the watch, the kept mail, the two workflows
use Osysharp.Mail.Gmail@0 { egress "oauth2.googleapis.com"; egress "gmail.googleapis.com"; }
use Osysharp.Mail.Microsoft365@0 { egress "login.microsoftonline.com"; egress "graph.microsoft.com"; }
use Osysharp.Mail.Imap@0;               // with `use Osysharp.Http;` — the app names its own servers

MailboxWatch WatchMailbox(string address);         // the address must be one of app.Mail.Mailboxes
void StopWatchingMailbox(MailboxWatch watch);
void RetryReceivedMail(ReceivedMail mail);         // send a Failed one through again
int  EraseMailFrom(string address);                // forget what came from a person, originals and all

public interface IMailbox {
  string Key { get; }
  string Address { get; }
  bool IsReady();
  string NotReadyReason() => …;                    // a default
  MailboxPage Fetch(string cursor, int max);       // an EMPTY cursor means "start now"
  string SendMail(MailMessage message, string idempotencyKey);
  // push (Osysharp.Mail 0.6.0) — every member defaulted, so a mailbox without them is simply polled
  bool CanPush() => false;
  MailboxSubscription Subscribe(string notifyUrl, string secret) { … }
  MailboxSubscription Renew(MailboxSubscription current, string notifyUrl, string secret) => Subscribe(…);
  void Unsubscribe(MailboxSubscription current) { }
  List<MailboxNotice> ReadPush(string body, string notifyUrl) => [];
  TimeSpan PollEvery() => TimeSpan.FromMinutes(1);
  TimeSpan SafetyNetEvery() => TimeSpan.FromMinutes(30);
  MailboxConnection Connection() => new MailboxConnection();   // how an admin connects it from a page
}
public interface IMailReceiver { void Received(MailMessage mail, MailReceipt receipt); }

Description#

Each of the three steps survives a restart#

1. MailboxPolling — one run per watch — asks the mailbox what arrived since its cursor: as soon as the watch is made, whenever a PUSH wakes it, and on an interval — the half-hourly safety net while push is live, the mailbox's own PollEvery() while it is not (a minute for the API mailboxes, five for IMAP; never more often than the app's plan allows). A mailbox that is not ready, or that cannot be reached, leaves the cursor where it was and says why on the watch (MailboxWatch.Problem); the next ask tries again.
2. Each message is KEPT before anything acts on it: a ReceivedMail row with its sender, subject, Message-ID and the whole original as a file. The provider's id is unique per mailbox and the Message-ID across mailboxes, so a message seen twice — a retried poll, a mail sent to support@ AND billing@ — is one message.
3. ReceivedMailHandling hands it to app.Mail.Receiver in a unit of work of its own, after the commit that kept it. A throw undoes what the receiver did; it is tried again — five minutes later, then backing off, at most two hours apart — for a day, then marked Failed with the reason, where staff can see it and send it through again.

A mailbox connected today does not import its history: the first poll answers only the cursor of "now".

One connected mailbox reads AND sends#

A message whose From is one of app.Mail.Mailboxes goes out through that mailbox rather than through app.Mail.Sender — so a reply to a customer leaves from [email protected], threaded, and sits in that mailbox's Sent folder like a reply a person wrote there. Everything else still goes through the sender.

Which mailboxes can it read?#

Each is a class implementing IMailbox; the app brings its OWN credentials, as platform secrets the package never learns the names of, and grants the hosts it may reach.

Gmail and Microsoft 365 need no app password; IMAP is the fallback for any other host. A mailbox whose history has expired at the provider (Gmail's history id, a Graph delta link that answers 410) says so on the watch and resumes from now — mail is never re-delivered twice.

Received mail is customer data#

Nobody reads the watch or the kept mail unless the app grants it:

partial entity ReceivedMail { security { allow read when IsAgent; } }
partial entity MailboxWatch { security { allow read, create when IsAdmin; } }

EraseMailFrom(address) forgets every received mail from that address, originals included — what a right-to-erasure request reaches. Osysharp.Conversations.Mail's erasure calls it for you.

Where staff add and fix a mailbox#

The kit ships Settings → Channels → Mailboxes (/settings/mailboxes), announced to the app's settings area like every kit's sections: each mailbox in app.Mail — watched or not, pushed or polled and why, when it was last asked and last pushed, what is wrong — with Start/Stop watching, Connect for a mailbox connected with the provider's own consent, and the mail the app could not take in, each sent through again with one press. The same verbs are functions an app's own page can call: MailboxViews(), StartWatchingMailbox(address), StopWatchingMailboxAt(address), MailboxConnectUrl(address), SendMailThroughAgain(mail).

Who may use it is ManagesMailboxes — by default the app's own IsAdmin. An app that means somebody else says so:

policy ManagesMailboxes => IsLead;

An app that defines neither ManagesMailboxes nor IsAdmin is refused naming both ([[security-naming-a-policy#supplied-default]]). To replace the page itself, declare a component named MailboxSettings.

Testing without a provider#

Every adapter is plain Osy# over Http.* (or Imap.Fetch/Smtp.Send), so a test stubs the provider's wire and asserts what the app kept. For an app's own tests, a mailbox of rows is the simplest seam: implement IMailbox over an entity the test fills, as Apps/harbour's DemoMailbox does.

A pushing mailbox is tested the same way: stub the provider's subscribe/watch calls, then POST the provider's push body to the app's own push door with Api.Post — through the real route — and assert the watch woke and what it kept. A Gmail push's sender is PushSenderOf(audience); stub it (PushSenderOf.Stub(a => "…@….iam.gserviceaccount.com")) to stand in for the token Google signs, which the platform verifies in production.

A live check needs what the app itself needs: a real mailbox, its credentials as secrets, the egress grants and, for push, the operator's set-up in Pushed mail — Gmail through Pub/Sub, Microsoft 365 through Graph notifications — then WatchMailbox and a mail sent to the address.

Examples#

A mailbox of your own and a receiver of your own — the shape Apps/harbour uses to stand in for a provider (rows a reviewer fills by hand), and the smallest complete consumer of the contract:

app Desk {
  model "model/**/*.osy";
  use Osysharp.Identity@0;
  use Osysharp.Mail@0;
  use Osysharp.Mail.Receiving@0;
  use Osysharp.Workflow;
  use Osysharp.Storage;
}

using Osysharp.Mail;
using Osysharp.Mail.Receiving;

[Role] enum AppRole { Authenticator, Agent }

/// What the mailbox holds — one row per message, its whole RFC 5322 text.
entity InboxRow {
  [Required] int Seq;
  [Required] byte[] Raw;
  security { allow read, create when IsAuthenticated; }
}

/// What the app made of each mail.
entity Request {
  [Required, MaxLength(320)] string From;
  [Required, MaxLength(300)] string Subject;
  security { allow read when IsAuthenticated; }
}

public class RowMailbox : IMailbox {
  public string Key { get; } = "rows";
  public string Address { get; set; } = "";
  public bool IsReady() { return Address != ""; }
  public MailboxPage Fetch(string cursor, int max) {
    if (cursor == "") { return new MailboxPage { Cursor = (InboxRow.Any() ? InboxRow.Max(i => i.Seq) : 0).ToString() }; }
    var page = new MailboxPage { Cursor = cursor };
    foreach (var i in InboxRow.Where(i => i.Seq > int.Parse(cursor)).OrderBy(i => i.Seq).Take(max).ToList()) {
      page.Items.Add(new MailboxItem { ProviderId = "row-" + i.Seq, Raw = i.Raw });
      page.Cursor = i.Seq.ToString();
    }
    return page;
  }
  public string SendMail(MailMessage message, string idempotencyKey) { return "kept-" + idempotencyKey; }
}

public class Requests : IMailReceiver {
  public void Received(MailMessage mail, MailReceipt receipt) {
    new Request { From = AddressPart(mail.From), Subject = mail.Subject };
  }
}

app.Mail = new MailSetup { Mailboxes = [new RowMailbox { Address = "[email protected]" }], Receiver = new Requests() };

partial entity MailboxWatch { security { allow read, create when IsAuthenticated; } }
partial entity ReceivedMail { security { allow read when IsAuthenticated; } }
// Who manages the mailboxes on the kit's settings page — the app's `IsAdmin` by default; this example has no admins.
policy ManagesMailboxes => IsAuthenticated;

public MailboxWatch Connect() { return WatchMailbox("[email protected]"); }

See also#

• Pushed mail — Gmail through Pub/Sub, Microsoft 365 through Graph notifications — push first: the push door, Gmail through Pub/Sub, Graph notifications, renewal, the safety net
• Connect an account from a page — Security.OAuthConnectUrl — "Connect" a mailbox from a page, its grant kept as an app secret
• The Mail kit — send a message, keep a record, pick a sender — MailMessage, the outbox, threading and bounces
• Mail on the wire — Mime.Parse, Mime.Build, Imap.Fetch, Smtp.Send — Mime.Parse/Mime.Build, Imap.Fetch/Smtp.Send
• The Conversations kit — one timeline per conversation, across every channel — MailIntoConversations: each mail into its conversation, files and quoted text included