The machine-translation kit — which language a text is in, and the text in another, with your own model

use Osysharp.MachineTranslation@0; · new ModelTranslator { KeySecret = "…" } · Detect(text) · Translate(text, from, to, terms) · new LanguageDetector().Guess(text)

`Osysharp.MachineTranslation` tells which language a text is in — offline first, from its script or the commonest words of twelve European languages, asking a model only when it cannot be sure — and translates it through the app's own model with `Llm.Complete` of `Osysharp.Llm`: the app's key, router and budget, glossary terms kept as written. A missing key is a state (`IsReady()` false, with a reason a person can act on), never an error.

Summary#

Two questions every multilingual app asks — what language is this? and what does it say in mine? — answered with the model the app already has:

var t = new ModelTranslator { KeySecret = "ModelKey" };
var language = t.Detect("Hej! Jag har inte fått min bokning.");   // "sv" — told offline, no call
var english  = t.Translate("Hej! Jag har inte fått min bokning.", language, "en", []);

Detection is local first. LanguageDetector reads the script a text is written in (Arabic, Persian, Cyrillic, Greek, Hebrew, Devanagari, Thai, Japanese, Korean, Chinese) and, for the Latin alphabet, the commonest words and letters of twelve European languages — free, offline, the same answer on every machine. The model is asked only when a text is too short or too mixed to be sure, and its answer is used only when it is a language tag.

Translation is a plain model call — one prompt in, one text out, through Llm.Complete (The Llm kit — call a model, and know what each call cost), so the app's own model, router and budget decide every call. No tools and no memory: the text is untrusted, and the prompt tells the model to treat everything between its markers as text to translate, never as instructions.

Signature#

using Osysharp.MachineTranslation;

public class ModelTranslator {
  string? Model = null;                   // one the app allows; null — the app's default model
  string KeySecret = "";                  // the NAME of the secret holding the model's key; "" — assume reachable
  LanguageDetector Detector;              // how it tells a language before (and usually instead of) asking
  int MaxChunk = 6000;                    // longer texts go a paragraph at a time

  bool   IsReady();                       // its key is set
  string NotReadyReason();                // why not, for a person who can fix it; "" when ready
  string Detect(string text);             // a BCP-47 tag, or "" when a text has no letters
  string Translate(string text, string from, string to, List<TranslationTerm> terms);
}

public class TranslationTerm { string Term; string Rendering = ""; }   // "" keeps Term as written

public class LanguageDetector {
  List<LanguageProfile> Profiles = DefaultProfiles();
  LanguageGuess Guess(string text);       // { Language, Confident }
  string ByScript(string text);           // the language a script alone names, or ""
}
public class LanguageProfile { string Language; List<string> Words; string Letters; }

bool ModelKeyIsSet(string secret);        // stub it in a test: ModelKeyIsSet.Stub(name => true)
bool SameLanguage(string? a, string? b);  // "sv-SE" and "sv" are one language
bool IsLanguageTag(string tag);

Description#

Not ready is a state#

With KeySecret naming a secret that has no value, IsReady() is false and NotReadyReason() says which secret to set, in the reader's language. A caller keeps the original rather than failing: a translator that is down must never stop a message from being written. Detect still answers what the offline detector knows.

Same language, no call#

Translate from a language into itself ("en-GB" into "en") answers the text with no call; so does an empty text.

Teaching the detector a language#

Add a LanguageProfile — its commonest short words and any letters only it uses — and the detector knows it:

var d = new LanguageDetector();
d.Profiles.Add(new LanguageProfile { Language = "is", Letters = "ðþ", Words = ["og", "ég", "ekki", "að", "takk"] });

Translating conversations#

Osysharp.Conversations.MachineTranslation is the bridge to the Conversations kit ([[ui-conversations-kit#language]]): every customer message gains a copy in the desk's language and every reply reaches each customer in theirs.

// Osysharp.Conversations.MachineTranslation arrives by itself in an app that uses Conversations and MachineTranslation.
app.Conversations = new ConversationsSetup {
  Translator = new ModelConversationTranslator { Translator = new ModelTranslator { KeySecret = "ModelKey" } },
};

The desk's glossary reaches the prompt as terms to keep. With no key, messages keep their originals and the desk is told why.

Testing it#

No test should call a real model. CompleteWith.Stub answers for it (Testing model calls without a model — ScriptedModel (Llm kit)) and ModelKeyIsSet.Stub(name => true) makes the translator ready where no key is ever set.

Examples#

A travel guide whose readers comment in their own language, read by the editors in English:

app TravelGuide {
  use Osysharp.MachineTranslation@0;
  use Osysharp.Llm@2;
  model "model/**/*.osy";
}

using Osysharp.MachineTranslation;
using Osysharp.Llm;

entity Comment {
  [Required, MaxLength(4000)] string Text;
  [MaxLength(20)] string? Language;
  [MaxLength(4000)] string? InEnglish;
  security { allow read, create, update when IsAnonymous || IsAuthenticated; }
}

ModelTranslator Translator() { return new ModelTranslator { KeySecret = "GuideModelKey" }; }

public Comment AddComment(string text) {
  var t = Translator();
  var c = new Comment { Text = text };
  var language = t.Detect(text);
  c.Language = language == "" ? null : language;
  if (t.IsReady() && c.Language != null) { c.InEnglish = t.Translate(text, c.Language!, "en", [new TranslationTerm { Term = "Fjällstugan" }]); }
  return c;
}

[Test]
void a_comment_is_read_in_english() {
  ModelKeyIsSet.Stub(name => true);
  CompleteWith.Stub((request, call) => new LlmResult { Response = Scripted.Text("We stayed at Fjällstugan and it was lovely.") });
  var c = AddComment("Vi bodde på Fjällstugan och det var underbart.");
  Assert.Equal("sv", c.Language);
  Assert.Equal("We stayed at Fjällstugan and it was lovely.", c.InEnglish);
}

[Test]
void with_no_key_the_comment_keeps_its_original() {
  var c = AddComment("Vi bodde på Fjällstugan och det var underbart.");
  Assert.Equal("sv", c.Language);
  Assert.Null(c.InEnglish);
}