# The stemming kit — an inflected word finds its other forms

> `Osysharp.Stemming` puts words in their base form, per language, so a search for "betalningen" finds "betalningar" and a search for "بالبطاقة" finds "البطاقات". Snowball stemmers for English, Swedish, Danish and Norwegian, a light stemmer for Arabic, and word for word for every other language — deterministic, offline and pure Osy#.

<!-- id: ui-stemming-kit · area: ui · stability: preview · html: https://osysharp.com/reference/ui/stemming-kit/ -->

## Summary        {#summary}
Full-text search matches words. In English that mostly works; in Swedish, a word has a dozen endings ("faktura",
"fakturan", "fakturor", "fakturorna"), and in Arabic an article and a preposition join the front of it. A reader who
types the form in their head misses the article that used another. **Stem both sides** — the text as it is written,
the query as it is asked — and two forms of one word meet:

```osy syntax
var s = new Stemmer();
s.StemText("Så betalar du dina fakturor", "sv");   // "så betal du din faktur"
s.StemText("hur betalar jag fakturan", "sv");      // "hur betal jag faktur" — both have "betal" and "faktur"
```

## Signature      {#signature}
```osy syntax
using Osysharp.Stemming;

public class Stemmer {
  string       StemText(string text, string language);   // every word, stemmed, joined by spaces
  string       Stem(string word, string language);       // one word, lower-cased and stemmed
  List<string> WordsOf(string text);                     // the words a text is split into
  bool         Knows(string language);                   // does it stem `language`, or match word for word?
}
```

## Description    {#description}

### Which languages   {#languages}
| language | how |
|---|---|
| English (`en`) | a light stemmer — plurals, `-ing`, `-ed`, `-ly` |
| Swedish (`sv`), Danish (`da`), Norwegian (`nb`, `no`) | the Snowball rules for each language |
| Arabic (`ar`) | a light stemmer — the article (ال), a leading conjunction or preposition (و، ف، ب، ك، ل), the common suffixes |
| any other | lower-cased, word for word |

A language tag with a region ("sv-SE") is its language. A stem is a pure function of the word and its language, so a
text's stems can be written once, when the text is, and searched like any other column.

### How to use it in a search   {#search}
Keep a stemmed copy beside the text, mark it searchable, and stem the query in the reader's language:

1. When the text is written: `row.Stems = new Stemmer().StemText(row.Text, row.Language);`
2. When searching: `var qs = new Stemmer().StemText(query, readerLanguage);` and match `r.Text.Matches(query) ||
   r.Stems.Matches(qs)`.

Matching the text as written as well keeps an exact phrase, a name or a code findable whatever the stemmer does with
it. The KnowledgeBase kit does exactly this ([[ui-knowledge-kit#search]]).

## Examples       {#examples}

A recipe site in Swedish and Arabic whose search finds a recipe by any form of its words:

```osy title="stems kept beside the text, and the query stemmed the same way" test app=ui-stemming-kit
app Recipes {
  use Osysharp.Memory;
  use Osysharp.Stemming@0;
  model "model/**/*.osy";
}

using Osysharp.Memory;
using Osysharp.Stemming;

entity Recipe {
  [Required, MaxLength(120)] string Title;
  [Searchable(Entity), MaxLength(4000)] string Method;
  [Searchable(Entity), MaxLength(4000)] string MethodStems = "";
  [Required, MaxLength(10)] string Language = "sv";
  security { allow read, create, update when IsAnonymous || IsAuthenticated; }
}

public Recipe AddRecipe(string title, string method, string language) {
  return new Recipe { Title = title, Method = method, Language = language,
                      MethodStems = new Stemmer().StemText(method, language) };
}

public List<string> FindRecipes(string query, string language) {
  var qs = new Stemmer().StemText(query, language);
  return Recipe.Where(r => r.Method.Matches(query) || r.MethodStems.Matches(qs)).Select(r => r.Title).ToList();
}

[Test]
void an_inflected_query_finds_the_other_forms() {
  AddRecipe("Kanelbullar", "Rulla degen, strö över kanel och grädda bullarna.", "sv");
  UnitOfWork.Commit();
  Assert.Equal("Kanelbullar", FindRecipes("bullen", "sv")[0]);
}

[Test]
void a_stem_is_the_same_on_both_sides() {
  var s = new Stemmer();
  Assert.Equal(s.Stem("البطاقات", "ar"), s.Stem("بالبطاقة", "ar"));
  Assert.True(s.Knows("sv-SE"));
  Assert.False(s.Knows("fi"));
}
```

## See also       {#see-also}

- [field search (Matches / TextScore / Similarity)](https://osysharp.com/reference/memory/field-search/) — full-text search over a field
- [The KnowledgeBase kit — a knowledge base and the help centre it is read in](https://osysharp.com/reference/ui/knowledge-kit/) — a help centre that searches each article in its own language
- [Which language a reader gets — and switching it](https://osysharp.com/reference/i18n/language/) — the reader's language