Osy#betaRun it on Osyrin: osyrin.com

Reference / Stdlib

VectorSearch.Search and VectorStore.* — the raw secured vector store under the memory kit

VectorSearch.Search(VectorQuery q) · VectorSearch.ByMeaning() · VectorStore.IndexFile(file, about) · VectorStore.ForgetFile(file) · VectorStore.Statistics(cutoff) · VectorStore.DropUnusedVectors(cutoff)

The platform's general vector store, the primitives the memory kit is built on. `VectorSearch.Search` answers the raw hits of one query under the caller's own row security — no link hops, citations or labels on top. `VectorStore.IndexFile`/`ForgetFile` queue a file's text as memory about a row, authored by whoever is signed in. `VectorStore.Statistics`/`DropUnusedVectors` are private to the memory kit. App code calls `Memory.*`.

stable1 example compiled by CIstdlibmemorysearchsecurity

Summary#

Memory.Search, Memory.Remember and Memory.Prune are the verbs an app uses. Under them sits a smaller, general store, and a kit can build on it:

callanswerswho may call it
VectorSearch.Search(q)the raw hits of one VectorQuery, as List<VectorHit>any server code: it only returns rows the caller may read
VectorSearch.ByMeaning()true when a search ranks by meaning (the app has an embedder); false when it runs on words aloneany server code; the memory kit's Memory.ByMeaning() is the same answer
VectorStore.IndexFile(file, about)queues a file's text as memory about a row; true when queuedany server code; the author is the signed-in principal, and an anonymous call is refused
VectorStore.ForgetFile(file)removes the memory read from a file; how many rowsany server code
VectorStore.Statistics(cutoff)the counted picture behind a prune, as MemoryStatisticsonly the package Osysharp.Memory
VectorStore.DropUnusedVectors(cutoff)drops the vectors of derived memory unused since cutoff; how manyonly the package Osysharp.Memory

Signature#

List<VectorHit>  VectorSearch.Search(VectorQuery q)
bool             VectorSearch.ByMeaning()
bool             VectorStore.IndexFile(FileAsset file, <row> about)
int              VectorStore.ForgetFile(FileAsset file)
MemoryStatistics VectorStore.Statistics(DateTime cutoff)          // Osysharp.Memory only
int              VectorStore.DropUnusedVectors(DateTime cutoff)   // Osysharp.Memory only

All of them need using Osysharp.Memory;, and all of them are server-only. The store's types (VectorQuery, VectorHit, MemoryKind, MemoryOrigin, IEmbedder) come from the platform package Osysharp.Memory.Store. That package declares them into the Osysharp.Memory namespace, so use Osysharp.Memory; (the memory kit, which builds on the store) brings them with it.

Description#

The search reads as the caller. VectorSearch.Search compiles one secured query over the memory rows with the caller's own row security. It embeds the query through the app's budgeted embedder, and with no embedder it searches keyword-only over the full-text index. It records usage against each hit it returns, so a prune never drops a vector somebody is finding. A row the caller may not read is never a hit. Its words are never a hit's Content either.

VectorQuery carries everything the search can be asked: Query, About and Hops (row ids), Of (type names), By (author ids), Kinds, Origin, Keyword, Identifiers, Limit (1–100, default 20) and Threshold (default 0.5). A list left null or empty does not narrow the search. Of takes entity types as written: Of = [FieldNote, Invoice] names the two types.

Records.Labels(rows) names the rows a hit is about, in the same caller's read. See Records — what any record is called, where it opens, who may read it.

The author is never an argument. VectorStore.IndexFile takes the author from the session. A caller cannot name somebody else as the one who chose to remember a file. With nobody signed in, the call is refused.

The counts and the prune belong to the memory kit. The kit answers them only to the app's memory operator (app.Memory.Operator). If any code could call the prune, that gate would mean nothing, so a call from an app or from another kit does not compile. Call Memory.Statistics and Memory.Prune instead.

Calling the store from your own code#

A search names what to look for and how far to narrow it. A hit carries the matched text and the row it is about. Filing a file names the row only. The author is the signed-in principal:

using Osysharp.Memory;
using Osysharp.Storage;

[Principal] entity Member {
  [MaxLength(200)] string Email;
  security { allow read when IsAuthenticated; }
}

entity FieldNote {
  [MaxLength(80)] string Title = "";
  [Searchable(Memory)] string Body = "";
  semantic => $"{Title}";
  security { allow read when IsAuthenticated; }
}

// Only notes the caller may read come back, closest first.
List<VectorHit> NotesAbout(string q) =>
  VectorSearch.Search(new VectorQuery { Query = q, Of = [FieldNote], Limit = 5 });

// Queued, not embedded yet: `true` when the file's text is on its way into memory about `note`.
bool File(FileAsset scan, FieldNote note) => VectorStore.IndexFile(scan, note);

Parameters that take entity types#

A list of entity names passed where a list of strings is wanted names those types. This works for a function's parameter, a property and a local. A kit function can take of: [FieldNote] the way Memory.Search does:

List<SearchHit> Notes(string q, List<string>? of = null) => …;
var hits = Notes("harbour", of: [FieldNote, Invoice]);   // of = ["FieldNote", "Invoice"]

See also#

Related

using Memory (semantic search)

Opt into semantic (vector) search over your app's content. `using Memory;` adds a searchable store to the app…

Memory.Remember / Memory.Forget

Put a file's text into the app's searchable memory, filed against the record it is about — and take it back out again…

Memory.Statistics / Memory.Prune

`Memory.Statistics` reports what your memory costs and what pruning at a given date would free — counted with the very…

Records — what any record is called, where it opens, who may read it

`Records.*` answers facts about a row of ANY type from what the app declared — what it is called (`[Title]`), where it…