stable

Search & RAG

Retrieval over an encrypted collection — lexical, semantic, and hybrid — then the in-ai toolset handing permission-scoped tools to an LLM loop.

Audience: apps that need "find it by words or by meaning" over encrypted records — and, optionally, an LLM answering questions over them. Bundle: core + withSearch() (one factory, all three retrieval modes) + @noy-db/in-ai for the LLM loop.

Verified by: showcases/src/recipe-search-rag.recipe.test.ts — the code below is the same code that runs there.

Everything on this page happens client-side, after decryption: the index, the vectors, the ranking, and the tool dispatch all run inside the hub. The store holds only ciphertext throughout — search adds no leakage beyond reading the collection.

Prerequisites

searchStrategy: withSearch() — one opt-in enables lexical, semantic, and hybrid (see search / search — AI, two pages, one subsystem).
A textIndexes allowlist on the collection for indexed lexical retrieve().
An embeddings encoder on the collection for semantic/hybrid — a { dim, model, source, encode } object you supply. An on-device model keeps zero-knowledge end-to-end; a hosted embeddings API adds that provider as a party that can read your text — choose deliberately.
@noy-db/in-ai for the RAG half: pnpm add @noy-db/hub @noy-db/in-ai.

Setup

import { createNoydb, withSearch } from '@noy-db/hub'
import { toBrowserIdb } from '@noy-db/to-browser-idb'

const db = await createNoydb({
  store: toBrowserIdb(),
  user: 'analyst',
  secret: 'search-rag-pass-2026',
  searchStrategy: withSearch(),
})
const vault = await db.openVault('firm')

const notes = vault.collection<Note>('notes', {
  textIndexes: ['text'],   // lexical retrieval reads this allowlist
  embeddings: encoder,     // your text→vector function (dim/model must match it)
})

The walkthrough's encoder is a deterministic stub (a bag-of-chars hash) so the paired test is reproducible — in an app you plug a real model; the shape is just:

const encoder = {
  dim: 384, model: 'your-model-name', source: 'text' as const,
  encode: async (text: string): Promise<Float32Array> => yourModel.embed(text),
}

Three retrieval modes, one call

await notes.retrieve('invoice payment', { mode: 'lexical' })
// BM25 over the textIndexes fields — matching *words*, ranked, with snippets

await notes.retrieve('invoice payment', { mode: 'semantic' })
// cosine similarity over the encrypted _vec side-cars — matching *meaning*

await notes.retrieve('invoice payment', { mode: 'hybrid' })
// both rankings fused (RRF): a record that appears in both lists rises
// above each modality's own leader — usually what an app wants by default

Zero-setup notes.search(field, query) also exists for ad-hoc scans with no index at all; the search page compares the paths.

The RAG half — permission-scoped tools for an LLM

@noy-db/in-ai turns the vault into function-calling tools — one per (collection × operation), in OpenAI / Anthropic / Vercel AI / JSON-Schema shapes — and dispatches the model's calls back. The LLM never touches keys; it sees only the decrypted records your host process chose to expose:

import { buildToolset, invokeToolCall } from '@noy-db/in-ai'

// Hand the model exactly these capabilities — an allowlist, not a firehose:
const tools = buildToolset(vault, {
  format: 'openai',
  operations: ['list', 'get'],
  collections: ['notes'],
})
// → [{ function: { name: 'notes_list', … } }, { function: { name: 'notes_get', … } }]

// The model answers with a tool call; you dispatch it:
const result = await invokeToolCall(vault, { name: 'notes_get', args: { id: 'n-2' } })

// Out-of-allowlist calls are refused at dispatch too — defense in both directions:
await invokeToolCall(vault, { name: 'notes_delete', args: { id: 'n-2' } },
  { allowedOperations: ['list', 'get'] }) // → rejects

The retrieval modes above are the natural retriever for this loop: retrieve() picks the passages, the toolset fetches the records, your prompt assembles the context.

Verify

Run the paired walkthrough: all three modes return real ranked ids (the fusion never invents rows), the toolset emits exactly the allowlisted tools, dispatch round-trips a record, and an out-of-allowlist operation is refused.