Analytics app

Audience: apps that need fast filtering, joining across collections, and pre-aggregated reports — dashboards, BI consoles, search-driven UIs. Bundle: core + withIndexing() + withReduce() + withLazy() + withSession(). No history, no blobs, no sync, no i18n by default — add them as needed.

Verified by: the runnable test showcases/src/recipe-analytics.recipe.test.ts in the documentation repository covers the indexed == filter, .groupBy() + .aggregate(), .join(), scan().aggregate(), and the error thrown when withReduce() is omitted. Lazy mode, multi-store routing, and session-policy behavior (the sections below covering withLazy(), routeStore(), and sessionPolicy) are not exercised by an automated test in this repository.

noy-db is an encrypted, offline-first document store: every record is encrypted inside your application process — in @noy-db/hub, the trust boundary — before it reaches any storage backend, and queries run inside the hub after decryption. The backend only ever holds ciphertext and never executes a query. That shapes how analytics work: filtering, joining, and aggregating are hub-side operations, and this recipe assembles the opt-in services that keep them fast and memory-bounded at thousands to tens of thousands of records.

What this gets you

Setup

Storage is pluggable; this recipe uses @noy-db/to-postgres, which accepts any client exposing query(sql, params) → { rows } (node-postgres Pool/Client, Neon, Vercel Postgres, …).

import { createNoydb } from '@noy-db/hub'
import { withIndexing } from '@noy-db/hub/indexing'
import { withReduce } from '@noy-db/hub/reduce'
import { withLazy } from '@noy-db/hub/lazy'
import { withSession } from '@noy-db/hub/session'
import { toPostgres } from '@noy-db/to-postgres'
import { Pool } from 'pg'

const db = await createNoydb({
  store: toPostgres({ client: new Pool({ connectionString: process.env.DATABASE_URL }) }),
  user: 'analyst@example.com',
  secret: process.env.NOYDB_SECRET!,

  indexingStrategy: withIndexing(),
  reduceStrategy: withReduce(),
  lazyStrategy: withLazy(),
  sessionStrategy: withSession(),

  sessionPolicy: { idleTimeoutMs: 30 * 60_000 }, // requires sessionStrategy
})

Declare indexes and references up front

Indexes are declared as field names. References (ref()) declare a soft foreign key by naming the target collection — the declaration is what later powers .join().

import { ref } from '@noy-db/hub'

interface Client {
  id: string
  name: string
  segment: 'enterprise' | 'sme' | 'consumer'
}

interface Invoice {
  id: string
  clientId: string
  amount: number
  status: 'draft' | 'open' | 'paid' | 'overdue'
  date: string
  region: 'EU' | 'US' | 'APAC'
}

const vault = await db.openVault('reporting')

const clients = vault.collection<Client>('clients')

const invoices = vault.collection<Invoice>('invoices', {
  indexes: ['clientId', 'status', 'date', 'region'],
  refs: { clientId: ref('clients') },  // FK by collection *name*
})

A small seed so the outputs below are concrete:

await clients.put('c1', { id: 'c1', name: 'Acme',  segment: 'enterprise' })
await clients.put('c2', { id: 'c2', name: 'Bevco', segment: 'sme' })
await clients.put('c3', { id: 'c3', name: 'Cosmo', segment: 'consumer' })

await invoices.put('i1', { id: 'i1', clientId: 'c1', amount: 1000, status: 'paid',    date: '2026-05-02', region: 'EU'   })
await invoices.put('i2', { id: 'i2', clientId: 'c1', amount: 2500, status: 'paid',    date: '2026-06-14', region: 'EU'   })
await invoices.put('i3', { id: 'i3', clientId: 'c2', amount:  500, status: 'paid',    date: '2026-06-20', region: 'US'   })
await invoices.put('i4', { id: 'i4', clientId: 'c2', amount:  300, status: 'open',    date: '2026-06-28', region: 'US'   })
await invoices.put('i5', { id: 'i5', clientId: 'c3', amount:  100, status: 'overdue', date: '2026-04-30', region: 'APAC' })
await invoices.put('i6', { id: 'i6', clientId: 'c1', amount:  900, status: 'paid',    date: '2026-07-01', region: 'APAC' })

Filter fast

In the default (eager) mode the whole collection is decrypted into memory on first access, and query() terminals are synchronous. With withIndexing() enabled, == and in predicates on declared fields dispatch through a hash index instead of a linear scan.

const overdue = invoices.query()
  .where('status', '==', 'overdue')
  .orderBy('date', 'desc')
  .toArray()

[
  { "id": "i5", "clientId": "c3", "amount": 100, "status": "overdue", "date": "2026-04-30", "region": "APAC" }
]

Join across collections

.join(field, { as }) attaches the referenced record under an alias. It requires the ref() declaration from above and is projection-only — it enriches rows, it never filters them.

const enriched = invoices.query()
  .where('status', '==', 'paid')
  .join('clientId', { as: 'client' })
  .toArray()

[
  {
    "id": "i1", "clientId": "c1", "amount": 1000, "status": "paid",
    "date": "2026-05-02", "region": "EU",
    "client": { "id": "c1", "name": "Acme", "segment": "enterprise" }
  },
  { "…": "3 more rows (i2, i3, i6), each carrying its joined client record" }
]

The join strategy is automatic: nested-loop when the target collection exposes its built-in primary-key lookup (the normal case for any Collection target — O(1) per row via the collection's own id map, unrelated to whether withIndexing() declares an index on the join field), falling back to hash join only for custom query sources that don't expose that lookup.

Aggregates per group

.groupBy(field) buckets the matching records, then .aggregate(spec) computes named reducers per bucket. .run() returns the rows synchronously; each row carries the group key under the grouping field's own name plus every reducer output.

import { sum, avg, count } from '@noy-db/hub/reduce'

const byRegion = invoices.query()
  .where('status', '==', 'paid')
  .groupBy('region')
  .aggregate({
    total: sum('amount'),
    avg: avg('amount'),
    n: count(),
  })
  .run()

[
  { "region": "EU",   "total": 3500, "avg": 1750, "n": 2 },
  { "region": "US",   "total": 500,  "avg": 500,  "n": 1 },
  { "region": "APAC", "total": 900,  "avg": 900,  "n": 1 }
]

For a dashboard that stays current, the same .aggregate(spec) wrapper also exposes .live(), which re-runs the reduction whenever the underlying data changes (call .stop() on the handle when done).

Streaming aggregates over very large sets

When the match set is too large to hold in memory, scan() streams records page by page (decrypting at most pageSize at a time) and reduces with O(reducers) memory — no row ceiling. Its .aggregate() terminal returns a promise.

const total = await invoices.scan()
  .where('status', '==', 'paid')
  .aggregate({ sum: sum('amount') })
// → { sum: 4900 }

Beyond memory: lazy mode

Eager mode loads every record of a collection into memory. For collections beyond that, declare the collection lazy: records are fetched per-id on demand into a bounded LRU cache, and reads go through the persisted index side-cars instead of a full snapshot. Lazy mode requires lazyStrategy: withLazy() plus a cache budget — at least one of maxRecords or maxBytes (a number of bytes).

const invoicesLazy = vault.collection<Invoice>('invoices', {
  prefetch: false,                                            // lazy — nothing bulk-loaded at open
  cache: { maxRecords: 5_000, maxBytes: 50 * 1024 * 1024 },   // 50 MB budget
  indexes: ['clientId', 'status', 'date', 'region'],
  refs: { clientId: ref('clients') },
})

// In lazy mode query() throws — use lazyQuery(), whose terminals are async.
// Every field used in where()/orderBy() must be a declared index,
// otherwise the terminal throws IndexRequiredError.
const recentOverdue = await invoicesLazy.lazyQuery()
  .where('status', '==', 'overdue')
  .orderBy('date', 'desc')
  .limit(50)
  .toArray()

[
  { "id": "i5", "clientId": "c3", "amount": 100, "status": "overdue", "date": "2026-04-30", "region": "APAC" }
]

Multi-store routing — hot vs. cold

For datasets that outgrow one backend, routeStore() from @noy-db/hub (always-core) multiplexes a single logical store across several physical ones: a default store, per-collection routes, vault-name prefixes (vaultRoutes), blob tiering (blobs), and age-based hot/cold tiering (age). With age tiering, records whose last modification is older than coldAfterDays migrate to the cold store during compact(), and reads fall through to the cold tier transparently.

import { createNoydb, routeStore } from '@noy-db/hub'
import { toPostgres } from '@noy-db/to-postgres'
import { toAwsS3 } from '@noy-db/to-aws-s3'
import { Pool } from 'pg'

const db = await createNoydb({
  store: routeStore({
    default: toPostgres({ client: new Pool({ connectionString: process.env.DATABASE_URL }) }),
    age: {
      cold: toAwsS3({ bucket: 'reporting-archive' }),
      coldAfterDays: 90,
    },
  }),
  user: 'analyst@example.com',
  secret: process.env.NOYDB_SECRET!,
})

Every routed store still only ever sees ciphertext — routing decides where an encrypted envelope (the ciphertext wrapper each record is stored as) lands, never what is inside it. See routing for middleware (retry, circuit breaker, metrics) around stores.

Performance ceilings to know about

• Eager joins: JoinTooLargeError at 50,000 rows per side (a one-shot warning fires at 80%). Override per join leg with { maxRows } if you've measured. Narrow the left side with where()/limit() first.
• groupBy: warns at 10,000 distinct groups, throws GroupCardinalityError at 100,000. The cap is not overridable — beyond it you probably want a streaming aggregate or an external OLAP system. Grouping on a high-uniqueness field like id is almost always a query mistake.
• Aggregations and counts ignore joins: .count() and .aggregate() reduce over base records only; joined fields are projection-only and cannot feed a reducer.
• Lazy mode cache: tune maxRecords / maxBytes to your working set; the LRU evicts least-recently-used entries. An unbounded lazy cache is rejected at declaration time.
• Heavy fields: a collection mixing small hot fields with large rarely-read ones pays full-envelope decrypt cost on every read; see satellite collections (experimental) to split them off-row.

See also

• indexing — eager fast-paths, persisted side-cars, rebuild/reconcile
• aggregate — reducers, groupBy, live aggregations
• joins — FK joins, strategies, ceilings
• lazy — lazy mode and the LRU working set
• routing — multi-store routing and store middleware
• session — session policy and idle timeout
• Services — the full service catalog