overlay-views

What it does

withOverlayedView() creates a virtual collection: a collection name that has no rows of its own. Instead, every read against it is computed on the fly by merging two real collections — a base collection (typically the output of a materialized view) and an overlay collection that a person can write to directly. One field on the overlay row, the "shadow field", decides which side wins for a given record: if it equals a value you choose, the overlay row is returned in full; otherwise the base row shows through.

This solves a narrow but common problem: a materialized view computes the "right" answer for a row, but occasionally a human needs to correct or annotate a specific record — without hand-editing the source data (which would silently change what the computation is supposed to represent) and without losing the original computed value. Available since 0.1.0-pre.14.

Prerequisites

• A concrete base — a real source collection or a materialized-view output, never another overlay.
• A real, writable overlay collection that is not a materialized-view output.

Opt in

import { createNoydb, withMaterializedView, withOverlayedView } from '@noy-db/hub'
import { sum } from '@noy-db/hub/reduce'

interface Invoice { id: string; clientId: string; amount: number }

// Base: a materialized view computing one total row per client.
const invoiceTotals = withMaterializedView<{ clientId: string; total: number }>({
  name: 'invoice-totals',
  query: (db) =>
    db.collection<Invoice>('invoices')
      .query()
      .groupBy('clientId')
      .aggregate({ total: sum('amount') }),
  rowKey: (row) => row.clientId,
  sources: ['invoices'],
  refresh: 'eager',
})

// Overlay: the virtual collection wiring base → overlay via a shadow field.
const invoiceTotalsLive = withOverlayedView({
  name: 'invoice-totals-live',        // the merged collection you read from
  base: 'invoice-totals',             // the materialized view's output
  overlay: 'invoice-totals-overlay',  // a real, writable collection
  shadowField: 'status',
  shadowValue: 'override',
})

const db = await createNoydb({
  store: /* your NoydbStore */,
  user: 'alice',
  secret: 'a-strong-secret',
  materializedViewStrategies: [invoiceTotals],
  overlayedViewStrategies: [invoiceTotalsLive],
})

overlayedViewStrategies is the only way to register an overlay — without it, the virtual collection name doesn't exist on the vault.

Usage

Reading the merged collection

const vault = await db.openVault('demo')

await vault.collection<Invoice>('invoices').put('inv-1', { id: 'inv-1', clientId: 'acme', amount: 120 })
await vault.collection<Invoice>('invoices').put('inv-2', { id: 'inv-2', clientId: 'acme', amount: 80 })

// No overlay row yet — the merged read falls through to the computed base row.
const row1 = await vault.collection('invoice-totals-live').get('acme')
// → { clientId: 'acme', total: 200 }

// An operator corrects the total by writing to the overlay collection directly.
await vault.collection('invoice-totals-overlay').put('acme', {
  clientId: 'acme', total: 250, status: 'override',
})

const row2 = await vault.collection('invoice-totals-live').get('acme')
// → { clientId: 'acme', total: 250, status: 'override' }

Writing and deleting through the virtual collection

Writes always route to the overlay collection — the base collection is never touched by a write through the virtual name:

// put(record): id is derived from the base materialized view's rowKey.
await vault.collection('invoice-totals-live').put({ clientId: 'acme', total: 300, status: 'override' })

// put(id, record): id must match rowKey(record), else throws OverlayIdMismatchError.
await vault.collection('invoice-totals-live').put('acme', { clientId: 'acme', total: 300, status: 'override' })

// delete removes the overlay row only — the base row resurfaces on the next read.
await vault.collection('invoice-totals-live').delete('acme')

Field-level merge instead of swapping the whole row

Pass mergeMode to pull only a declared subset of fields from the overlay when an intermediate status matches, leaving every other field on the base row:

withOverlayedView({
  name: 'invoice-totals-live',
  base: 'invoice-totals',
  overlay: 'invoice-totals-overlay',
  shadowField: 'status',
  shadowValue: 'override',   // full-override path is unchanged by mergeMode
  mergeMode: {
    kind: 'field-merge',
    rules: [
      { whenStatus: 'amount-corrected', overlayFields: ['status', 'total'] },
      { whenStatus: 'flagged', overlayFields: ['status', 'reviewer'] },
    ],
  },
})

Rules are checked in declaration order; the first rule whose whenStatus equals the overlay row's shadow field wins, and only that rule's overlayFields are pulled from the overlay — every other field still comes from the base row. The binary full-override check (shadowField === shadowValue) always runs first and ignores mergeMode entirely, so adding mergeMode never changes existing full-override behavior.

API reference

withOverlayedView(spec)

Registers an overlay strategy and returns a handle you pass to createNoydb({ overlayedViewStrategies: [...] }). Throws ValidationError immediately if name, base, overlay, or shadowField is missing, or if name, base, and overlay aren't three distinct collection names.

The merged collection — vault.collection(name)

Errors

All extend NoydbError, importable from @noy-db/hub/overlay-views:

• ValidationError — thrown by withOverlayedView() itself for a missing or duplicate name, base, overlay, or a missing shadowField.
• OverlayNameCollisionError(name) — thrown at db.openVault() when name collides with a materialized-view output or another registered overlay's virtual name.
• OverlayBaseIsVirtualError(name, base) — thrown at db.openVault() when base names another overlay's virtual collection; overlay-on-overlay stacking isn't supported.
• OverlayCollectionUnavailableError(name, overlay) — thrown at db.openVault() when overlay names a materialized-view output instead of a real writable collection.
• OverlayIdMismatchError(actual, expected) — thrown from put(id, record) when id doesn't equal the base MV's rowKey(record).

Limits & gotchas

• The merged collection only implements get, list, put, and delete. Calling .query(), .subscribe(), .live(), .scan(), .lazyQuery(), .first(), .count(), .putManyAtomic(), or .deleteMany() on it throws a clear "not yet implemented" error instead of silently misbehaving. Query or subscribe to base / overlay directly instead.
• base must be concrete. A real source collection or a materialized-view output — never another overlay's virtual name. Multi-overlay stacking isn't supported.
• overlay must be a real, writable collection that isn't a materialized-view output. Direct writes to it go through its own normal write pipeline, so any guards or derivations registered on that collection still apply.
• put(record) (single-arg) only works when base is a materialized view. With no rowKey to derive an id from, you must call put(id, record) explicitly.
• An overlay-only row that doesn't match shadowValue (and no mergeMode rule matches) is invisible. If there's no base row either, get() returns null even though a row exists in the overlay collection.
• Cross-reference validation happens at db.openVault(), not at withOverlayedView() call time. Name collisions and bad base/overlay references only surface when the vault opens.

See also

• materialized-views — the usual source of base: a query kept fresh as a queryable collection
• derivations — the row-level sibling that projects one source record into typed outputs
• guards — write-time rules that still apply to direct writes on the overlay collection
• Services overview · Orientation map
• Proven by showcase 83-with-overlay.