manifest

What it does

This service gives a pod one place to answer "what schema generation is this pod at, and does each collection's stored schema still match what it claims?" — without opening every collection.

It stores a single schema manifest record in a reserved _manifest collection. The manifest is an index, not a copy: for each collection it holds the schema-fence generation, a SHA-256 of that collection's canonicalised JSON Schema, and the stable field ids, but never the schema body itself. The bodies stay at _schemas/<collection>, each encrypted under that collection's own DEK. Inlining them would either leak one collection's schema to a principal scoped to a different one, or force the manifest onto a shared key it has no business holding — so the manifest carries metadata and leaves bodies where they are.

An aggregateHash — sha256 over the canonical JSON of the whole per-collection map — binds the index as a unit. Canonical JSON sorts keys at every depth, so the hash does not depend on insertion order: any change to any entry changes aggregateHash too.

The manifest is a derivable cache, not a source of truth. deriveSchemaManifest() is a pure projection of the fence generation plus the _schemas entries the caller can decrypt, so re-deriving against unchanged state always produces the same manifest — which is what makes dump → restore → re-derive match the original with no restore-side special-casing.

When you need it

• Opening a .noydb pod and needing to know, before touching data, whether the reader's code is behind the pod's schema generation
• Verifying that a collection's stored schema still matches the content hash the pod recorded
• Carrying stable field identity (from the schema-identifiers work) across a pod boundary

Opt in

No with*() factory — the surface tree-shakes through direct named imports:

import { open, deriveSchemaManifest, MANIFEST_COLLECTION } from '@noy-db/hub/pod'

Usage

Opening a pod through the full read path

open() composes the whole sequence: read the header → optionally verify its signature → unlock and restore → re-derive the schema manifest → check the generation fence → hand back the vault.

import { open } from '@noy-db/hub'

const { db, vault } = await open(podBytes, {
  user: 'alice',
  secret: 'a-strong-secret',
  trustedKeys,          // omit to skip signature verification
})

If the pod was written by a newer schema generation than the opening code understands, open() throws MigrationRequiredError rather than reading data under the wrong assumptions. A caller that knowingly wants to proceed passes allowGenerationAhead: true.

A pod written before this service existed opens normally — its manifest is simply re-derived from _schemas on the fly.

Reading the manifest directly

import { deriveSchemaManifest } from '@noy-db/hub/pod'

const { manifest, undecodableCollections } = await deriveSchemaManifest(store, 'acme', lookupDEK)

manifest.generation        // fence generation at derivation time
manifest.collections       // per-collection { generation, contentHash, fieldIds? }
manifest.aggregateHash     // binds the whole index

undecodableCollections lists collections that exist in _schemas but which this principal cannot decrypt — see below.

Reserved collection

_manifest is reserved: vault.collection('_manifest') is refused, even for the owner. Records inside are keyed by manifest kind; today only schema exists. Like other reserved collections, it travels inside a pod dump.

Concurrency: refuse, don't merge

Direct manifest edits use strict CAS — refuse, not retry. If the expected version does not match, the write throws ManifestConflictError. This is the one writer in the hub that behaves this way; every other reserved-collection writer retries on conflict.

The divergence is deliberate. A manifest write encodes a specific point-in-time claim about the pod's whole schema generation and content-hash set. Two concurrent direct edits must be surfaced, never silently merged — a caller that lost the race needs to re-derive against fresh state and decide, not have a stale claim quietly re-applied on top of someone else's.

The sync path that runs after a schema write is different: it is a re-derivation triggered by a write that already landed, so it is allowed to retry, and it loops until the manifest provably converges. That matters when several collections are declared in one openVault() call and their schema writes race each other.

Scoped principals see less

A collection-scoped grantee can decrypt _manifest but not a sibling collection's _schemas entry. When that happens, the derivation reports the collections it could not read, and the sync path skips entirely — no compare, no write. Writing a manifest derived from a partial view would drop the invisible siblings and assert an aggregateHash over entries it never verified.

This is safe because the source of truth was already updated by the principal's own declare, open() always re-derives rather than trusting the cache, and the next fully-visible principal's declare refreshes it.

Disclosure surface

Worth stating plainly: _manifest has one DEK for the whole pod, not one per collection, and the system-prefix propagation rule wraps it into every granted principal's keyring regardless of their permission scope.

So any vault principal, however narrowly scoped, can decrypt the entire index — every collection name, every field name, every content hash and generation, including for collections they have no data access to. A principal scoped to invoices can still learn that a salaries collection exists and what its fields are called.

Field values are never exposed. But collection and field naming conventions are themselves sensitive in some domains, and this is an accepted consequence of the index design rather than an oversight. Choose names accordingly.

Availability

Not opted into: nothing writes _manifest, and a pod simply has no manifest record — open() re-derives from _schemas when it needs one.