blobs

What it does

Blobs let you attach binary files — PDFs, images, audio — to the records of a noy-db vault (the end-to-end-encrypted document store; see Vaults & collections). Each record carries named slots ('receipt', 'avatar', 'contract', …), and like everything in noy-db the bytes are encrypted client-side before the storage backend ever sees them (Enclave).

Storage is content-addressed — a file is identified by a keyed hash of its bytes (eTag = HMAC-SHA-256(blob key, plaintext)), so two records attaching the same file share one stored copy, and the backend cannot recognize known content because the hash is keyed. Large files are split into chunks (256 KB by default), and each chunk is encrypted with AES-256-GCM whose AAD (additional authenticated data — a value the ciphertext is cryptographically bound to) is {eTag}:{index}:{count}: a store that reorders, substitutes, or truncates chunks produces ciphertext that fails to decrypt. MIME types are sniffed from magic bytes when you don't provide one, compressible payloads are gzipped before encryption, and a consumer-scheduled compaction pass enforces retention policies. The whole service is ~2,376 LOC that stay out of your bundle until you opt in.

Prerequisites

• None — withBlobs() alone enables collection.blob(id) slots.
• For erasure flows only: crypto-shredding blob content via forget() pairs historyStrategy: withHistory() (the erasure proof) with forgetStrategy: withForget() — see forget.

Opt in

import { createNoydb } from '@noy-db/hub'
import { withBlobs } from '@noy-db/hub/blobs'

const db = await createNoydb({
  user: 'alice',                    // omitting `store` uses the built-in in-memory store;
  secret: 'alice-secret-2026',  // pass to-file / to-browser-idb / … in production
  blobsStrategy: withBlobs(),
})
const vault = await db.openVault('books')

interface Invoice { id: string; ref: string; reviewed?: boolean }
const invoices = vault.collection<Invoice>('invoices')

Usage

Attach, list, read, delete

collection.blob(id) returns a BlobSet — the handle for every slot on that record. No I/O happens until you call a method.

await invoices.put('inv-001', { id: 'inv-001', ref: 'INV-001' })

const pdfBytes = new Uint8Array(await fetch('/receipt-2026-06.pdf').then(r => r.arrayBuffer()))

const blobs = invoices.blob('inv-001')
await blobs.put('receipt', pdfBytes, {
  mimeType: 'application/pdf',            // omit to auto-detect from magic bytes
  filename: 'receipt-2026-06.pdf',        // display/download name; defaults to the slot name
})

const bytes = await blobs.get('receipt')
// → Uint8Array(48213) — the decrypted original bytes; null if the slot does not exist

const slots = await blobs.list()

[
  {
    "name": "receipt",
    "eTag": "9f2c41d0e6a8b37c51f0d2e94ab6c8d1…",
    "filename": "receipt-2026-06.pdf",
    "size": 48213,
    "mimeType": "application/pdf",
    "uploadedAt": "2026-07-08T09:30:00.000Z",
    "uploadedBy": "alice"
  }
]

await blobs.delete('receipt')
// → void — removes the slot; the underlying chunks are freed once no other
//   slot or published version references the same content

A record can hold any number of slots (.put('receipt', …) and .put('contract', …)), and because storage is content-addressed, uploading the same bytes to two different records stores the chunks once and bumps a reference count.

Serve to a browser

const response = await blobs.response('receipt', { inline: true })
// → Response — body streams the decrypted bytes; headers:
//   Content-Type: application/pdf
//   Content-Length: 48213
//   Content-Disposition: inline; filename="receipt-2026-06.pdf"
//   ETag: "9f2c41d0e6a8b37c51f0d2e94ab6c8d1…"
//   (null if the slot does not exist; omit `inline` for attachment/download)

const built = await blobs.objectURL('receipt')
// → { url: 'blob:https://app.example/1f2e0c9a-…', revoke: [Function] }
//   (null if the slot does not exist)
const img = document.querySelector<HTMLImageElement>('#receipt-preview')!
img.src = built!.url
// later — the caller owns the lifetime:
built!.revoke()

objectURL() throws where URL.createObjectURL is unavailable (Node without DOM, restricted workers). If you use Vue, the useBlobURL composable in @noy-db/in-vue wraps this with auto-revoke on reactive id change and scope dispose, and stays at null during SSR.

Retention & compaction

Declare per-slot policies with the collection's blobFields option, then run vault.compact() whenever your workflow allows — a cron job, a manual "tidy" button, export prep. noy-db never runs compaction on its own.

const invoices = vault.collection<Invoice>('invoices', {
  blobFields: {
    receipt: {
      retainDays: 365,                          // age-based TTL
      evictWhen: (rec) => rec.reviewed === true, // predicate over the decrypted record
    },
  },
})

const preview = await vault.compact({ dryRun: true })

{
  "evicted": 12,
  "records": 480,
  "collections": 1,
  "auditEntries": 0,
  "held": 3,
  "byCollection": { "invoices": { "records": 480, "evicted": 12 } }
}

In a real run (no dryRun) auditEntries equals evicted: every eviction writes one tamper-evident entry to the internal _blob_eviction_audit collection carrying the actor, the reason ('ttl' / 'predicate' / 'both'), and the opaque eTag — no plaintext. held counts slots that were due for eviction but kept by a legalHold or retainUntil floor (see the policy table).

Erasure — blobs shred with forget()

Compaction removes slots on a schedule; forget() removes them on a right-to-erasure request. When you crypto-shred a record with vault.forget(subjectId), the blob content attached to that record is shredded too — each attachment's own content key is destroyed, so its encrypted chunks can never be decrypted again. A blob whose bytes another record still references (same content-addressed eTag) is retained and counted separately:

const result = await vault.forget('buyer-123')
// result.blobsShredded        → 2   attachments whose content key was destroyed
// result.blobsRetainedShared  → 1   attachments kept because another record shares the bytes

By default, this shred runs for every collection in a blob-enabled vault — whether or not the collection declared a blobFields policy. A record that only ever called .blob(id) directly, with no blobFields declaration, still has its blobs shredded on forget().

Scoping the scan with scopedPurge

Since 0.3.0-pre.11, withForget({ …, scopedPurge: true }) turns that default off: under it, forget() scans and shreds blobs only for collections that declared a blobFields policy. An undeclared collection's blobs are skipped entirely — no shred call and no residue-detection listing pass, which is the performance win the opt-in buys. The skip is reported, never silent: ForgetResult.scopedPurgeResidue carries a { reason: 'skipped-undeclared-blob-scan', collection, count } entry, where count is the number of that collection's records whose blob scan was skipped. Those blobs stay untouched until you either declare blobFields for the collection or turn scopedPurge back off.

import { withForget } from '@noy-db/hub/forget'
import { withHistory } from '@noy-db/hub/history'

const db = await createNoydb({
  user: 'alice',
  secret: 'alice-secret-2026',
  blobsStrategy: withBlobs(),
  historyStrategy: withHistory(),                       // forget() writes its erasure proof here
  forgetStrategy: withForget({
    subjects: { invoices: 'buyerId' },
    scopedPurge: true,                                   // only scan blobs on declared collections
  }),
})

The declaration is read from collections opened in the current session: a collection whose blobFields config was never instantiated before the forget() call counts as undeclared and is skipped. Open it (with its config) first.

Tiered blobs — content follows the record's tier

Since 0.3.0-pre.13 (#724/#741) a collection may declare both blobFields and tiers (the record-elevation ladder) — a composition that was previously refused with UnsupportedTierCompositionError. A blob's storage tier equals its owning record's tier: its content key, chunk address, slot-map metadata, and published versions are all keyed under that tier's _blob DEK, so an elevated record's attachment is unreadable at rest by a tier-0 key holder.

Tiered blobs are one face of the record-tier system — the model, the elevation API, and the cross-service map live in tiers.

• Reads are tier-gated at runtime. blob(id) is the tier-0 surface: get(), list(), blobInfo(), and every other accessor treat an elevated record's blobs as nonexistent, for everyone, before any decrypt. The sanctioned cleared path is blob(id).atTier() — it peeks the owning record's live tier and, for an elevated record, runs the same tier-access check elevate()/putAtTier() use (against both the data collection and the _blob collection at that tier) before minting any key material, then returns a BlobSet that reads through the gate. An ungranted caller gets TierNotGrantedError and no key is minted; owner/admin/custodian clear automatically.
• Re-home on elevate/demote is reversible. elevate() re-homes a record's blobs (and published versions) under the destination tier's DEK; demote() restores tier-0 readability. The move is journaled and crash-safe (#746) — a half-moved record heals on the next tier op or blob touch. forget() of an elevated blob-owner crypto-shreds under the record's pre-tombstone tier.
• blobTierPolicy — how a shared blob behaves on elevate. Content-addressed dedup means one stored object can back several records. The collection option blobTierPolicy?: 'isolate' | 'dedup' (default 'isolate') chooses what happens when one co-owner elevates: 'isolate' forks a private tier-scoped copy so the co-owning tier-0 records are untouched; 'dedup' (#741) leaves the shared object in place. Under 'dedup' the runtime read gate still hides the object from a tier-0 caller, but the shared chunks remain decryptable at rest under the flat _blob DEK — a documented, accepted residue (the same class as the aggregate-inference channel). Choose 'dedup' only when at-rest isolation of shared content is not required.
• perRecordKeys is mandatory. A tiered collection that declares blobFields must set perRecordKeys — a legacy blob has no per-record key and cannot be tier-isolated, so writing one to a tiered collection is refused.

import { withTiers } from '@noy-db/hub/tiers'

const db = await createNoydb({
  user: 'alice',
  secret: 'alice-secret-2026',
  blobsStrategy: withBlobs(),
  tiersStrategy: withTiers(),
})
const vault = await db.openVault('books')

const invoices = vault.collection<Invoice>('invoices', {
  tiers: [0, 1, 2],
  perRecordKeys: true,        // required alongside blobFields on a tiered collection
  blobTierPolicy: 'isolate',  // default — fork a private copy of a shared blob on elevate
  blobFields: { receipt: { retainDays: 365 } },
})

Bulk export

vault.exportBlobs(options?) is the framework-agnostic primitive: an async-iterable handle over every blob in the vault, decrypting lazily as you pull. It checks the caller's export capability once at handle creation and writes a single audit entry to _export_audit before the first yield.

const handle = vault.exportBlobs({ collections: ['invoices'] })
for await (const { blobId, recordRef, bytes, meta } of handle) {
  // blobId    → '9f2c41d0e6a8b37c51f0d2e94ab6c8d1…'  (the eTag)
  // recordRef → { collection: 'invoices', id: 'inv-001', slot: 'receipt' }
  // bytes     → Uint8Array(48213) — decrypted plaintext
  // meta      → { size: 48213, filename: 'receipt-2026-06.pdf',
  //               mimeType: 'application/pdf', createdAt: '2026-07-08T09:30:00.000Z' }
  // …write `bytes` to your sink: zip stream, S3 multipart, cold storage
}

For materializing into a real directory, @noy-db/to-file ships exportBlobsToDirectory, which adds filename sanitization, path-escape containment (throws PathEscapeError), and collision handling:

import { exportBlobsToDirectory } from '@noy-db/to-file'

const result = await exportBlobsToDirectory(vault, './out', {
  filenameProfile: 'macos-smb',   // the most restrictive default
  onCollision: 'suffix',          // 'overwrite' | 'fail' | (existing, attempt) => string
})

{
  "written": 2,
  "bytes": 96426,
  "entries": [
    { "blobId": "9f2c41d0e6a8…", "path": "/work/out/receipt-2026-06.pdf" },
    { "blobId": "4b8e02c7d19f…", "path": "/work/out/contract-signed.pdf" }
  ],
  "manifestPath": null
}

The 'opaque' profile renames every file to {blobId}.{ext} and writes a sidecar manifest.json mapping the opaque names back to the originals — useful for handing a directory to a recipient who should not learn filenames from the listing alone:

const opaque = await exportBlobsToDirectory(vault, './out', { filenameProfile: 'opaque' })
// → opaque.manifestPath === '/work/out/manifest.json'

{
  "format": "noydb-opaque-export",
  "version": 1,
  "entries": [
    {
      "opaqueName": "9f2c41d0e6a8b37c51f0d2e94ab6c8d1….pdf",
      "originalName": "receipt-2026-06.pdf",
      "collection": "invoices",
      "recordId": "inv-001",
      "slot": "receipt",
      "blobId": "9f2c41d0e6a8b37c51f0d2e94ab6c8d1…",
      "mimeType": "application/pdf"
    }
  ]
}

The standalone sanitizer lives in @noy-db/hub/util for adopters writing their own sinks:

import { sanitizeFilename } from '@noy-db/hub/util'

sanitizeFilename('report: Q2/final?.pdf', { profile: 'windows' })
// → 'report_ Q2_final_.pdf'

sanitizeFilename('CON.txt', { profile: 'windows' })
// → '_CON.txt'  (Windows reserved device name defused)

sanitizeFilename('scan-'.repeat(80) + '.pdf', { profile: 'posix', maxBytes: 240 })
// → 'scan-scan-…' truncated to 240 bytes on a code-point boundary — headroom for '-1', '-2', … suffixes

API reference

BlobSet — from collection.blob(id)

The subpath also exports the MIME helpers detectMimeType / detectMagic / isPreCompressed and the internal-collection constants (BLOB_COLLECTION, BLOB_INDEX_COLLECTION, BLOB_CHUNKS_COLLECTION, DEFAULT_CHUNK_SIZE, …).

BlobPutOptions

BlobResponseOptions

blobFields policy

Declared per collection: vault.collection(name, { blobFields: { [slotName]: policy } }).

vault.compact(options?) accepts { now?, maxEvictions?, dryRun? } and returns a CompactionResult (evicted, records, collections, auditEntries, held, byCollection).

vault.exportBlobs(options?)

Returns an ExportBlobsHandle — AsyncIterable<ExportedBlob> plus abort() and a readonly aborted flag. Requires the caller's keyring to grant plaintext blob export; aborting makes the next iteration throw ExportBlobsAbortedError.

exportBlobsToDirectory(vault, targetDir, options?)

Returns { written, bytes, entries, manifestPath }; manifestPath is the absolute path of manifest.json for the 'opaque' profile, null otherwise.

sanitizeFilename(name, options)

Pure string in / string out; throws FilenameSanitizationError when the input cannot be made safe (NUL byte, .. traversal segment, empty after normalization, missing opaqueId for the opaque profile).

Always-on transforms across all profiles: NFC normalization, bidi-override stripping (defeats ‮-style extension spoofing), NUL rejection (hard fail — silent stripping would enable truncation bypass), and trimming of leading/trailing whitespace and control characters.

Behavior when NOT opted in

• collection.blob(id) throws an error pointing you at createNoydb({ blobsStrategy: withBlobs() }) and the @noy-db/hub/blobs import.
• All blob machinery stays out of the bundle — ~2,376 LOC saved.
• vault.compact() with no blobFields declared anywhere is a no-op returning zeros.

Limits & gotchas

• response() is buffered, not truly streaming — all chunks are loaded into memory before the body is enqueued. Budget accordingly for very large files.
• objectURL() is browser-only — it throws where URL.createObjectURL is missing — and the caller owns revoke(); an unrevoked URL pins the decrypted bytes for the lifetime of the document.
• Chunk size resolution: explicit chunkSize on put() > the store's advertised maxBlobBytes (e.g. 256 KB for DynamoDB-backed stores) > the 256 KB default.
• Mutating a slot publishes new content — blobs are content-addressed and immutable; overwriting a slot points it at a new eTag and releases the old one. Use publish(slot, label) to pin a version that survives later overwrites.
• Compaction is consumer-scheduled — no background daemon. Policy predicates are fail-closed: a throwing evictWhen never evicts, a throwing legalHold/retainUntil holds the slot.
• presignedUrl() serves ciphertext, works only for single-chunk blobs on stores that support presigning, and must be paired with decryptResponse() (or a service worker) client-side.
• external blob fields are unencrypted by design — they exist to serve assets directly from S3/CDN and sit outside the zero-knowledge guarantee. Keep sensitive attachments on the default encrypted-chunk path.
• Tiered blobs need perRecordKeys, and blobTierPolicy: 'dedup' keeps shared content under the flat DEK. A blobFields + tiers collection must set perRecordKeys; dedup policy hides a shared blob from tier-0 callers at the read gate but leaves its chunks decryptable at rest under the flat _blob DEK. Use 'isolate' (the default) when at-rest isolation of shared content matters. See Tiered blobs.
• A blobFields slot is not queryable. Since 0.3.0-pre.9, a declared blob field carries an enforced queryable: 'none' posture (see via): query().where() on it throws FieldNotQueryableError instead of silently matching nothing. Query record metadata, not blob slots.
• forget() shreds blobs on every collection by default — not only collections that declared blobFields. A collection that only ever wrote blobs via .blob(id) still has them crypto-shredded on erasure. Opt into scopedPurge: true (see Erasure — blobs shred with forget()) to scan only declared collections; undeclared ones are then skipped and reported, never silently.

See also

• Accounting app — invoices with attached receipt scans, retention, and export in one recipe
• bundle — .noydb containers carry blobs alongside records
• history — slot metadata is itself versioned envelope data
• routing — send blob chunks to a separate backend (S3 / R2) while record metadata stays on the primary store
• satellites — the sibling heavy-field strategy for large structured/text fields that still need schema validation and query
• via — the field-feature posture a blobFields declaration carries
• forget — right-to-erasure crypto-shredding; how attached blobs shred with a record and the scopedPurge scan-scoping knob
• Stores · Enclave
• Services overview · Orientation map
• Proven by showcase 08-with-blobs.