stable

Email archive — threads & revisions

Threading, inline-image HTML rendering, and cross-revision document diffing on the email-archive record/blob layout.

Audience: the same apps as Email archive — this page continues that recipe. Bundle: core + withBlobs() (unchanged), plus the diff utilities from @noy-db/hub/history (no service opt-in required).

This page continues the Email archive recipe — it builds threading, HTML rendering, thread-level views, and document-revision tracking on top of the record/blob layout defined there.

Threading without a separate collection

threadId is derived at ingest from References / In-Reply-To. An email that references a parent already in the vault inherits the parent's thread; otherwise this email is the thread root.

import { generateULID, type Collection } from '@noy-db/hub'

async function sha256Hex(input: string): Promise<string> {
  const digest = await crypto.subtle.digest('SHA-256', new TextEncoder().encode(input))
  return Array.from(new Uint8Array(digest), (b) => b.toString(16).padStart(2, '0')).join('')
}

async function deriveThreadId(
  emails: Collection<Email>,
  parsed: ParsedEmail,
): Promise<string> {
  const parents = parsed.headers['references']?.split(/\s+/) ?? []
  for (const parentMessageId of parents) {
    const existing = emails.query()
      .where('messageId', '==', parentMessageId)
      .first()
    if (existing) return existing.threadId
  }
  // New thread — use a messageId hash as a stable id
  return (await sha256Hex(parsed.headers['message-id'] ?? generateULID())).slice(0, 16)
}

Querying a thread is a plain query — synchronous, because the default eager-mode collection holds the decrypted working set in memory:

const email = (await emails.get(id))!          // id from ingestEml above

const thread = emails.query()
  .where('threadId', '==', email.threadId)
  .orderBy('receivedAt', 'asc')
  .toArray()

[
  { "id": "01J1ZC4Q4T9GQ0V4Y3F2M8KDXP", "subject": "Q2 invoice",
    "from": { "email": "ana@example.com", "name": "Ana" },
    "receivedAt": "2026-04-10T09:12:00Z", "threadId": "9f2c41d8a0b7e3c6" },
  { "id": "01J1ZD7N2W5RQ8V0X4H6T3EBSA", "subject": "Re: Q2 invoice",
    "from": { "email": "leo@example.com", "name": "Leo" },
    "receivedAt": "2026-04-10T14:03:00Z", "threadId": "9f2c41d8a0b7e3c6" },
  { "id": "01J1ZF9S6K2TQ4V8Y0B3W7DCRM", "subject": "Re: Q2 invoice",
    "from": { "email": "ana@example.com", "name": "Ana" },
    "receivedAt": "2026-04-11T08:40:00Z", "threadId": "9f2c41d8a0b7e3c6" }
]

Rendering HTML with inline images

Emails embed images via <img src="cid:abc123"> where abc123 matches an attachment's contentId. Resolve at render time. blob(id).get() returns Uint8Array | null — a missing slot is a normal outcome, so handle it:

import { type Collection } from '@noy-db/hub'

function toBase64(bytes: Uint8Array): string {
  let bin = ''
  for (const b of bytes) bin += String.fromCharCode(b)
  return btoa(bin)
}

async function renderHtml(emails: Collection<Email>, emailId: string): Promise<string> {
  const email = await emails.get(emailId)
  if (!email) throw new Error(`email ${emailId} not found`)

  const htmlBytes = await emails.blob(emailId).get('body-html')
  if (!htmlBytes) return email.bodyText ?? email.bodyPreview   // no HTML part stored

  let html = new TextDecoder().decode(htmlBytes)
  for (const att of email.attachments ?? []) {
    if (!att.contentId) continue
    const bytes = await emails.blob(emailId).get(att.slotName)
    if (!bytes) continue                                       // slot missing — leave the cid: ref
    const dataUrl = `data:${att.mimeType};base64,${toBase64(bytes)}`
    html = html.replaceAll(`cid:${att.contentId}`, dataUrl)
  }
  return html
}

const html = await renderHtml(emails, id)     // id from ingestEml above
// → '<html>...<img src="data:image/png;base64,iVBORw0KGgo...">...'
//   (the cid:abc123 reference was replaced with an inline data: URL)

In a browser consumer, prefer emails.blob(id).objectURL(slotName) over building data: URLs by hand — it decrypts the slot and wraps the bytes in a browser object URL sized for <img src> / <a href>, without the base64 blowup of inlining bytes into the HTML string:

const inline = await emails.blob(id).objectURL('att-0-invoice.pdf')
if (inline) {
  imgEl.src = inline.url
  // ...later, when the element is unmounted or the URL is no longer shown:
  inline.revoke()
}

objectURL() returns null for a missing slot, or { url, revoke } for an existing one. You must call revoke() when the URL is no longer needed — until then, both the object URL and the decrypted bytes behind it stay pinned in memory, which leaks on long-lived pages. For a full Response with HTTP metadata headers instead of a URL (e.g. to pipe into a fetch-style API), use blob(id).response(slotName), which returns null for a missing slot the same way.

Thread-level view

threadId on the record is the index. The meaning is the thread. Users don't think "those 7 emails on April 12th"; they think "the thread about the Q2 invoice". Most UI operations fold across the thread: title (first email's subject minus Re:/Fwd:), participants (union across every email), time span, unread / starred disjunction, document set grouped by content address.

Compute these on demand with a query (cheap under ~50 emails per thread). For heavier UIs, maintain a vault.collection<Thread>('threads') as a manually maintained secondary collection, updated as a side-effect of every email put — this is application code you write, not the materialized-views service (that service auto-syncs from a registered query; here the "query" is a handful of fields folded across the thread's emails, which doesn't fit its query/aggregate shape).

Shared documents across a thread

The common case: someone attaches a PDF; three replies quote it back. Because blobs are content-addressed (eTag = HMAC-SHA-256(blob key, plaintext bytes) — keyed, so the storage backend cannot recognise known content), noy-db stores one blob regardless of how many slots reference it. Surface this in the UI by grouping the thread's slots by eTag:

const threadId = '9f2c41d8a0b7e3c6'            // the thread the user is viewing

const thread = emails.query()
  .where('threadId', '==', threadId)
  .toArray()

const byETag = new Map<string, { filename: string; emails: string[] }>()
for (const email of thread) {
  for (const slot of await emails.blob(email.id).list()) {
    if (!slot.name.startsWith('att-')) continue
    const entry = byETag.get(slot.eTag) ?? { filename: slot.filename, emails: [] }
    entry.emails.push(email.id)
    byETag.set(slot.eTag, entry)
  }
}

console.log(Object.fromEntries(byETag))

{
  "9f41c2d8ab07e3…": {
    "filename": "invoice.pdf",
    "emails": ["01J1ZC4Q4T…", "01J1ZD7N2W…", "01J1ZF9S6K…"]
  }
}

Render each group once — "invoice.pdf · appears in 3 emails" — instead of three identical attachment rows.

Revision tracking — different eTags, same logical document

Harder case: someone sends invoice-draft1.pdf, later replies with invoice-draft2.pdf and then invoice-final.pdf. Different bytes, different eTags, but conceptually ONE document across three revisions.

Model this with a DocumentGroup collection — a first-class entity whose identity spans email boundaries:

interface DocumentGroup {
  id: string                    // ULID
  label: string                 // human-readable
  threadId?: string
  filenameStem?: string         // common stem used for matching
  revisions: Array<{
    eTag: string                // from blob(id).list() at ingest
    emailId: string
    slotName: string
    filename: string
    receivedAt: string
    revisionLabel?: string      // 'draft', 'draft2', 'final'
    size: number
    ocrText?: string            // extracted at ingest, if OCR ran — enables diffing
  }>
  latestETag?: string
  createdAt: string
  updatedAt: string
}

Ingest-time revision detection uses three signals:

Byte match — is this eTag already in the thread? It's the SAME file (dedup). Reuse the existing DocumentGroup.
Stem match — strip version suffixes and extension from the filename; matching stem in the thread → new revision of that group.
No match — create a new DocumentGroup. filenameStem becomes the search key for future revisions.

Optional LLM enhancement: when OCR text is available, feed both versions plus the body context into an LLM and ask "is this a revision of that?". Use as confirmation, not primary signal.

Diffing two revisions

If OCR text was extracted at ingest, the hub's diff() / formatDiff() utilities compare the two revisions. They are plain functions exported from the @noy-db/hub/history subpath — no service opt-in required:

import { diff, formatDiff } from '@noy-db/hub/history'

const groups = vault.collection<DocumentGroup>('document_groups')
const groupId = '01J1ZG8B3PV3Q6T9W2K5RD7HCN'   // the DocumentGroup the user selected
const group = await groups.get(groupId)
if (!group || group.revisions.length < 2) throw new Error('nothing to diff')

const [prev, curr] = group.revisions.slice(-2)
const changes = diff({ text: prev!.ocrText ?? '' }, { text: curr!.ocrText ?? '' })
console.log(formatDiff(changes))
// → ~ text: "Total due: 1,200 EUR (draft)" → "Total due: 1,450 EUR"

For binary-only revisions (PDFs with no OCR), diffing is lossy — ship a "visual diff" link that opens both PDFs side-by-side in a viewer instead.