The reference architecture

Hosting: a static site on GitHub Pages

The whole showcase is static files — HTML, CSS, JS, JSON, and a service worker — served from GitHub Pages under a project path. There is no application server: the catalogue is a models.json the page fetches, and each model lives at its own stable route under models/<slug>/. Weights are pulled at runtime from the Hugging Face CDN (cross-origin) and cached on-device; nothing round-trips through a backend.

origin

https://paulkinlan.github.io (a *.github.io project site)

base path

/web-ai-showcase/ — absolute in-app URLs are written against it

app shell

cached by sw.js for offline; model blobs are not re-stored (see caching)

weights host

huggingface.co/.../resolve/… → Xet CDN, fetched cross-origin with CORS

Static hosting is the point, not a limitation to route around: it's free, it's CDN-backed, it has no secrets to leak, and it forces the honest constraint that all intelligence ships to the client. The trade-offs it does impose — no custom response headers, best-effort storage — are called out explicitly below.

Worker topology: dedicated module workers, a typed protocol, transferables

Inference never runs on the main thread. Each model page owns a dedicated module worker and talks to it over a small, typed message protocol. Model classes and the ONNX runtime are import()-ed inside the worker, so parsing and executing multi-hundred-megabyte graphs can't jank the UI thread.

// front-end helper (e.g. models/yolos-detection/yolos.js, models/sam2-segmentation/sam2.js)
this.worker = new Worker(WORKER_URL, { type: "module" });
this.worker.addEventListener("message", (e) => this._onMessage(e.data));
this.worker.addEventListener("error", (e) => showOnPage(e.message)); // errors surface on the page

The message contract is a small tagged union — every message is { type, id, … }. type names the operation; id correlates each response with the request that triggered it, so a stale reply from a superseded request can be dropped instead of racing the UI. That correlation id is the cancellation and backpressure mechanism: the worker processes messages serially (natural backpressure), the UI ignores any result whose id is no longer current, and a hard cancel is a worker.terminate() + fresh worker. Weight caching is versioned separately, by the model's Hugging Face repo revision (commit sha) — see caching.

// worker.js — the dispatch half of the protocol (from models/sam2-segmentation/worker.js)
self.addEventListener("message", async (e) => {
  const { type } = e.data;
  try {
    if (type === "load") await ensureLoaded();
    else if (type === "embed") await embed(e.data.id, e.data.image);
    else if (type === "segment") await segment(e.data.id, e.data.points, e.data.box);
  } catch (err) {
    post({ type: "error", id: e.data?.id, message: String(err?.message ?? err) });
  }
});

// worker → main messages: { type: "progress" | "ready" | "embedded" | "result" | "error", id, … }
function post(msg, transfer) {
  self.postMessage(msg, transfer || []); // Transferables move buffers zero-copy
}

The per-demo pattern above is now formalised as reusable canonical modules that new demos and retrofit waves build on: lib/worker-protocol.js (a typed, versioned protocol with request ids, transferables both ways, AbortSignal cancellation, stale-response suppression, bounded-queue backpressure, worker lifecycle states, and deterministic dispose + object-URL cleanup); lib/model-download.js (the resumable HTTP-Range downloader described below); lib/capture-ux.js (upload / camera / short webcam video / mic, user-initiated, with full permission and cleanup states); and lib/media-pipeline.js (OffscreenCanvas/ImageBitmap and AudioWorklet preprocessing off the main thread, with the non-isolated fallback that is the default on GitHub Pages).

Large binary results are transferred, not copied. A segmentation worker hands its mask planes back as detached ArrayBuffers in the transfer list, so a multi-megapixel mask crosses the thread boundary with no serialization cost:

post(
  { type: "result", id, width, height, masks: planes, scores, ms },
  planes.map((p) => p.buffer), // transfer list — ownership moves to the main thread
);

Data flow: main ↔ worker ↔ model

The main thread is a controller and a renderer; the worker is where the model lives. Input crosses in (an image as a data URL / ImageBitmap, or text), inference runs in the worker, and results — tokens, boxes, masks, embeddings — cross back out, with progress streamed the whole time so the UI is never a frozen spinner.

Loading is mediated by the shared, accessible loader in lib/model-loader.js (backed by lib/model-cache.js): a valid, current on-device model auto-initialises — returning visitors never click "Load" — while a genuinely absent model, a partially-evicted one, or a newer available revision each get an honest, labelled action. Progress is announced through a role="status" aria-live region, and a failed load offers Retry, never a faked result.

Caching & the resumable-download lifecycle

Model weights are the expensive part — tens to hundreds of megabytes — so cache ownership is deliberate. Transformers.js keeps its own Cache Storage bucket (transformers-cache) and writes every model file there, including fetches the service worker never sees (first visit before control, hard reloads). The service worker therefore serves model files by matching across all caches and never stores a second copy — double-storing gigabyte blobs is exactly what pushes an origin into quota eviction.

// sw.js — serve model blobs cache-first across ALL caches; never double-store
if (MODEL_HOSTS.includes(url.hostname)) {
  e.respondWith(caches.match(req).then((hit) => hit || fetch(req)));
  return; // the library owns transformers-cache; we only cache the small app shell
}

Versioning is by Hugging Face repo revision: lib/model-cache.js records which files a successful load cached plus the repo's commit sha, then on revisit compares recorded files against what's still cached (missing ⇒ evicted ⇒ partial) and the recorded revision against the live one (newer ⇒ optional update). Offline, the cached copy is treated as current so offline use keeps working.

Resumable downloads — validated for this exact deployment

For the large models, restarting a failed download from zero is painful, so the reference design is a real HTTP-Range resume. The download-resume research confirmed every prerequisite holds cross-origin from a *.github.io page against the HF CDN, on files from 23 MB to 786 MB:

• Always fetch the …/resolve/main/… URL. It 302-redirects to the Xet CDN and the browser re-sends your Range header through the redirect; the final response is 206 Partial Content, type:"cors" (not opaque), so JS can read the body, Content-Range, and the strong ETag.
• The ETag is strong (a content hash), so If-Range: <etag> conditional resume works. A changed file answers 200 (full body) instead of 206 — the built-in "restart cleanly" signal.
• Size and integrity are CORS-exposed: X-Linked-Size (total bytes) and X-Linked-ETag (the git-LFS sha256 of the whole file), so the assembled blob can be integrity-checked, not just length-checked.
• No Content-Encoding on the binary payload — byte offsets map 1:1 to file bytes, so offset math is safe.

// Resumable model download — the researched lifecycle (reference implementation).
async function downloadResumable(resolveUrl, store /* IndexedDB */) {
  let { receivedBytes = 0, etag, total } = (await store.get(resolveUrl)) ?? {};
  while (total == null || receivedBytes < total) {
    const res = await fetch(resolveUrl, {
      headers: { Range: `bytes=${receivedBytes}-`, ...(etag ? { "If-Range": etag } : {}) },
    });
    if (res.status === 200) { receivedBytes = 0; }        // file changed → honest restart
    total ??= Number(res.headers.get("content-range")?.split("/")[1] ?? res.headers.get("x-linked-size"));
    etag = res.headers.get("etag") ?? etag;
    for await (const chunk of res.body) {                 // append + persist the partial
      await store.appendChunk(resolveUrl, chunk, (receivedBytes += chunk.byteLength));
    }
  }
  // Completion gate: verify sha256(blob) === X-Linked-ETag before marking "ready".
  if (await sha256Hex(await store.assemble(resolveUrl)) !== expectedSha) throw new Error("integrity mismatch → discard + restart");
}

Isolation-header limits on GitHub Pages

A static host can't set arbitrary response headers, and GitHub Pages won't send Cross-Origin-Opener-Policy / Cross-Origin-Embedder-Policy. Those two headers are the switch for cross-origin isolation, and without them self.crossOriginIsolated is false.

// On GitHub Pages this is false — no COOP/COEP means no cross-origin isolation.
if (!self.crossOriginIsolated) {
  // SharedArrayBuffer is unavailable / non-shareable → multi-threaded WASM can't spin up a thread pool.
  // The non-isolated path (single-thread WASM, or WebGPU) is the DEFAULT here, by design.
}

The practical consequence: multi-threaded WebAssembly (ONNX Runtime Web's threaded backend) relies on SharedArrayBuffer, which requires cross-origin isolation. On this deployment that isn't available, so the non-isolated fallback is the default: models run on single-threaded WASM or, when a real adapter exists, on WebGPU — which is the fast path anyway and needs no SAB. Every page probes for an actual WebGPU adapter (mere "gpu" in navigator is not enough) and falls back to WASM honestly, labelling WebGPU-only models as such rather than faking a result.

Audio, video, and image pipelines

Media decode and pixel work also belong off the main thread, and the platform provides the transferable primitives to keep them there.

images

Decode to an ImageBitmap (transferable) and hand it straight into the worker; render results — overlays, masks — onto an OffscreenCanvas so drawing doesn't compete with the UI. Masks come back as transferred ArrayBuffers and paint via putImageData.

audio

Capture and resample on an AudioWorklet (its own real-time thread), buffer the Float32 PCM the model expects (e.g. 16 kHz mono for Whisper), and post it to the inference worker — never block the audio render quantum with model work.

video

Pull frames as ImageBitmaps (or VideoFrames) and run the per-frame model in the worker, reusing cached per-image state where the model allows it (e.g. SAM's image embedding is computed once, then every prompt is cheap).

The unifying rule: get pixels and samples into a transferable as early as possible, do the heavy work in a worker, and transfer results back — copies and main-thread decodes are the things that jank.

Security & privacy: everything stays on your device

Because there is no backend, there is nowhere for your data to go. Images you drop, audio you record, and text you type are processed entirely in your browser; they are never uploaded, and there are no API keys because there is no API to call. The only network traffic is fetching the model weights from the Hugging Face CDN (and the library JS from jsDelivr) the first time — after which the model is cached and the demo works offline.

• No uploads, no telemetry of your inputs — inference is local, full stop.
• Only canonical weights are loaded — from the model's official HF repo (or MLC / MediaPipe), never arbitrary remote code; env.allowLocalModels is off so the library can't be pointed at a surprise path.
• Secure context — the site is HTTPS, which WebGPU, service workers, and SharedArrayBuffer (where available) all require anyway.
• Integrity on completion — the resumable-download design verifies the assembled weights' sha256 against the server's X-Linked-ETag before a model is marked ready.

Performance budgets

Responsiveness is a budget, not an afterthought. Two numbers frame everything: at 60 fps a frame is ~16 ms, so main-thread work should stay under roughly 8 ms per frame to leave the browser headroom for style, layout, paint, and input; and any single task over 50 ms is a "long task" that directly hurts Interaction to Next Paint (INP). The whole architecture exists to keep the main thread inside those budgets.

When post-processing genuinely has to touch the main thread — decoding a mask into pixels, building thousands of DOM nodes — it's sliced so it can't become a long task. Yield on a ~50 ms deadline, with a setTimeout fallback where scheduler.yield() isn't supported (it's unavailable in Safari):

let deadline = performance.now() + 50; // 50 ms = the long-task boundary
for (const item of items) {
  processItem(item);
  if (performance.now() >= deadline) {
    if ("scheduler" in globalThis && "yield" in scheduler) await scheduler.yield();
    else await new Promise((r) => setTimeout(r, 0)); // fallback: Safari has no scheduler.yield
    deadline = performance.now() + 50;
  }
}

Budgets and techniques follow the project's modern-web-guidance references: performance, break-up-long-tasks, defer-rendering-heavy-content, and deliver-optimized-decorative-images.