Paul’s learnings

Kev, in this tab.
Nothing sent.

A 0.6 billion parameter decision model, downloaded to your browser and run on your machine. Ask it typed questions and read the distribution.

Nothing you type leaves this tab. The page downloads model weights from huggingface.co — that is the only network request it makes — and the runtime is served from this site rather than a third-party CDN. No Jev key is used, there is no server, and no account is involved. The cost is the download: about 375 MB, once, cached by the browser afterwards.

This runs on the CPU/WebAssembly build of ONNX Runtime. The WebGPU build is a further 27 MB binary and is not vendored here, so no WebGPU path is offered and none is claimed. Why this page exists, and the trap in the model card.

Nothing has been downloaded. The model does not load on page load.

Press the button to begin. Expect a slow first load on a slow connection.

The finding this page produced

Ordering holds where calibration does not. On the cases above this model is sharp: a double-charge ticket routes to billing at 0.995, a refund question lands at 0.749, and it still routes correctly when the option list is reversed, which is what proves the readout indexes the right tokens.

But on a question shape unlike its training data the picture changes. Asking whether a message is trying to obtain a password or card number, a genuine credential grab scored 0.142 while a harmless message scored 0.055. The model ranked those correctly — the grab above the harmless one — and both sat far below the midpoint. A threshold at 0.5 answers “no” to both and throws the correct ranking away. Read the ranking before trusting the threshold, and move a threshold only against labelled data. This is the failure mode the upstream card describes as uneven on product-shaped questions with no training analogue, and it is why the exercises above include the negative cases: a model that answers yes to everything passes half of them.

Measured in Node on the 22nd of September 2026 on this machine: 12 of 13 known-answer cases matched, latency 33–82 ms per decision on the CPU backend, warm. The browser numbers are printed above from your own run. Latency depends on the device; treat any single figure as one machine''s measurement.