An embedding is one vector per image: render the image to a square, pass it to the vision encoder, reduce the output to a single vector, and L2-normalize. Everything (invariance, similarity, the floor) compares those vectors by cosine.
The pooling step is load-bearing. image-feature-extraction returns different shapes per model: a pooled [1,512] for CLIP, but raw patch-token grids for SigLIP [1,196,768] and DINOv2 [1,257,384]. Mean-pooling every model's tokens to one vector is what makes them comparable; skipping it compares flattened patch grids, where a rotation just reshuffles token order (the artifact behind the old "patch size" claim). JPEG (q0.9/0.5/0.2) and tiling are extra axes on top of this.
encode_image gives [1,577,768] → mean-pool → 768-d. "An embedding" is one pooled hidden state; the alternative is running that hidden state through the output layer to pick the next word. See the VLM playground.| Model | Provider | Training | Input | Size | Dim |
|---|---|---|---|---|---|
| CLIP ViT-B/32 | OpenAI | Language-contrastive | 224 | ~100MB | 512 (pooled) |
| SigLIP B/16 | Language-contrastive (sigmoid) | 224 | ~370MB | 768 (196 tokens) | |
| DINOv2 Small | Meta | Self-supervised | 224 | ~85MB | 384 (257 tokens) |
| SigLIP B/16 @384 | Language-contrastive | 384 | ~370MB | 768 (576 tokens) | |
| CLIP Large/14 @336 | OpenAI | Language-contrastive | 336 | ~600MB | 768 (pooled) |
| Florence-2-base-ft | Microsoft | VLM (seq2seq) | 768 | ~350MB | 768 (577 tokens) |
npm install node cli.mjs test-images/*.jpg --model clip # human-readable table node cli.mjs test-images/* --model siglip384 # higher-res encoder node cli.mjs test-images/* --model dinov2 --tiles 3 # 3×3 AnyRes tiling node cli.mjs test-images/* --model clip --json > results-clip.json python3 scripts/generate_report.py # regenerate report.html
Models: clip, siglip, dinov2, siglip384, clip-l. Each transform is embedded at raw pixels + JPEG q0.9/0.5/0.2; --tiles N tiles the image N×N. The CLI and browser share lib/experiment.mjs so they measure identically.
vlm.html runs Florence-2 in the browser: caption, read text (OCR), detect objects, extract the image embedding, and compare two images by cosine. It also runs the rotation-invariance test on Florence-2's own embedding.
Gemma 4's vision encoder is a submodel extracted from Gemma 4 E2B, loaded via ONNX Runtime Web (not Transformers.js). It currently fails at ONNX session creation in-browser (a WASM abort (likely an unsupported op in the q4 build), so it's kept as experimental. The other encoders work.
Model blobs (~85–600MB) are cached via the Cache API. Network-first for the app shell so updates appear immediately; model blobs stay cache-first.
Batch report: 15 images × 3 encoders × 20 transforms, with the corrected numbers.
Tiling report: does tiling help the encoder see detail? (it depends on how you combine the tiles).
FINDINGS.md: the write-up.
Patch size does not rank the encoders; that earlier claim was a pooling artifact. With embeddings pooled, SigLIP (patch 16) matches CLIP (patch 32) on rotation (both ~85%); DINOv2 is ~72%. The real story is the baseline: the different-image floor varies a lot (CLIP 58%, DINOv2 15%), so CLIP's high absolute scores are partly a compressed embedding space. Floor-normalized, the encoders are comparable. No encoder is fully transformation-invariant — a rotated image is not treated as identical, because ViT positional embeddings are absolute.