Tiling: does the encoder see more detail?
15 images (11 "detail": webpages, text, charts, diagrams, maps, signs, logos) × 3 encoders × 3 tiling levels. AnyRes tiling splits each image into a grid of crops, embeds each at full model resolution, and mean-pools.
The hypothesis, and why it was wrong. A 224px encoder can't read text: a whole webpage is squished to 224, so it sees "a document". The prediction was that tiling (showing each crop at full resolution) would let the model tell documents apart, i.e. lower the detail-image floor. The data shows the opposite: the floor rises with tiling.
Why it rises: mean-pooling averages the detail away. This lab pools the tile embeddings into one fixed-length vector (so cosine stays comparable). Averaging 4 or 9 local crops pulls every image toward a bland mean, so unrelated images look more alike: the floor goes up, and rotation similarity goes up too (rotations get smoothed together). Real AnyRes VLMs don't do this: they concatenate all the tile tokens and feed the lot to the LLM, keeping the detail. The takeaway is about the combining step, not tiling itself: mean-pool tiling trades discrimination for smoothness. Concatenation is the next thing to try.
CLIP (OpenAI)
| Tiling | Floor · all | Floor · detail | Floor · photo | Rotation | Crop | Flip H |
|---|
| 1×1 | 58% | 56% | 68% | 84% | 76% | 95% |
| 2×2 | 71% | 69% | 82% | 93% | 86% | 96% |
| 3×3 | 79% | 77% | 89% | 95% | 90% | 97% |
CLIP (OpenAI): tiling raised the floor 21 pts. Pooling many tiles averaged the representations together, making images look more alike here.
SigLIP (Google)
| Tiling | Floor · all | Floor · detail | Floor · photo | Rotation | Crop | Flip H |
|---|
| 1×1 | 52% | 54% | 53% | 87% | 77% | 94% |
| 2×2 | 64% | 65% | 67% | 93% | 80% | 94% |
| 3×3 | 71% | 72% | 74% | 96% | 84% | 96% |
SigLIP (Google): tiling raised the floor 19 pts. Pooling many tiles averaged the representations together, making images look more alike here.
DINOv2 (Meta)
| Tiling | Floor · all | Floor · detail | Floor · photo | Rotation | Crop | Flip H |
|---|
| 1×1 | 15% | 20% | 13% | 69% | 73% | 97% |
| 2×2 | 22% | 27% | 25% | 78% | 73% | 98% |
| 3×3 | 31% | 35% | 32% | 83% | 75% | 99% |
DINOv2 (Meta): tiling raised the floor 15 pts. Pooling many tiles averaged the representations together, making images look more alike here.
How to read it. "Floor" = mean cosine between unrelated images' originals (a discrimination floor, not invariance; lower is better discrimination). "Detail" vs "photo" splits the floor by whether the image is text/diagram-heavy. Rotation/crop/flip are invariance means (higher = more invariant). Tiling changes both, and not always in the same direction; pooling many tiles can also average away distinctions. Reproduce: node scripts/tiling-experiment.mjs > tiling-results.json && node scripts/generate-tiling-report.mjs.