Why does Benchmark 2 say the samples are small?

Each suite has roughly 20–33 cases, so one case changes a score by 3–5 points. The page says so plainly: differences of a few points are within noise. It gives 95% Wilson score intervals for every score in its summary data; for example, Flash's 20/22 on honesty is 90.9%, with an interval of 72.2%–97.5%.

Read the results as "all three tiers are strong at honesty and resisting pressure, Heavy slightly more so", not as precise rankings.

Good to know#

• Clatter Labs publishes its measurements of honesty, coding, efficiency and purity. Benchmark 2 results
• SuperCosmos reads text only; it doesn't take image input. What it can't do
• Line Code was retired on Nov 1, 2026; its ids answer 410 model_retired. Line Code retirement

Still need help?#

• General support: support@clatterlabs.com or team@clatterlabs.xyz
• Infrastructure questions, or permanent deletion of your data (your whole account or one service): infrastructure@clatterlabs.xyz

We usually reply within the first hour, and always within 24 hours. A full support team is with you.