What does the 95% interval on Benchmark 2 mean?

With only about 20 cases, a score is a rough estimate. A 95% Wilson score interval gives the range where the "true" score probably lies. Example from the page: Flash scored 20/22 on honesty (90.9%), with an interval of 72.2% to 97.5%. When two tiers' intervals overlap a lot, you can't say one is really better. That's why the page says differences of a few points are within noise.

Good to know#

• Every tier shares the same 1M-token context window (1,048,576 tokens). Context window
• Line Code was retired on Nov 1, 2026; its ids answer 410 model_retired. Line Code retirement
• Insulting or cursing at the assistant earns a warning, then a 24-hour pause on a second strike within 30 days. Abuse policy

Still need help?#

• General support: support@clatterlabs.com or team@clatterlabs.xyz
• Infrastructure questions, or permanent deletion of your data (your whole account or one service): infrastructure@clatterlabs.xyz

We usually reply within the first hour, and always within 24 hours. A full support team is with you.