Why does Benchmark 2 use medians for time and tokens?

Output tokens and time per task are shown as medians: the middle value. A few unusually long answers would pull an average up a lot; the median shows what a typical task looks like (Flash 57 tokens and 3.5 s; Max 206 and 7.2 s; Heavy 204 and 8.7 s). Cost per 1,000 tasks uses the actual totals at list prices.

Good to know#

• Clatter Labs publishes its measurements of honesty, coding, efficiency and purity. Benchmark 2 results
• SuperCosmos reads text only; it doesn't take image input. What it can't do
• Every tier shares the same 1M-token context window (1,048,576 tokens). Context window

Still need help?#

• General support: support@clatterlabs.com or team@clatterlabs.xyz
• Infrastructure questions, or permanent deletion of your data (your whole account or one service): infrastructure@clatterlabs.xyz

We usually reply within the first hour, and always within 24 hours. A full support team is with you.