Why does MBPP+ show a dash (—) on Benchmark 2?

Benchmark 2 shows a score only when at least 20 cases completed for that model. Requests that never got an answer aren't counted (they're listed in the raw data). MBPP+ was a 20-problem sample and fewer than 20 completed, so the page shows "—" rather than a number from too few cases. Those completed problems still count in the combined coding score.

Good to know#

• SuperCosmos 1.1 is built to say "I don't know" rather than invent facts. How it's built to be honest
• A request whose prompt is over 262K tokens bills its whole input at 1.5×; output prices don't change. Long-context pricing
• Every tier shares the same 1M-token context window (1,048,576 tokens). Context window

Still need help?#

• General support: support@clatterlabs.com or team@clatterlabs.xyz
• Infrastructure questions, or permanent deletion of your data (your whole account or one service): infrastructure@clatterlabs.xyz

We usually reply within the first hour, and always within 24 hours. A full support team is with you.