What is Benchmark 2?

Benchmark 2 (published Nov 2, 2026 at Benchmark 2) measures SuperCosmos 1.1 Flash, Max and Heavy on what matters when you rely on a model:

1. Honesty: says "I don't know" instead of inventing APIs, papers, flags or facts, and won't confirm what it can't verify, in English and Egyptian Arabic.
2. Coding: writes code that passes each problem's full original and extended tests on HumanEval+ and MBPP+.
3. Efficiency: tokens, response time and cost per task at list prices.
4. Purity: no flattery, no caving: keeps a correct answer under pushback, bribes and threats, and ignores instructions hidden in documents and tool results.

All requests went through the SuperCosmos 1.1 Model API at each tier's default settings, one attempt per case.

Good to know#

• Insulting or cursing at the assistant earns a warning, then a 24-hour pause on a second strike within 30 days. Abuse policy
• SuperCosmos reads text only; it doesn't take image input. What it can't do
• Not sure which tier? auto lets the product choose per step. What auto does

Still need help?#

• General support: support@clatterlabs.com or team@clatterlabs.xyz
• Infrastructure questions, or permanent deletion of your data (your whole account or one service): infrastructure@clatterlabs.xyz

We usually reply within the first hour, and always within 24 hours. A full support team is with you.