How do I compare tiers on the same task?

1. Pick a realistic task with a clear check: "Make the failing tests pass and show the test output."
2. Start two runs with the same input, one with "tier": "flash" and one with "tier": "max". Tag them with metadata, like {"experiment": "tier-test"}.
3. When both finish, compare:
   • Did it meet the "done" condition?
   • steps: how many steps it took.
   • usage.tokens: tokens used.
   • Cost in the Console's Usage page.
4. Repeat on a few tasks before you change your default.

Remember Max tokens cost more per token, but fewer steps can make a run cheaper overall.

Learn more#

• Choose a tier
• SuperCosmos models