Research · Evaluations
Framework first. Results when they exist.
The evaluation framework is defined and the first runs are in progress. We publish results as runs complete — including unflattering ones. There are no benchmark numbers on this page yet because we will not invent them.
What will be published
- RunsConfiguration, date, and environment for each evaluation run
- ScoresTask-level results with error bars where sample sizes allow
- FailuresDocumented failure cases, not just averages
Track progress in the lab evaluations section.