Benchmark Hall
70 benchmarks, each built by the institutions named on its page. We show the current frontier result and link to the full leaderboard at the source. Grouped by status: from open problems to benchmarks that the frontier has already passed.
By function: Issue resolution 8Long-horizon autonomy 8Enterprise workflows 7Feature development 7Vulnerability research 6Web research 6Incident response 5Professional deliverables 5Customer service 4Data engineering and SQL 4Legal work 4Spreadsheet work 4Terminal operations 4Clinical records work 3Computer use 3Financial analysis 3Robot manipulation 3Shipping pull requests 3Asking for clarification 2Codebase evolution 2CRM operations 2Freelance projects 2Household chores 2MCP tool use 2ML engineering 2Penetration testing 2Research replication 2Vulnerability patching 2Autonomous R&D 1Bookkeeping and month-end close 1Business workflow automation 1CAD modeling 1Code review 1Compliance operations 1Consulting work 1Contract redlining 1Driving 1FinOps 1Hardware design and debugging 1Infrastructure as code 1Legal research 1Medical image reading 1Performance engineering 1Procedure execution 1Running a business 1Tax research 1Test integrity 1Tutoring 1
Emerging 22
The best result is at 20–60% of the ceiling, or at 50–100% of human parity.
- BIRD-Interactby HKU and Google Cloud22.7%MERIT + GPT-5.4 (GPT-4o user simulator) · 22 May 2026Emerging
- 45.9%Claude Opus 4.6 · 16 Mar 2026Emerging
- 36.7%GPT-4 with retrieval-augmented generation · 2024Emerging
- 54.5%arito · 13 Sep 2026Emerging
- 42.9%TTE-MatrixAgent + DeepSeek-V3.2 · 10 Nov 2025Emerging
Strong 23
The best result is at 60–90% of the ceiling, or at or above human parity.
- 22Claude Mythos Preview (early checkpoint) · 13 Apr 2026Strong
- ARTEMIS live penetration test (agents vs. professionals)by Stanford University, Carnegie Mellon University and Gray Swan AI95.2ARTEMIS A2 (ensemble supervisor, Claude Sonnet 4 sub-agents) · 10 Dec 2025Strong
- 61%Claude Opus 5.5 (max) with Claude Code 2.1.280 · 24 Sep 2026Strong
- 64.8%Claude Code + Opus 5.5 (max) · 6 Oct 2026Strong
Unrated 9
No fixed reference point (for example Elo scores or field signals).
- 1385 EloClaude Sonnet 4.5 · 3 Nov 2025Unrated
- 1735 EloDreamZero (dreaming_zebra) · 24 Sep 2026Unrated
- 82%Waymo Driver (rider-only, 271.3M miles through June 2026) · Jun 2026Unrated
No results yet 1
We have not recorded a result yet.
Saturated 10
The best result is at 90% or more of the ceiling.
- OSWorld-Verifiedby HKU, Salesforce AI Research, Carnegie Mellon University and University of Waterloo90.2%Intelligence-Indeed Agent · 25 Jul 2026Saturated
- 96.7%Genloop's Sentinel Agent v2 Pro · 1 Mar 2026Saturated
Solved 1
Its maintainers or a major evaluator declared it solved.
- 95.5%Claude Code + Claude Opus 4.5 (after manual regrading) · 3 Dec 2025Solved
Retired 4
Maintainers or a major user stopped using it as a frontier measure.
- 79.2%Sonar Foundation Agent + Claude 4.5 Opus · 5 Dec 2025Retired
- 84.7%NexAU-AHE + GPT-5.5 · 23 Apr 2026Retired
Status rules are on the method page.