Benchmarks / GDPval-AA
GDPval-AA
Built by Artificial Analysis · released Dec 2025
Artificial Analysis runs OpenAI's public 220-task GDPval gold set through its own agent harness, Stirrup, and ranks models by Elo from blind pairwise comparisons of the deliverables. The judges are a panel of three frontier language models, not human experts.
Frontier
1866 Elo
GDPval-AA Elo
Claude Opus 5.5 (Max, Default Fallback) · harness: Stirrup
10 Oct 2026 · Source: Artificial Analysis (benchmark maintainers)
Rank 1 of 288 models on the GDPval-AA v2.1 board, seen 2026-10-10. Confidence interval shown as -26 / +26. Model release date shown as 2026-09-22. Claude Sonnet 5.5 (Max, Default Fallback, 1838) and Claude Opus 5.5 (Xhigh, Default Fallback, 1836) follow within the interval. Judged by a panel of three frontier LLMs, not humans. Updated 2026-10-10: the board refit moved this from 1846 (seen 2026-09-23, then labeled "Adaptive Reasoning, Max Effort, Default Fallback") to 1866; the anchor is unchanged (see the conflict file).
No fixed reference point (for example Elo scores or field signals).
What it measures
How well a model, acting as an agent with a Linux sandbox, code execution, web search, web fetch, and image viewing, produces work deliverables (documents, slides, spreadsheets, media) for the 220 GDPval gold tasks across 44 occupations. A judge sampled from a panel of three frontier LLMs blindly picks the better of two submissions to the same task. Ratings are fitted with a Crowd-BT model and reported as Elo with 95% confidence intervals.
GDPval-AA Elo: Crowd-BT rating from blind pairwise LLM-judge comparisons of deliverables. Anchored to DeepSeek V4.1 Flash (max) at 1600 in v2.1. Ties count as half a win for each side. Higher is better.
No fixed reference point, so we do not rate its status. Elo scale with a model anchor. Scores are relative and shift when the anchor changes.
Facts
- Grain
- Project level: whole projects judged by an acceptance standard
- Environment
- cli, api-tools, documents
- Grading
- llm-judge
- Tasks
- 220
- Human reference
- None on the current v2.1 scale. In GDPval-AA v2 (June to September 2026) the scale was anchored so that human expert performance was 1000 Elo. v2.1 re-anchored the scale to DeepSeek V4.1 Flash (max) at 1600, so the human anchor no longer applies.
- Contamination
- Uses the public GDPval gold subset, so models may have seen the tasks. Artificial Analysis has not published a contamination analysis.
- Reuse
- Tasks are OpenAI's public GDPval gold set. The Stirrup harness is open source on GitHub. Artificial Analysis publishes Elo scores and example submissions on its site. (cite-only)
Limits to keep in mind
- Judges are language models (Claude Opus 5, GPT-5.6 Sol, Gemini 3.8 Flash in September 2026), not the human experts used in OpenAI's own GDPval grading. Source
- The Elo scale has been re-anchored twice (v2 in June 2026 to human experts at 1000, v2.1 in September 2026 to DeepSeek V4.1 Flash at 1600). Scores from different versions are not comparable. Source
- Each model runs once per task, and 95% confidence intervals are about plus or minus 20 to 30 Elo, so nearby ranks can swap. Source
- Many entries are the same model at different reasoning-effort settings, so the board mixes model quality with compute budget. Source
Recorded results
A selection that shows the frontier over time. The full leaderboard is at the source.
| System | GDPval-AA Elo | Date | Source |
|---|---|---|---|
| GPT-5 (High) harness: Stirrup | 917 Elo | 10 Oct 2026 | Artificial Analysis · primary |
| Claude Opus 4.7 (Max) harness: Stirrup | 1353 Elo | 10 Oct 2026 | Artificial Analysis · primary |
| GPT-5.6 Sol (Max) harness: Stirrup | 1609 Elo | 10 Oct 2026 | Artificial Analysis · primary |
| Claude Opus 5 (Max) harness: Stirrup | 1722 Elo | 10 Oct 2026 | Artificial Analysis · primary |
| Claude Opus 5.5 (Max, Default Fallback) · frontier harness: Stirrup | 1866 Elo | 10 Oct 2026 | Artificial Analysis · primary |
Where sources disagree
Anthropic's Claude Opus 5.5 post gives 1846 Elo on GDPval-AA v2.1. The live Artificial Analysis board now shows 1866 for the same entry. Both are v2.1 with the same anchor.
- 1866 — artificialanalysis.ai (primary) · Claude Opus 5.5 (Max, Default Fallback), -26 / +26, board seen 2026-10-10.
- 1846 — www.anthropic.com (lab-reported) · Opus 5.5 at max effort in Anthropic's launch post, which matches the board as we read it on 2026-09-23.
We show 1866. Status: resolved.
Timeline
Where it sits in the atlas
Work ladder: Context only. It spans many occupations, so it does not set the step of a job family. How the ladder works
Management and business operationsFinance and accounting (partial)Legal (partial)Software engineering (partial)Sales and marketing (partial)Office and administrative support (partial)Healthcare (partial)Design, media, and writing (partial)Architecture and engineering (partial)
Go to the source
- Website artificialanalysis.ai
- Paper arxiv.org
- Full leaderboard artificialanalysis.ai
- Code github.com
- Announcement artificialanalysis.ai
- Dataset huggingface.co
Last checked 10 Oct 2026 against 5 primary sources. See an error? Tell us.
How to cite
Credit the original work first: GDPval-AA by Artificial Analysis (https://arxiv.org/abs/2510.04374).
Then, if you used this page:
Can Agents Work. "GDPval-AA: frontier results and sources." https://canagentswork.com/benchmarks/gdpval-aa/ (accessed 2026-10-11). CC BY 4.0.@misc{caw-gdpval-aa,
title = {{GDPval-AA: frontier results and sources}},
author = {{Can Agents Work}},
year = {2026},
howpublished = {\url{https://canagentswork.com/benchmarks/gdpval-aa/}},
note = {Accessed 2026-10-11. CC BY 4.0}
}