Software Developers

Benchmarks for this occupation

These benchmarks name this occupation in their job mapping.

Also relevant

These benchmarks cover part of this family's work, or have no result yet. They do not set its step.

• DARPA AI Cyber Challenge, Final Competitionby DARPA and ARPA-HProject level · Synthetic vulnerabilities found (scored round)

  43Team Atlanta · 8 Aug 2025

  Strong
• CyberBench v1.1by Vals AITask level · Overall accuracy (mean of PoC and Patch pass rates)

  78%GPT-6 Sol · 7 Oct 2026

  Strong

Benchmarks for the job family

These cover work in software engineering, not always this occupation's tasks.

Direct evidence

Work benchmarks mapped primary to this family. They set the step when they report how many tasks agents complete.

• Aider Polyglot leaderboardby AiderTask level · Percent correct

  88%gpt-5 (high) · 23 Aug 2025

  Strong
• AIDevby Queen's UniversityTask level · PR acceptance rate (AIDev-pop)

  65.3%OpenAI Codex · 20 Jul 2025

  Unrated
• Code Review Benchby MartianTask level · F2 score (Core profile, offline set)

  65.1Qodo Extended · 20 Aug 2026

  Strong
• CodeClashby Stanford University and Princeton UniversityProject level · Elo (all arenas)

  1385 EloClaude Sonnet 4.5 · 3 Nov 2025

  Unrated
• PR Arenaby aavetisTask level · Merge rate of ready PRs

  96%Copilot · 10 Oct 2026

  Unrated
• SWE-Bench Pro V2by Scale AI and ReflectionTask level · Resolve rate (V2 Full)

  99.4%Opus 5 (Claude Code) xhigh · 22 Sep 2026

  Saturated
• SWE-Lancerby OpenAITask level · IC SWE Diamond pass@1

  80%gpt-5.1-codex-max · 18 Nov 2025

  Strong
• SWE-rebenchby NebiusTask level · Resolved rate

  64.5%Anthropic Fable 5 [high] · 1 Jul 2026

  Strong
• Terminal-Bench 4.0by Laude Institute and Stanford UniversityTask level · Resolution rate

  64.8%Claude Code + Opus 5.5 (max) · 6 Oct 2026

  Strong

Also relevant

These benchmarks cover part of this family's work, or have no result yet. They do not set its step.

• BIRD-Interactby HKU and Google CloudTask level · Success rate, a-Interact, Full, both phases

  22.7%MERIT + GPT-5.4 (GPT-4o user simulator) · 22 May 2026

  Emerging
• CyberGymby UC BerkeleyTask level · Success rate (Level 1)

  99.2%Lyrie Agent (multi-model) · 15 Sep 2026

  Saturated
• HWE-Benchby Peking UniversityTask level · Resolved rate

  79.9%GPT-5.6 Sol max (Codex CLI) · 31 Jul 2026

  Strong
• Spider 2.0by HKU and Salesforce AI ResearchTask level · Execution accuracy (Spider 2.0-Snow)

  96.7%Genloop's Sentinel Agent v2 Pro · 1 Mar 2026

  Saturated
• Terminal-Bench 2.0by Laude Institute and Stanford UniversityTask level · Resolution rate

  84.7%NexAU-AHE + GPT-5.5 · 23 Apr 2026

  Retired
• Will It Survive? Agent code survival studyby Concordia UniversityTask level · Line-level modification (death) rate

  No result recorded

  No results yet

Context only

These benchmarks span many occupations, or they test how agents behave. They give context but do not set the step.

• Ambig-SWEby Carnegie Mellon UniversityTask level · Underspecificity detection accuracy (best prompt)

  89%Claude Sonnet 4 · 21 Feb 2026

  Strong
• GDPvalby OpenAIProject level · Wins plus ties against experts

  84.9%GPT-5.5 · Apr 2026

  Strong
• GDPval-AAby Artificial AnalysisProject level · GDPval-AA Elo

  1866 EloClaude Opus 5.5 (Max, Default Fallback) · 10 Oct 2026

  Unrated
• ImpossibleBenchby Carnegie Mellon University and AnthropicTask level · Cheating rate (Conflicting-SWEbench, full scaffold)

  54%GPT-5 · 23 Oct 2025

  Unrated
• MCP Atlasby Scale AITask level · Pass rate

  88.1%Muse Spark 1.1 · 9 Jul 2026

  Strong
• MCPMark Verifiedby EVAL SYS, LobeHub and NUS TRAILTask level · Pass@1 (single run)

  96.1%kimi-k3-max · 20 Jul 2026

  Saturated
• METR Task-Completion Time Horizonsby METRTask level · 50% task-completion time horizon

  17 hClaude Mythos Preview (early) · 7 Apr 2026

  Unrated
• Remote Labor Indexby CAIS and Scale AIProject level · Automation rate

  21.3%Opus 5.5 · 7 Oct 2026

  Emerging
• TheAgentCompanyby Carnegie Mellon UniversityTask level · Tasks resolved

  42.9%TTE-MatrixAgent + DeepSeek-V3.2 · 10 Nov 2025

  Emerging

Retired

Their maintainers or a major evaluator stopped using them as a frontier measure. They do not count.

• SWE-Bench Proby Scale AITask level · Resolve rate (public set)

  61.5%Muse Spark 1.1 · 9 Jul 2026

  Retired
• SWE-bench Verifiedby Princeton University and OpenAITask level · Resolved

  79.2%Sonar Foundation Agent + Claude 4.5 Opus · 5 Dec 2025

  Retired

Typical tasks

From O*NET 31.0. Core tasks first. Few of these tasks have a benchmark today.

• Analyze user needs and software requirements to determine feasibility of design within time and cost constraints.
• Develop or direct software system testing or validation procedures, programming, or documentation.
• Confer with systems analysts, engineers, programmers and others to design systems and to obtain information on project limitations and capabilities, performance requirements and interfaces.
• Modify existing software to correct errors, adapt it to new hardware, or upgrade interfaces and improve performance.
• Prepare reports or correspondence concerning project specifications, activities, or status.
• Analyze information to determine, recommend, and plan installation of a new system or modification of an existing system.
• Store, retrieve, and manipulate data for analysis of system capabilities and requirements.
• Design, develop and modify software systems, using scientific analysis and mathematical models to predict and measure outcomes and consequences of design.
• Determine system performance standards.
• Consult with customers or other departments on project status, proposals, or technical issues, such as software system design or maintenance.
• Confer with data processing or project managers to obtain information on limitations or capabilities for data processing projects.
• Monitor functioning of equipment to ensure system operates in conformance with specifications.