Computer Systems Engineers/Architects

Benchmarks for the job family

These cover work in devops, sre, and it operations, not always this occupation's tasks.

Direct evidence

Work benchmarks mapped primary to this family. They set the step when they report how many tasks agents complete.

• AIOpsLabby Microsoft Research, UIUC, UC Berkeley and IIScTask level · Accuracy

  59.3%Flash · 12 Jan 2025

  Emerging
• IaC-Evalby University of Michigan and Cisco ResearchTask level · pass@1

  36.7%GPT-4 with retrieval-augmented generation · 2024

  Emerging
• ITBenchby IBM Research and UIUCTask level · SRE incidents resolved

  25%ITBench-SRE-Agent-GPT-4o · 2 May 2025

  Emerging
• ITBench-AAby Artificial Analysis and IBM ResearchTask level · ITBench-AA score

  56.2%GPT-5.6 Sol (Max) · 10 Oct 2026

  Emerging
• SREGymby UIUC and University of TorontoTask level · End-to-end success rate

  72.2%GitHub Copilot (GPT-5.6 Sol, max) · 3 Aug 2026

  Strong

Also relevant

These benchmarks cover part of this family's work, or have no result yet. They do not set its step.

• EnterpriseOps-Gymby ServiceNow, Mila and Université de MontréalTask level · Task success rate

  45.9%Claude Opus 4.6 · 16 Mar 2026

  Emerging
• OSWorld-Verifiedby HKU, Salesforce AI Research, Carnegie Mellon University and University of WaterlooTask level · Success rate

  90.2%Intelligence-Indeed Agent · 25 Jul 2026

  Saturated
• Terminal-Bench 4.0by Laude Institute and Stanford UniversityTask level · Resolution rate

  64.8%Claude Code + Opus 5.5 (max) · 6 Oct 2026

  Strong
• WorkArenaby ServiceNowTask level · WorkArena-L1 success rate

  63.3%GenericAgent-Claude-4-Sonnet · 7 Aug 2025

  Strong

Retired

Their maintainers or a major evaluator stopped using them as a frontier measure. They do not count.

• Terminal-Bench 2.0by Laude Institute and Stanford UniversityTask level · Resolution rate

  84.7%NexAU-AHE + GPT-5.5 · 23 Apr 2026

  Retired

Typical tasks

From O*NET 31.0. Core tasks first. Few of these tasks have a benchmark today.

• Communicate with staff or clients to understand specific system requirements.
• Investigate system component suitability for specified purposes, and make recommendations regarding component use.
• Provide customers or installation teams guidelines for implementing secure systems.
• Direct the analysis, development, and operation of complete computer systems.
• Direct the installation of operating systems, network or application software, or computer or network hardware.
• Monitor system operation to detect potential problems.
• Identify system data, hardware, or software components required to meet user needs.
• Perform ongoing hardware and software maintenance operations, including installing or upgrading hardware or software.
• Verify stability, interoperability, portability, security, or scalability of system architecture.
• Research, test, or verify proper functioning of software patches and fixes.
• Configure servers to meet functional specifications.
• Collaborate with engineers or software developers to select appropriate design solutions or ensure the compatibility of system components.