SREGym

Built by UIUC and University of Toronto · released 8 May 2026

A live Kubernetes benchmark for SRE agents. Fault injectors break real cloud-native applications (up to 40 microservices), optionally with background noise, and the agent must find the root cause and fix the system. It builds on AIOpsLab and ITBench and adds OS-level, hardware, metastable, and concurrent failures.

Frontier

72.2%

End-to-end success rate

GitHub Copilot (GPT-5.6 Sol, max) · agent: GitHub Copilot

2026 · Source: UIUC (benchmark maintainers)

Full 90-problem suite (cohort SREGym-0508), without noise. Diagnosis 83.3%, mitigation 83.7%. The leaderboard does not date its entries.

Strong

The best result is at 60–90% of the ceiling, or at or above human parity.

See the full leaderboard at UIUC

Dots are recorded results; the line is the best result so far; the red dot is the current frontier. Sources for every point are in the table below.

Embed this card

The image names the institutions behind the numbers and links back to this page.

HTML:

<a href="https://canagentswork.com/benchmarks/sregym/"><img src="https://canagentswork.com/og/benchmarks-sregym.png" width="600" height="315" alt="SREGym: the best result is 72.2% (GitHub Copilot (GPT-5.6 Sol, max), 2026)." loading="lazy"></a>

Markdown:

[![SREGym: the best result is 72.2% (GitHub Copilot (GPT-5.6 Sol, max), 2026).](https://canagentswork.com/og/benchmarks-sregym.png)](https://canagentswork.com/benchmarks/sregym/)

What it measures

End-to-end success: the agent must both diagnose the root cause correctly (an LLM judge with a fixed checklist) and mitigate the failure (checked programmatically) in the same run. Diagnosis and mitigation rates are also reported, along with time to diagnose, time to mitigate, and tokens. The full suite has 90 problems; SREGym-Lite is a 21-problem subset.

End-to-end success rate: Share of runs where diagnosis and mitigation both succeed on the 90-problem suite, without injected noise. Higher is better.

Status compares the frontier with a ceiling of 100%.

Facts

Grain: Task level: bounded tasks with a clear spec
Environment: live-system, cli
Grading: state-check, llm-judge
Tasks: 90
Reuse: MIT (GitHub repository license). Paper is CC BY 4.0. (open-mit)

Limits to keep in mind

Diagnosis is graded by an LLM judge (Claude Sonnet 4.6 in the paper), which adds variance that programmatic checks do not have. Source
Deployed applications are far smaller than production systems; the largest has 40 microservices. Source
Scores fall with injected noise. Claude Code with Sonnet 4.6 drops from 60.7% to 53.7% end to end. Leaderboard entries for newer models are without noise. Source
SREGym-Lite leaderboard scores (best 81.0%) use a 21-problem subset and are not comparable with the 90-problem suite. Source

Recorded results

A selection that shows the frontier over time. The full leaderboard is at the source.

System	End-to-end success rate	Date	Source
GitHub Copilot (GPT-5.6 Sol, max) · frontier	72.2%	2026	UIUC · primary
Stratus (Kimi K2.5)	27.4%	May 2026	UIUC · primary
Codex (GPT-5.4)	53.3%	May 2026	UIUC · primary
Claude Code (Claude Sonnet 4.6)	60.7%	May 2026	UIUC · primary

Timeline

8 May 2026 — SREGym paper released with 90 live SRE problems. Source

Where it sits in the atlas

DevOps, SRE, and IT operations

Incident response Terminal operations

Go to the source

Website sregym.com
Paper arxiv.org
Full leaderboard sregym.com
Code github.com

Last checked 23 Sep 2026 against 5 primary sources. See an error? Tell us.

How to cite

Credit the original work first: SREGym by UIUC and University of Toronto (https://arxiv.org/abs/2605.07161).

Then, if you used this page:

Can Agents Work. "SREGym: frontier results and sources." https://canagentswork.com/benchmarks/sregym/ (accessed 2026-09-24). CC BY 4.0.

@misc{caw-sregym,
  title        = {{SREGym: frontier results and sources}},
  author       = {{Can Agents Work}},
  year         = {2026},
  howpublished = {\url{https://canagentswork.com/benchmarks/sregym/}},
  note         = {Accessed 2026-09-24. CC BY 4.0}
}