AIOpsLab

Built by Microsoft Research, UIUC, UC Berkeley and IISc · released 12 Jan 2025

A framework from Microsoft Research and university partners that deploys microservice applications on Kubernetes, injects faults, generates load, and exposes telemetry so that AI agents can be tested on cloud operations tasks. The paper ships a suite of 48 problems.

Frontier

59.3%

Accuracy

Flash

12 Jan 2025 · Source: Microsoft Research (benchmark maintainers)

Best accuracy in the paper. Flash is the authors' simplified re-implementation of a workflow agent with hindsight generation; the paper does not state its backbone model in the results table. The maintainers' leaderboard lists "FLASH (GPT-4)" at 59.27, the mean of its four task-type accuracies, not the share of all problems (see data/conflicts/aiopslab-flash-accuracy.yaml).

Emerging

The best result is at 20–60% of the ceiling, or at 50–100% of human parity.

See the full leaderboard at Microsoft Research

Dots are recorded results; the line is the best result so far; the red dot is the current frontier. Sources for every point are in the table below.

Embed this card

The image names the institutions behind the numbers and links back to this page.

HTML:

<a href="https://canagentswork.com/benchmarks/aiopslab/"><img src="https://canagentswork.com/og/benchmarks-aiopslab.png" width="600" height="315" alt="AIOpsLab: the best result is 59.3% (Flash, 12 Jan 2025)." loading="lazy"></a>

Markdown:

[![AIOpsLab: the best result is 59.3% (Flash, 12 Jan 2025).](https://canagentswork.com/og/benchmarks-aiopslab.png)](https://canagentswork.com/benchmarks/aiopslab/)

What it measures

Whether an agent answers each operations problem correctly. Problems sit at four levels: detect that something is wrong, localize the faulty service, analyze the root cause, and mitigate the fault in the live cluster. The headline is accuracy across all 48 problems. The framework also records time, steps, and tokens.

Accuracy: Share of the 48 problems that the agent solves correctly across detection, localization, root cause analysis, and mitigation. Higher is better.

Status compares the frontier with a ceiling of 100%.

Facts

Grain: Task level: bounded tasks with a clear spec
Environment: live-system, cli, api-tools
Grading: state-check, automated-tests
Tasks: 48
Reuse: MIT (GitHub repository license). (open-mit)

Limits to keep in mind

Published results cover agents built on GPT-4-turbo and GPT-3.5-turbo from early 2025. The maintainers' leaderboard was last updated in June 2025, ranks agents by the mean of four task-type accuracies rather than the share of all problems, and is no longer linked from the project site. Source
The problem set in the paper is 48 problems from a small number of faults across a few applications. The repository has since added more problems. Source
SREGym, which ports the AIOpsLab problems, reports that strong agents now mitigate more than 80% of these ported problems. Source
Until a fix on 2026-09-14, the mitigation check read pod health at the instant of submission, so some correct fixes counted as failures. The fix waits up to 120 seconds; the paper's results predate it. Source

Recorded results

A selection that shows the frontier over time. The full leaderboard is at the source.

System	Accuracy	Date	Source
GPT-3.5-w-Shell harness: shell-only baseline	15.3%	12 Jan 2025	Microsoft Research · primary
GPT-4-w-Shell harness: shell-only baseline	49.1%	12 Jan 2025	Microsoft Research · primary
ReAct	55.9%	12 Jan 2025	Microsoft Research · primary
Flash · frontier	59.3%	12 Jan 2025	Microsoft Research · primary

Where sources disagree

The AIOpsLab paper gives Flash 59.32% accuracy across all problems. The maintainers' leaderboard gives "FLASH (GPT-4)" an average of 59.27. The same leaderboard ranks OpsAgent (GPT-4) first at 78.75. We show the paper value.

59.32 — arxiv.org (primary) · Paper Table 3, accuracy across all problems, read again 2026-10-10.
59.27 — microsoft.github.io (primary) · Leaderboard row "FLASH (GPT-4)", Average 59.27 with Detection 100, Localization 46.15, Diagnosis 36.36, and Mitigation 54.55. Seen 2026-10-10.
78.75 — microsoft.github.io (primary) · Leaderboard row "OpsAgent (GPT-4)", AIOpsLab & NKU, added 2025-06-10, with no linked paper. Its four task scores (100, 60, 83.34, 66.67) average 77.50, not 78.75. Seen 2026-10-10.

We show 59.32. Status: resolved.

Timeline

14 Sep 2026 — AIOpsLab mitigation check now waits up to 120 seconds, so correct fixes stop counting as failures. Source
12 Jan 2025 — AIOpsLab paper released with 48 cloud operations problems. Source

Where it sits in the atlas

Work ladder: Direct evidence for DevOps, SRE, and IT operations. On the work ladder it counts as 59.3% of tasks completed. How the ladder works

Work it measures (O*NET work activities): Monitor operation of computer or information technologies; Resolve computer problems.

DevOps, SRE, and IT operations

Incident response Terminal operations

Go to the source

Website microsoft.github.io
Paper arxiv.org
Full leaderboard microsoft.github.io
Code github.com

Last checked 10 Oct 2026 against 8 primary sources. See an error? Tell us.

How to cite

Credit the original work first: AIOpsLab by Microsoft Research, UIUC, UC Berkeley, and IISc (https://arxiv.org/abs/2501.06706).

Then, if you used this page:

Can Agents Work. "AIOpsLab: frontier results and sources." https://canagentswork.com/benchmarks/aiopslab/ (accessed 2026-10-11). CC BY 4.0.

@misc{caw-aiopslab,
  title        = {{AIOpsLab: frontier results and sources}},
  author       = {{Can Agents Work}},
  year         = {2026},
  howpublished = {\url{https://canagentswork.com/benchmarks/aiopslab/}},
  note         = {Accessed 2026-10-11. CC BY 4.0}
}