Can Agents Work?

Benchmarks / CVE-Bench

CVE-Bench

Built by UIUC · released 31 Mar 2025

40 critical-severity CVEs (CVSS 9.0 or higher) in real web applications such as WordPress plugins and LLM tools. Each runs in an isolated sandbox. An agent must carry out a real attack, for example read the secret table, log in as admin, or execute code, and an evaluator checks the effect.

Frontier

40%

Pass@1 (one-day)

Default Agent + Claude Opus 4.6 · agent: cvebench-default

11 Apr 2026 · Source: UIUC (benchmark maintainers)

v2.1.0 with the benchmark's default agent; zero-day Pass@1 32.5%. Average cost $0.33 per task one-day; trajectories are public. Self-reported results not on the board are higher (see the benchmark notes), but they do not follow the board's protocol or version.

Emerging

The best result is at 20–60% of the ceiling, or at 50–100% of human parity.

See the full leaderboard at UIUC

CVE-Bench: Pass@1 (one-day) over time, 2 recorded results. 0%20%40%60%80%100%Jan 2026 T-Agent + GPT-4o (2024-11-20): 7% (21 Mar 2025) Default Agent + Claude Opus 4.6: 40% (11 Apr 2026)
Dots are recorded results; the line is the best result so far; the red dot is the current frontier. Sources for every point are in the table below.
Embed this card

The image names the institutions behind the numbers and links back to this page.

HTML:

<a href="https://canagentswork.com/benchmarks/cve-bench/"><img src="https://canagentswork.com/og/benchmarks-cve-bench.png" width="600" height="315" alt="CVE-Bench: the best result is 40% (Default Agent + Claude Opus 4.6, 11 Apr 2026)." loading="lazy"></a>

Markdown:

[![CVE-Bench: the best result is 40% (Default Agent + Claude Opus 4.6, 11 Apr 2026).](https://canagentswork.com/og/benchmarks-cve-bench.png)](https://canagentswork.com/benchmarks/cve-bench/)

What it measures

Whether an agent can exploit a real web vulnerability end to end. In the zero-day setting the agent gets only the target; in the one-day setting it also gets a short description of the vulnerability. Success is one of eight attack outcomes checked by the sandbox. The paper reports success with one and five attempts; the maintainers' leaderboard (cvebench.com) reports Pass@1 for each setting and benchmark version. The headline here is one-day Pass@1 from that leaderboard.

Pass@1 (one-day): Share of the 40 CVEs exploited when the agent is given a vulnerability description, scored as Pass@1 on the maintainers' leaderboard. The board lists each run with its benchmark version. Higher is better.

Status compares the frontier with a ceiling of 100%.

Facts

Grain
Task level: bounded tasks with a clear spec
Environment
cli, browser
Grading
state-check
Tasks
40
Contamination
CVEs are public in the National Vulnerability Database with published details. Since v2.0, secrets are generated at runtime instead of hard-coded, in case old secrets appear in training data.
Reuse
Apache-2.0 (GitHub repository license). Reference exploits are available on request. (open-apache)

Limits to keep in mind

  • The maintainers' leaderboard has few entries: one current-version run (Claude Opus 4.6 with the default agent) and one v1.0 run. Other published numbers use their own harnesses, subsets, attempt counts, or the superseded v1.0 grader, so they are not comparable. Source
  • v2.0 (2025-10-30) closed two shortcuts that agents used to pass tasks. The maintainers say GPT-4o agent success rates fell by up to 32.5 points after the fixes, so v1.0 numbers overstate ability. Source
  • v2.1.0 (2026-01-12) replaced arbitrary file upload with remote code execution as an attack goal, and v2.2.0 (2026-09-24) changed several graders and challenge setups, so results across versions are not directly comparable. Source
  • 40 tasks is a small set; one task equals 2.5 percentage points. Source

Recorded results

A selection that shows the frontier over time. The full leaderboard is at the source.

SystemPass@1 (one-day)DateSource
Default Agent + Claude Opus 4.6 · frontier40%11 Apr 2026UIUC · primary
T-Agent + GPT-4o (2024-11-20)7%21 Mar 2025UIUC · primary

Timeline

  • 24 Sep 2026 — CVE-Bench v2.2.0 fixes several graders and challenge setups. Source
  • 11 Apr 2026 — Claude Opus 4.6 exploits 40% of CVE-Bench v2.1.0 targets in the one-day setting. Source
  • 12 Jan 2026 — CVE-Bench v2.1.0 replaces the file-upload goal with remote code execution. Source
  • 30 Oct 2025 — CVE-Bench v2.0 closes grading shortcuts. Source
  • 31 Mar 2025 — UIUC releases CVE-Bench with 40 critical web CVEs. Source

Where it sits in the atlas

Work ladder: Direct evidence for Security. On the work ladder it counts as 40% of tasks completed. How the ladder works

Work it measures (O*NET work activities): Test performance of computer or information systems.

Security

Vulnerability research

Last checked 10 Oct 2026 against 11 primary sources. See an error? Tell us.

How to cite

Credit the original work first: CVE-Bench by UIUC (https://arxiv.org/abs/2503.17332).

Then, if you used this page:

Can Agents Work. "CVE-Bench: frontier results and sources." https://canagentswork.com/benchmarks/cve-bench/ (accessed 2026-10-11). CC BY 4.0.
@misc{caw-cve-bench,
  title        = {{CVE-Bench: frontier results and sources}},
  author       = {{Can Agents Work}},
  year         = {2026},
  howpublished = {\url{https://canagentswork.com/benchmarks/cve-bench/}},
  note         = {Accessed 2026-10-11. CC BY 4.0}
}