Benchmarks / Penetration testing

Penetration testing

Attack a network or system end to end, from reconnaissance to exploitation, as a pentester or red team would.

Part of Security.

• AISI multi-step cyber attack rangesby AI Security InstituteProject level · Average steps completed, "The Last Ones" (100M tokens)

  15.6Claude Opus 4.6 · 11 Mar 2026

  Emerging
• ARTEMIS live penetration test (agents vs. professionals)by Stanford University, Carnegie Mellon University and Gray Swan AIProject level · Total score (complexity plus weighted severity of valid findings)

  95.2ARTEMIS A2 (ensemble supervisor, Claude Sonnet 4 sub-agents) · 10 Dec 2025

  Strong