Benchmarks / Asking for clarification

Asking for clarification

Whether an agent asks good questions when a task is underspecified.

• Ambig-SWEby Carnegie Mellon UniversityTask level · Underspecificity detection accuracy (best prompt)

  89%Claude Sonnet 4 · 21 Feb 2026

  Strong
• BIRD-Interactby HKU and Google CloudTask level · Success rate, a-Interact, Full, both phases

  22.7%MERIT + GPT-5.4 (GPT-4o user simulator) · 22 May 2026

  Emerging