τ²-bench

Built by Sierra · Victor Barres, Honghua Dong, Soham Ray, Xujie Si, et al. · released 9 Jun 2025

Simulated customer-service conversations from Sierra. An agent chats with an LLM-played user, calls tools, and must follow a written domain policy. The telecom domain adds dual control: the user also has tools, so the agent must guide the user through steps that only the user can perform.

Frontier

97.8%

Telecom pass^1

Qwen3.5-397B-A17B (thinking) · harness: tau2-bench default agent

27 Feb 2026 · Source: Sierra (benchmark maintainers)

Best telecom pass^1 among runs that Sierra ran and verified with trajectories. User simulator gpt-5.2 (low reasoning), 4 trials, self-hosted with vLLM. pass^2 95.76, pass^3 93.86, pass^4 92.11. Same submission: airline 81.5, retail 84.43, banking knowledge 9.79 pass^1. Open weights: the model is on Hugging Face under Apache-2.0 (Qwen/Qwen3.5-397B-A17B). Corrected 2026-09-24: we showed 97.8; the source shows 97.81 (found in a spot-check).

Saturated

The best result is at 90% or more of the ceiling.

See the full leaderboard at Sierra

Dots are recorded results; the line is the best result so far; the red dot is the current frontier. Sources for every point are in the table below.

Embed this card

The image names the institutions behind the numbers and links back to this page.

HTML:

<a href="https://canagentswork.com/benchmarks/tau2-bench/"><img src="https://canagentswork.com/og/benchmarks-tau2-bench.png" width="600" height="315" alt="τ²-bench: the best result is 97.8% (Qwen3.5-397B-A17B (thinking), 27 Feb 2026)." loading="lazy"></a>

Markdown:

[![τ²-bench: the best result is 97.8% (Qwen3.5-397B-A17B (thinking), 27 Feb 2026).](https://canagentswork.com/og/benchmarks-tau2-bench.png)](https://canagentswork.com/benchmarks/tau2-bench/)

What it measures

Whether the final database state and required outputs match the expected result for each task. Each task runs several times. pass^k is the chance that all k tries of a task succeed, so higher k rewards reliability. Domains are airline (50 tasks), retail (114), and telecom (114). We use telecom pass^1 as the headline because telecom is the domain that τ²-bench introduced. The repository now ships as τ³-bench with a banking knowledge domain and a voice mode; those are not scored here.

Telecom pass^1: Share of telecom tasks that the agent completes correctly on a single try, averaged over trials. The leaderboard reports pass^1 to pass^4 per domain. Higher is better.

Status compares the frontier with a ceiling of 100%.

Facts

Grain: Task level: bounded tasks with a clear spec
Environment: api-tools, chat
Grading: state-check
Tasks: 278
Contamination: Tasks, policies, and tools are public in the repository. The leaderboard labels models trained on τ-bench domains or tasks as custom submissions, but it cannot detect training on the public data.
Reuse: MIT (repository license covers tasks, policies, and code). (open-mit)

Limits to keep in mind

The user is an LLM simulator. The choice of user model changes scores, and submissions use different user models (gpt-4.1, gpt-5.2, or the agent's own model). Sierra recommends gpt-5.2. Source
Telecom pass^1 is close to the ceiling: several models score 97-98%. Since May 2026 Sierra's own runs cover only the banking knowledge domain ("other domains intentionally excluded"), so no new verified telecom results arrive. pass^4 and banking (best 55.2% pass^1) still separate models. Source
In February 2026 Sierra fixed 50+ airline and retail tasks. Airline pass^1 rose by 14 to 20 points after the fixes, so airline and retail scores from before and after are not comparable. Source
Some leaderboard entries were submitted by model developers with modified prompts and no trajectories. Sierra marks these as unverified in the submission files. Source

Recorded results

A selection that shows the frontier over time. The full leaderboard is at the source.

System	Telecom pass^1	Date	Source
Qwen3.5-397B-A17B (thinking) · frontier harness: tau2-bench default agent	97.8%	27 Feb 2026	Sierra · primary
Qwen3-Max-Thinking harness: tau2-bench default agent	98.2%	23 Jan 2026	Qwen team (leaderboard submission) · lab-reported
GPT-5 harness: tau2-bench default agent	95.8%	9 Aug 2025	Sierra · primary
GPT-4.1 harness: tau2-bench default agent	34%	9 Jun 2025	Sierra · primary
o4-mini harness: tau2-bench default agent	50.2%	9 Jun 2025	Sierra · primary

Timeline

4 Mar 2026 — Sierra launches τ-Knowledge (τ³-Banking), a banking support domain where frontier models reach about 25.5% pass^1. Source
Feb 2026 — Sierra fixes 50+ airline and retail tasks for τ³-bench. Source
2 Oct 2025 — GPT-5 reaches 95.8% telecom pass^1 on τ²-bench. Source
9 Jun 2025 — Sierra releases τ²-bench with a dual-control telecom domain. Source

Where it sits in the atlas

Work ladder: Direct evidence for Customer support. On the work ladder it counts as 97.8% of tasks completed. How the ladder works

Work it measures (O*NET work activities): Respond to customer problems or inquiries; Explain technical details of products or services; Provide information to guests, clients, or customers; Execute financial transactions; Maintain sales or financial records.

Customer support

Customer service

Go to the source

Website taubench.com
Paper arxiv.org
Full leaderboard taubench.com
Code github.com
Announcement sierra.ai

Last checked 10 Oct 2026 against 7 primary sources. See an error? Tell us.

How to cite

Credit the original work first: τ²-bench by Sierra (https://arxiv.org/abs/2506.07982).

Then, if you used this page:

Can Agents Work. "τ²-bench: frontier results and sources." https://canagentswork.com/benchmarks/tau2-bench/ (accessed 2026-10-11). CC BY 4.0.

@misc{caw-tau2-bench,
  title        = {{τ²-bench: frontier results and sources}},
  author       = {{Can Agents Work}},
  year         = {2026},
  howpublished = {\url{https://canagentswork.com/benchmarks/tau2-bench/}},
  note         = {Accessed 2026-10-11. CC BY 4.0}
}