Benchmarks / Legal Research Bench
Legal Research Bench
Built by Vals AI · released 23 Jun 2026
Vals AI's test of whether an agent can research a US legal question the way a junior associate or paralegal would. The agent gets case-law search, web search, page parsing, and document retrieval, and must return a supported answer. Practicing lawyers wrote and reviewed the questions, answers, and rubrics.
Frontier
55.3%
All-pass accuracy
Muse Spark 1.3 Max · harness: Vals legal-research harness
22 Sep 2026 · Source: Vals AI (benchmark maintainers)
Three-way tie for first place with Claude Opus 5 and Claude Fable 5.1, all at 55.29% (page data 55.288, stderr 3.456). Vals lists Muse Spark 1.3 Max first. Page data: about 5 minutes and $0.59 per task. Date is the page's "updated" field. Open-weights status not confirmed.
The best result is at 20–60% of the ceiling, or at 50–100% of human parity.
What it measures
All-pass accuracy on a private test set of 208 questions (413 in total, with 5 public samples and 200 validation questions available for license). Questions span eight practice areas of US federal and state law, for example administrative, criminal, business, and immigration. Each question has a lawyer-written rubric of 1 to 31 required items (mean 9.35). An LLM judge (GPT 5.4) checks every item; a question counts only when every item passes. Vals also reports a partial-credit weighted pass rate, cost, latency, and tool calls per task.
All-pass accuracy: Share of private test questions where the agent's answer satisfies every item of the lawyer-written rubric, as judged by GPT 5.4. Higher is better.
Status compares the frontier with a ceiling of 100%.
Facts
- Grain
- Task level: bounded tasks with a clear spec
- Environment
- api-tools, documents
- Grading
- llm-judge, rubric
- Tasks
- 413
- Human reference
- Practicing lawyers wrote and peer-reviewed the questions, gold answers, and rubrics. Gold answers average 536 words and cite 6.2 sources. No human accuracy score is published.
- Contamination
- The test set is private, and Vals says it will stay private. All page results use the test set only.
- Reuse
- The page labels the benchmark "Proprietary". 5 public sample questions and the agent harness are open in the repository (MIT). 200 validation questions are available for license. The 208-question test set is private. (cite-only)
Limits to keep in mind
- Each score has a standard error of about 3.5 points, so the three tied leaders (55.29%) and the next models (Gemini 4 Argon 54.81%, Claude Opus 5.5 50.48%) overlap. Source
- Grading uses an LLM judge (GPT 5.4). Vals reports that the judge agrees with a majority of three human expert reviewers 87.4% of the time, against 83.4% for an average single human reviewer. Source
- Agents run in a fixed Vals harness with five tools and a three-hour limit per task. Access to the Vals platform to run the private set is gated. Source
- US federal and state law only, in eight practice areas. Vals says that models score lowest on questions that need reconciling conflicting authority across jurisdictions. Source
Recorded results
A selection that shows the frontier over time. The full leaderboard is at the source.
| System | All-pass accuracy | Date | Source |
|---|---|---|---|
| Muse Spark 1.3 Max · frontier harness: Vals legal-research harness | 55.3% | 22 Sep 2026 | Vals AI · primary |
| Claude Opus 5 harness: Vals legal-research harness | 55.3% | 22 Sep 2026 | Vals AI · primary |
| Claude Fable 5.1 harness: Vals legal-research harness | 55.3% | 22 Sep 2026 | Vals AI · primary |
| Claude Opus 5.5 harness: Vals legal-research harness | 50.5% | 22 Sep 2026 | Vals AI · primary |
Timeline
- 23 Jun 2026 — Vals AI releases Legal Research Bench with 413 lawyer-written US legal research questions. Source
Where it sits in the atlas
Work ladder: Direct evidence for Legal. On the work ladder it counts as 55.3% of tasks completed. How the ladder works
Work it measures (O*NET work activities): Research laws, precedents, or other legal data; Consult legal materials or public records.
Go to the source
- Website www.vals.ai
- Paper arxiv.org
- Full leaderboard www.vals.ai
- Code github.com
Last checked 10 Oct 2026 against 5 primary sources. See an error? Tell us.
How to cite
Credit the original work first: Legal Research Bench by Vals AI (https://arxiv.org/abs/2610.00609).
Then, if you used this page:
Can Agents Work. "Legal Research Bench: frontier results and sources." https://canagentswork.com/benchmarks/vals-legal-research/ (accessed 2026-10-11). CC BY 4.0.@misc{caw-vals-legal-research,
title = {{Legal Research Bench: frontier results and sources}},
author = {{Can Agents Work}},
year = {2026},
howpublished = {\url{https://canagentswork.com/benchmarks/vals-legal-research/}},
note = {Accessed 2026-10-11. CC BY 4.0}
}