Artificial Analysis launches Harvey LAB-AA benchmark to test AI performance on complex legal tasks

1. Overview of Harvey LAB-AA

Artificial Analysis has introduced Harvey LAB-AA (Legal Agent Benchmark), a new evaluation framework designed to test AI models on real-world legal tasks. Developed in collaboration with Harvey, the benchmark consists of 120 private tasks covering 24 practice areas, including corporate M&A, tax, litigation, and bankruptcy. Performance is measured by an "all-pass rate," which tracks the percentage of tasks where a model satisfies every criterion in a binary grading rubric. This metric is intended to reflect the high standards required for professional legal work.

2. Performance and Capability Findings

As of July 7, 2026, Claude Fable 5 (with fallback) leads the leaderboard with an all-pass rate of 14.2%. Claude Opus 4.8 and GLM-5.2 follow, tying at 7.5%. The results indicate that while many models can pass a majority of individual rubric criteria, fully completing complex legal tasks remains a significant challenge. Approximately 46% of the 28 evaluated models failed to pass any tasks entirely. The top-performing models generally utilize long agentic loops, with Claude Sonnet 5 averaging 161 turns per task, the highest among those tested.

3. Cost and Efficiency Metrics

The evaluation highlights a wide disparity in the cost and speed of AI agents performing legal work. Claude Fable 5 is the most expensive model tested, costing approximately $18.94 per task, while GLM-5.2 achieves a similar all-pass rate at roughly 15% of that cost ($1.30 per task). Across the entire spectrum, the cost per task varies by a factor of approximately 950x, with models like Gemini 3.1 Flash-Lite costing as little as $0.02 per task. Additionally, stronger models typically require more time to complete tasks, with top performers averaging between 5 and 23 minutes per task.