Artificial Analysis benchmark reveals performance and speed trade-offs for AI models in long-horizon tasks
New benchmark data highlights the relationship between model capability and processing speed across complex, long-horizon knowledge tasks.
1. Overview of AA-Briefcase Metrics
Artificial Analysis recently introduced AA-Briefcase, a benchmark designed to evaluate AI models on long-horizon knowledge work, such as creating financial models, board presentations, and design mock-ups. A primary metric for this benchmark is the average time per task, which is calculated based on evaluation token usage, model output speeds, and tool execution time. The data indicates that tool execution accounts for only about 12% of the total time, with the remainder attributed to inference speed, turn usage, and output verbosity.
2. Model Performance and Efficiency
Performance results show a trade-off between model capability and speed. Claude Opus 4.8 is currently the highest-scoring model but requires approximately 23 minutes per task. In contrast, GPT-5.5 (xhigh) offers higher efficiency, completing tasks in about 11 minutes while maintaining a top-five ranking on the AA-Briefcase Elo scale. GLM-5.2 also performs well, ranking as the top open-weights model with a score of 1261, though it requires 16.3 minutes per task. Historical data for the discontinued Claude Fable 5 suggests it would have required approximately 28.5 minutes per task based on its output token volume and speed.