Artificial Analysis Launches Speech to Speech Index to Benchmark AI Conversational Performance

1. Overview of the Speech to Speech Index

On June 23, 2026, Artificial Analysis introduced the Speech to Speech Index, a new metric designed to evaluate the quality of native speech-to-speech AI models. The index provides a unified score based on three equally weighted datasets: Big Bench Audio, which tests speech reasoning; Full Duplex Bench, which measures conversational dynamics like turn-taking and interruption handling; and 𝜏-Voice, which assesses agentic performance in customer service scenarios. To be included in the index, models must provide valid results across all three categories.

2. Performance and Capability Rankings

The index reveals that OpenAI’s GPT-Realtime-2 (High) currently leads with an overall score of 77.2%, followed by xAI’s Grok Voice Think Fast 1.0 at 75.7%, GPT-Realtime-1.5 at 72.0%, and Google’s Gemini 3.1 Flash Live Preview (High) at 69.5%. Conversational dynamics and agentic performance serve as the primary differentiators between top-tier models. While GPT-Realtime-2 excels in conversational dynamics, Grok Voice Think Fast 1.0 leads in agentic performance. Notably, agentic performance remains the most challenging dimension, with all tested models scoring below 53%.

3. Speed and Cost Metrics

The index also tracks operational efficiency through time to first audio (TTFA) and cost per hour of input audio. Deepslate Opal is the fastest model, achieving a TTFA of 0.44 seconds. In contrast, GPT-Realtime-2 (High) records 2.33 seconds, and Gemini 3.1 Flash Live Preview (High) records 2.98 seconds. Regarding costs, Gemini 3.1 Flash Live Preview (Minimal) is the most economical option at $1.50 per hour, while GPT-Realtime-2 (High) costs $4.14 per hour. Artificial Analysis intends to continue updating the index with additional models and refined benchmarks.