Liva-FullDuplex-Bench
Liva-FullDuplex-Bench
A Multilingual Benchmark for Conversational Companion Performance in Full-Duplex Speech Models
Current full-duplex benchmarks like Full-Duplex-Bench are limited to English, rely partly on synthetic data, and evaluate models strictly for professional/assistant use. Liva-FullDuplex-Bench extends this work to companion-style conversation and is the first multilingual full-duplex benchmark built entirely from real conversational data between friends and acquaintances, assessing four 1-on-1 conversational events across English, Hindi, and Arabic.
Last updated May 28, 2026
Pause Handling
Can the model recognize mid-utterance pauses and not take over? Lower TOR is better.
Backchanneling
Can the model produce backchannels at appropriate times without interrupting? Lower TOR/JSD is better; higher rating (from LLM-as-judge) is better.
Turn-Taking
Can the model detect end-of-turn and respond relevantly and timely? Higher TOR/rating is better; lower latency is better.
Interruption Management
Can the model detect speaker interruptions and handle them gracefully? Higher TOR/rating is better; lower latency is better.
Methodology
Motivation. Full-duplex speech models can process overlapping speech and backchannel in real time, unlocking companion-style AI that feels natural to talk to. However, existing benchmarks focus on assistant/professional use cases. Full-Duplex-Bench (FDB) evaluates turn-taking awareness but is limited to English and partially built with synthetic data, which may not capture the paralinguistic nuances of real conversation. MTR-DuplexBench extends to multi-round settings but remains English-only. No prior benchmark uses naturalistic multilingual data from casual settings.
Source data. All samples are drawn from Yapdo, Liva AI's ~50,000-hour multilingual conversational speech corpus comprising ~17,000 speakers across 60+ languages. Conversations are between friends and acquaintances, natively recorded with separate speaker channels via consumer social apps designed to encourage casual, unscripted interaction. This in-the-wild collection produces more authentic turn-taking dynamics than lab-recorded or task-oriented datasets.
Benchmark construction. Following FDB v1's methodology, we created an audio stream (input.wav) to simulate user speech and captured the model's response (output.wav). Unlike FDB, our samples come exclusively from real conversations - not synthetic or acted data - so the cues that elicit specific conversational behaviors are naturally present. 100 samples per language (English, Hindi, Arabic) per event.
Conversational events. Four events extend FDB v1 to multilingual, companion-style settings. These are pause handling (mid-utterance pauses of 0.4–1s), backchanneling, turn-taking (turns within 0.05–0.8s gap), and interruption management (speaker A interrupts during B's turn).
Metrics. Takeover Rate (TOR) measures how often the model takes the floor (0–1). Jensen-Shannon Divergence (JSD) measures temporal alignment of backchannels vs. human reference ground truth from the original Yapdo conversation (0–1, lower = better alignment). Model latency is the time from end of user utterance to model response (seconds). Latency delta is model latency minus real-world human-to-human latency (seconds). LLM-judge scores come from model-user conversations transcribed with ElevenLabs Scribe v2 and rated 0–5 by an LLM for response content quality and timing.
Models evaluated. GPT-Realtime, Gemini 2.5 Flash Native Audio, Amazon Nova Sonic, and Grok. Results averaged over 100 samples per language.