Good-Pal-Bench
MIRRORING
A Multilingual Benchmark for Emotional Accommodation in Full-Duplex Speech Models
Despite strong progress on turn-taking and voice naturalness, speech-to-speech models are never evaluated on whether they adapt their emotional tone to match a conversational partner. MIRRORING measures this by testing whether, given an emotionally expressive human turn, the model responds with the same emotion a human interlocutor actually produced. Also check out Emo-Com, a dataset of raw, unfiltered interactions between people with emotions ranging from jokes to discussing grief.
Last updated May 28, 2026
Overall Rankings
1,005 samples across 12 emotions and 5 languages. The shuffled baseline answers how well a model would appear to perform just by chance, given its own emotional biases. It is computed by randomly re-pairing model outputs with different reference emotions (1,000 iterations), so only match rate above this threshold reflects genuine emotional accommodation. Click headers to re-sort.
| # | Model | Match Rate | Shuffled | Lift (pp) | Cosine Sim | Target Score | Ref Score | N |
|---|
Match Rate is the fraction where the model's top emotion matches the human reference. Cosine similarity measures how closely the model's response resembles the human reference across all 12 emotions. Target score is the model's intensity on the expected emotion, compared to the human reference's intensity on that same emotion (human reference ~0.95).
Per-Emotion Breakdown
Match rate (%) by emotion, averaged across models. Positive/high-arousal emotions are mirrored well above the 8.3% uniform baseline; negative-valence emotions remain near zero, with models producing a "helpful assistant" tone that rarely sounds angry, sad, or impatient regardless of the situation. Mirroring negative emotions can be important in building more sympathetic models or models that are generally more comfortable to talk to.
Per-Language Heatmap
Match rate (%) by language. English and Tagalog tend to yield the highest rates; Arabic tends to yield the lowest.
Methodology
Source data. 567 two-speaker voice call recordings from Yapdo, Liva AI's ~50,000-hour multilingual conversational speech corpus, were used to create the benchmark dataset. All speakers were friends or acquaintances having naturally occurring conversations. 11,124 candidate turn-taking pairs (one turn of Speaker A followed by one turn of Speaker B) were extracted from those recordings. Recordings spanned English, Hindi, Arabic, Italian, and Tagalog.
Emotion annotation. For each turn-taking pair, each speaker's turn was independently scored using Empathic-Insight-Voice-Small, a 47-dimension vocal emotion classifier that assigns intensity scores across emotion categories. The 12 emotions selected for the benchmark were Amusement, Anger, Elation, Impatience, Surprise, Emotional Numbness, Contemplation, Disappointment, Confusion, Pride, Affection, and Sadness.
Benchmark construction. Turn pairs were selected where both speakers independently shared the same dominant emotion and occurred within a 5-second gap. Speaker A's turn was extended backward along natural pause boundaries to provide approximately 10 seconds of audio context for the model. From the candidate pairs, a balanced subset of 1,005 samples was created by limiting the four most overrepresented emotions to 150 samples each.
Evaluation. Each model received Speaker A's audio and generated a spoken response, which was then scored by the same Empathic-Insight-Voice-Small classifier. A match occurred when the model's top predicted emotion equaled Speaker B's reference emotion. The three metrics were top emotion match rate, emotion vector cosine similarity, and target emotion intensity score.
Shuffled baseline. For each model, reference emotions were randomly re-paired with model outputs 1,000 times to estimate how often a match would occur by chance alone. The difference between a model's actual match rate and this baseline reflects genuine emotional accommodation rather than the model's tendency to favor certain emotions regardless of input.
Validation. To confirm that emotional co-occurrence reflects genuine conversational behavior rather than chance, all 37,173 unfiltered adjacent turn pairs across 200 recordings were analyzed. 18.6% shared the same dominant emotion, compared to a 15.8% permutation baseline (p < 0.001), confirming that speakers naturally mirror each other's emotions in real conversation.