Rethinking Automated Voice Similarity by Shifting from EER to Embedding Geometry

A NVIDIA-linked study challenges the standard use of equal error rate, or EER, as a proxy for human voice similarity in speech generation. Across speaker-embedding models trained on VoxCeleb2-dev with 5994 speakers, verification quality on VoxCeleb1-O showed almost no relationship to human judgments from VoxSim: the pooled Spearman correlation between EER and perceptual alignment was only +0.07. The decisive factor was the training objective and the geometry of the embedding space. Prototypical and Angular Prototypical losses consistently aligned better with listeners than classification losses such as AM-Softmax and AAM-Softmax, even without sacrificing EER; on ECAPA-TDNN, Angular Prototypical achieved a perceptual-alignment score of 0.406 versus 0.111 for AAM-Softmax, while both reached an EER of 1.47%. The proposed geometric measure, effective dimensionality, estimates how many directions speaker representations occupy. Across 35 model conditions, it correlated with perceptual alignment at −0.95, indicating that lower-dimensional speaker geometry better matches human perception. Explicit bottlenecking confirmed causality: reducing ECAPA-TDNN’s embedding dimension made the weakest 192-dimensional AM-Softmax model rise from 0.080 to 0.738 in perceptual alignment, although verification accuracy deteriorated. The results suggest that voice-similarity evaluation should prioritize embedding geometry over raw verification performance, with implications for TTS and voice-conversion systems built around models such as WavLM and ECAPA-TDNN.