NTH

Kezek: Splitting Mono Two-Person Conversations into Per-Speaker Tracks for Full-Duplex Dialogue Data

AuthorsBagzhan Karl, Alizhan Karl

AffiliationsManifestro

November 21, 2026 2 min read
Watch on YouTube
The one-line take

Kezek turns a mixed two-person recording into speaker-specific tracks while preserving overlaps and short replies for full-duplex dialogue data.

Key results

0.830
Open Yap short-reply retention

Kezek’s share of short replies retained on the correct track; the pyannote-to-DialogueSidon baseline scores 0.687.

0.934
Open Yap no-swap rate

Share of recordings with no channel swaps; the baseline scores 0.793.

1.000
Open Yap coverage

Share of input duration returned; the baseline scores 0.735.

231
Processing throughput

Hours of audio per GPU-hour on Open Yap; the baseline processes 51.

0.796
CallHome short-reply retention

Share of replies over the other speaker’s speech retained on the correct track in unseen telephone recordings.

0.171
Quiet-reply retention

Kezek’s retention for replies more than 10 dB below the other voice; an ideal ratio mask scores 0.743.

What the paper found

Kezek turns a mono recording of exactly two people into synchronized, per-speaker tracks, targeting the two-channel training data needed by full-duplex dialogue systems such as Moshi. Its two-stage design assigns speaker identities and activity once for the whole recording, then uses a FiLM-conditioned Conv-TasNet-style separator to carry out that assignment; a mixture-consistency constraint ensures the tracks sum exactly to the input. Word-gated activity labels help prevent microphone crosstalk from causing channel swaps, while phase-robust speaker fingerprints support identity tracking. Against the pyannote-to-DialogueSidon pipeline on Open Yap, Kezek retains 0.830 of short replies, avoids swaps in 0.934 of recordings, and returns 1.000 of the audio; it processes 231 hours of audio per GPU-hour. On unseen CallHome telephone speech, it retains 0.796 of short replies over the other speaker’s speech. A key remaining weakness is quiet backchannels: when replies are more than 10 dB below the other voice, Kezek keeps 0.171 under the paper’s stricter audibility test, compared with 0.743 for an ideal ratio mask.

Original abstract

Full-duplex spoken dialogue models learn turn-taking, overlaps and backchannels from conversations recorded with one channel per speaker, but most available conversational audio is a single mono mix. We present Kezek, a model that turns a mono recording of exactly two people into one track per person with the original length and timing; overlaps and short replies ("yeah", "mm-hm") stay on the track of the person who said them. Kezek decides who speaks when and who is who once for the whole recording (stage 1: a two-voice activity model guided by a speaker fingerprint trained to be insensitive to phase), and a conditioned time-domain separator only carries out that decision (stage 2), with outputs that always sum to the input. Standard separation metrics miss the failures that matter for dialogue data, so we report short-reply retention, leakage onto the other person's solo, channel swaps and coverage. Against the open pipeline pyannote -> DialogueSidon on the same recordings, Kezek keeps more short replies (0.83 vs. 0.69 on Open Yap test; 0.80 vs. 0.71 over the other's speech on unseen telephone speech from CallHome English), mostly because it returns the whole recording while the external pipeline drops part of it (on the audio both return, reply retention is about the same: 0.859 vs. 0.865 on Open Yap); it also leaks less, swaps channels less often, and runs 4-5 times faster (about 230 hours of audio per GPU-hour). We also report negative results and an ideal-mask ceiling that locate the remaining gap: quiet replies under a louder voice. Code and weights are released under permissive licences.

Read the original paper

More in Speech AI

Browse all 48 papers →
01Speech

Marimba: A System One Model for Speech

Solano Amadori

Marimba reuses a speech recognition model’s features to answer questions about words, speakers, language, and turn endings.

Read analysis
02Speech

VoxMel TTS Technical Report

Wenhao Guan, Jiaxin Ye, Haitao Li, Yifan Duan, Junxi Liu, Wangjin Zhou, Lin Li, Qingyang Hong, Xie Chen

VoxMel generates speech by autoregressively modeling transformed mel-spectrograms, avoiding a learned tokenizer while aiming for high fidelity and faster inference.

Read analysis