PINT: Invariant Speech Tokenization from Parallel Utterances
Researchers introduce PINT (Parallel INvariant Tokenization), a method that fine-tunes a self-supervised speech encoder using alignment losses across parallel utterances to isolate linguistic content while removing speaker identity, prosody, and channel effects. PINT achieves a 98.7% relative reduction in speaker probe accuracy, a 42% lower ABX error rate, and 27-30% lower language model perplexity compared to baselines, indicating improved disentanglement of linguistic and non-linguistic information.
Why it matters: This approach advances speech tokenization by more effectively isolating linguistic content, which can benefit tasks like speech recognition and audio coding.
Full story at: arXiv Computation and Language ↗