Skip to Main content Skip to Navigation
Journal articles

Phoneme Level Lyrics Alignment and Text-Informed Singing Voice Separation

Abstract : The goal of singing voice separation is to recover the vocals signal from music mixtures. State-of-the-art performance is achieved by deep neural networks trained in a supervised fashion. Since training data are scarce and music signals are extremely diverse, it remains challenging to achieve high separation quality across various recording and mixing conditions as well as music styles. In this paper, we investigate to which extent the separation can be improved when lyrics transcripts are used as additional information. To this end, we propose a joint approach to phoneme level lyrics alignment and text-informed singing voice separation. It is based on DTW-attention, a new monotonic attention mechanism including a differentiable approximation of dynamic time warping. Experimental results show that the method can align phonemes with mixed singing voice with high precision given accurate transcripts. It also achieves competitive results on challenging word level alignment test sets using less training data than state-of-the-art methods. Sequential alignment and informed separation lead to improved separation quality according to objective measures. Text information helps preserving spectral phoneme properties in the separated voice signals.
Document type :
Journal articles
Complete list of metadata
Contributor : Roland Badeau Connect in order to contact the contributor
Submitted on : Tuesday, August 3, 2021 - 12:06:36 PM
Last modification on : Tuesday, January 4, 2022 - 7:44:01 PM
Long-term archiving on: : Thursday, November 4, 2021 - 6:59:09 PM


Files produced by the author(s)




Kilian Schulze-Forster, Clement S J Doire, Gael Richard, Roland Badeau. Phoneme Level Lyrics Alignment and Text-Informed Singing Voice Separation. IEEE/ACM Transactions on Audio, Speech and Language Processing, Institute of Electrical and Electronics Engineers, inPress, ⟨10.1109/TASLP.2021.3091817⟩. ⟨hal-03255334⟩



Record views


Files downloads