HAL will be down for maintenance from Friday, June 10 at 4pm through Monday, June 13 at 9am. More information
Skip to Main content Skip to Navigation
Conference papers

Training Deep Pitch-Class Representations With a Multi-Label CTC Loss

Abstract : Despite the success of end-to-end approaches, chroma (or pitch-class) features remain a useful mid-level representation of music audio recordings due to their direct interpretability. Since traditional chroma variants obtained with signal processing suffer from timbral artifacts such as overtones or vibrato, they do not directly reflect the pitch classes notated in the score. For this reason, training a chroma representation using deep learning ("deep chroma") has become an interesting strategy. Existing approaches involve the use of supervised learning with strongly aligned labels for which, however, only few datasets are available. Recently, the Connectionist Temporal Classification (CTC) loss, initially proposed for speech, has been adopted to learn monophonic (single-label) pitch-class features using weakly aligned labels based on corresponding score--audio segment pairs. To exploit this strategy for the polyphonic case, we propose the use of a multi-label variant of this CTC loss, the MCTC, and formalize this loss for the pitch-class scenario. Our experiments demonstrate that the weakly aligned approach achieves almost equivalent pitch-class estimates than training with strongly aligned annotations. We then study the sensitivity of our approach to segment duration and mismatch. Finally, we compare the learned features with other pitch-class representations and demonstrate their use for chord and local key recognition on classical music datasets.
Complete list of metadata

Contributor : Christof Weiss Connect in order to contact the contributor
Submitted on : Monday, September 20, 2021 - 4:54:38 PM
Last modification on : Wednesday, December 1, 2021 - 3:12:20 AM
Long-term archiving on: : Tuesday, December 21, 2021 - 7:17:56 PM


Publisher files allowed on an open archive


  • HAL Id : hal-03349734, version 1



Christof Weiss, Geoffroy Peeters. Training Deep Pitch-Class Representations With a Multi-Label CTC Loss. International Society for Music Information Retrieval Conference (ISMIR), Nov 2021, Virtual Event, France. ⟨hal-03349734⟩



Record views


Files downloads