VQCPC-GAN: VARIABLE-LENGTH ADVERSARIAL AUDIO SYNTHESIS USING VECTOR-QUANTIZED CONTRASTIVE PREDICTIVE CODING - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année :

VQCPC-GAN: VARIABLE-LENGTH ADVERSARIAL AUDIO SYNTHESIS USING VECTOR-QUANTIZED CONTRASTIVE PREDICTIVE CODING

(1, 2, 3, 4) , (1) , (1) , (2, 3, 4)
1
2
3
4

Résumé

Influenced by the field of Computer Vision, Generative Adversarial Networks (GANs) are often adopted for the audio domain using fixed-size two-dimensional spectrogram representations as the "image data". However, in the (musical) audio domain, it is often desired to generate output of variable duration. This paper presents VQCPC-GAN, an adversarial framework for synthesizing variablelength audio by exploiting Vector-Quantized Contrastive Predictive Coding (VQCPC). A sequence of VQCPC tokens extracted from real audio data serves as conditional input to a GAN architecture, providing step-wise time-dependent features of the generated content. The input noise z (characteristic in adversarial architectures) remains fixed over time, ensuring temporal consistency of global features. We evaluate the proposed model by comparing a diverse set of metrics against various strong baselines. Results show that, even though the baselines score best, VQCPC-GAN achieves comparable performance even when generating variable-length audio. Numerous sound examples are provided in the accompanying website, 1 and we release the code for reproducibility. 2 Index Terms-Generative Adversarial Networks, Audio Synthesis, Vector-Quantized Contrastive Predictive Coding * Nistal received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 765068. 1 sonycslparis.github.io/vqcpc-gan.io 2 github.com/SonyCSLParis/vqcpc-gan
Fichier principal
Vignette du fichier
2021-Waspaa-nistal.pdf (428.27 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03413460 , version 1 (03-11-2021)

Identifiants

  • HAL Id : hal-03413460 , version 1

Citer

Javier Nistal Hurlé, Cyran Aouameur, Stefan Lattner, Gael Richard. VQCPC-GAN: VARIABLE-LENGTH ADVERSARIAL AUDIO SYNTHESIS USING VECTOR-QUANTIZED CONTRASTIVE PREDICTIVE CODING. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), Oct 2021, New Paltz, United States. ⟨hal-03413460⟩
73 Consultations
78 Téléchargements

Partager

Gmail Facebook Twitter LinkedIn More