An Exploration of the Latent Space of a Convolutional Variational Autoencoder for the Generation of Musical Instrument Tones

Authors

Anastasia Natsiou, Technological University DublinFollow
Sean O'Leary, Technological University DublinFollow
Luca Longo, Technological University DublinFollow

Author ORCID Identifier

https://orcid.org/0000-0002-2916-0134

Document Type

Conference Paper

Disciplines

Computer Sciences, Acoustics

Publication Details

The 1st World Conference on eXplainable Artificial Intelligence (xAI 2023)

https://xaiworldconference.com/

Abstract

Variational Autoencoders (VAEs) constitute one of the most significant deep generative models for the creation of synthetic samples. In the field of audio synthesis, VAEs have been widely used for the generation of natural and expressive sounds, such as music or speech. However, VAEs are often considered black boxes and the attributes that contribute to the synthesis of a sound are yet unsolved. Existing research focused on the way input data can influence the generation of latent space, and how this latent space can create synthetic data, is still insufficient. In this manuscript, we investigate the interpretability of the latent space of VAEs and the impact of each attribute of this space on the generation of synthetic instrumental notes. The contribution to the body of knowledge of this research is to offer, for both the XAI and sound community, an approach for interpreting how the latent space generates new samples. This is based on sensitivity and feature ablation analyses, and descriptive statistics.

DOI

https://doi.org/10.21427/2082-7N65

Recommended Citation

Natsiou, A. (2023). An Exploration of the Latent Space of a Convolutional Variational Autoencoder for the Generation of Musical Instrument Tones. xAI2023 Conference. DOI: 10.21427/2082-7N65

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 4.0 International License.