Available under a Creative Commons Attribution Non-Commercial Share Alike 4.0 International Licence
In this study we propose how to modify a standard approach for text-to-speech alignment to apply in the case of alignment of lyrics and singing voice. We model phoneme durations by means of a duration-explicit hidden Markov model (DHMM) phonetic recognizer based on MFCCs. The phoneme durations are empirically set in a probabilistic way, based on prior knowledge about the lyrics structure and metric principles, specific for the Beijing opera music tradition. Phoneme models are GMMs trained directly on a small corpus of annotated singing voice. The alignment is evaluated on a cappella material from Beijing opera, which is characterized by its particularly long syllable durations. Results show that the incorporation of music-specific knowledge results in a very high alignment accuracy, outperforming significantly a baseline HMM-based approach.
Dzhambazov, G., Yang, Y., Repetto, R., Serra, X. (2016). Automatic Alignment of Long Syllables in a Cappella Beijing Opera. 6th International Workshop on Folk Music Analysis, Dublin, 15-17 June, 2016.