Author ORCID Identifier

https://orcid.org/0000-0002-6153-3797

Document Type

Conference Paper

Disciplines

Computer Sciences

Publication Details

  • Published in: Proceedings of the ACM International Conference on Intelligent Virtual Agents (IVA ’25)

  • Conference: IVA 2025, September 16–19, 2025, Berlin, Germany

  • Publisher: ACM, New York, NY, USA

  • doi:10.1145/3717511.3747074

Abstract

The creation of virtual humans increasingly leverages automated synthesis of speech and gestures, enabling expressive, adaptable agents that effectively engage users. However, the independent development of voice and gesture generation technologies, alongside the growing popularity of virtual reality (VR), presents significant questions about the integration of these signals and their ability to convey emotional detail in immersive environments. In this paper, we evaluate the influence of real and synthetic gestures and speech, alongside varying levels of immersion (VR vs. 2D displays) and emotional contexts (positive, neutral, negative) on user perceptions. We investigate how immersion affects the perceived match between gestures and speech and the impact on key aspects of user experience, including emotional and empathetic responses and the sense of co-presence. Our findings indicate that while VR enhances the perception of natural gesture–voice pairings, it does not similarly improve synthetic ones—amplifying the perceptual gap between them. These results highlight the need to reassess gesture appropriateness and refine AI-driven synthesis for immersive environments.

DOI

https://doi.org/10.1145/3717511.3747074

Funder

Research Ireland Centre for Research Training in Digitally-Enhanced Reality (d-real)

Creative Commons License

Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License
This work is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 4.0 International License.

Download
Contact the Author


Share

COinS