ORCID
- Jane Courtney: 0000-0002-9175-7855
- Robert Ross: 0000-0001-7088-273X
Abstract
The dataset was collected from 28 participants (17 female, 9 male, and 1 non-binary) for a study aimed at modelling and detecting user social behaviours with different confusion states in task-oriented situated human-robot interaction (HRI). The dataset consists of user facial body video recordings synchronised with user speech across three designed experiment scenarios (Tasks 1 - 3). Each experiment lasted approximately one hour per participant. The videos are segmented into individual clips corresponding to specific experimental conversations under predefined conditions: general confusion and non-confusion for Task 1 and 3; and productive confusion, unproductive confusion, and non-confusion for Task 2. In total, the dataset contains 789 video clips (body: 392, face: 397). Each video is recorded in high-definition RGB format, capturing user facial expressions or body language along with their speech. These multimodal data provide a valuable resource for studying user cognitive and mental states in human-robot interaction and human-computer interaction. The data collected for Task 2 was used in [9]. In compliance with GDPR (General Data Protection Regulation) and DPIA (data protection impact assessment) guidelines, the dataset is freely available upon request at https://sites.google.com/view/hridatarequst/home.
Keywords
Confusion, Human-robot interaction, Multimodal data, User social behaviours
DOI Link
Publication Date
2025-01-01
Publication Title
Data in Brief
Volume
62
Deposit Date
2026-11-05
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Additional Links
Recommended Citation
Li, N., Courtney, J., & Ross, R. (2025) 'HRI-confusion: A multimodal dataset for modelling and detecting user confusion in situated human-robot interaction', Data in Brief, 62. Available at: 10.1016/j.dib.2025.112047