Document Type

Dissertation

Disciplines

1.2 COMPUTER AND INFORMATION SCIENCE, Computer Sciences

Publication Details

A dissertation submitted in partial fulfilment of the requirements of Dublin Institute of Technology for the degree of M.Sc. in Computing - Advanced Software Development

Abstract

AI decision support systems aim to assist people in highly complex and consequential domains to make efficient, effective, and high-quality decisions. AI alone cannot be guaranteed to be correct in these complex decision tasks, and a human is often needed to ensure decision accuracy. The ambition is for these human+ AI teams to perform better together than either would individually. To realise this, decision makers must trust their AI partners appropriately, knowing when to rely on their recommendations and when to be sceptical. However, research has shown that decision makers often either mistrust and underutilise these systems, or trust them blindly. Researchers in the fields of HCI and XAI have worked on developing methods that continuously manage an appropriate level of user trust. Despite the probabilistic nature of ML-based AI, little attention has been given to understand how the research area of uncertainty communication might provide solutions to this challenge. This study draws on that research, and asks how different forms of expressing probability in AI decision support systems might affect human+ AI team performance. A series of task-based user tests were conducted to evaluate the use of numerical, verbal, and verbal-numerical probability expressions in communicating AI prediction confidence to decision makers. Results indicated that numerical expressions may be most effective when decision makers use AI decision support. However, findings were inconclusive due to a limited number of participants who used AI decision support during testing.

Creative Commons License

Creative Commons Attribution-Share Alike 4.0 International License
This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.

Download


Share

COinS