Fast, Accurate, But Sometimes Too-Compelling Support: The Impact of Imperfectly Automated Cues in an Augmented-Reality Head-Mounted Display on Visual Search Performance

Abstract

While the visual search for targets in a complex scene might benefit from using augmented-reality (AR) head-mounted display (HMD) technologies by helping to efficiently direct human attention, imperfectly reliable automation support could manifest in occasional errors. The current study examined the effectiveness of different HMD cues that might support visual search performance and their respective consequences following automation errors. A total of 56 participants searched a three-dimensional environment containing 48 objects in a room, in order to locate a target object that was viewed prior to each trial. They searched either unaided or assisted by one of the three HMD types of cues: an arrow pointing to the target, a plan-view minimap highlighting the target, and a constantly visible icon depicting the appearance of the target object. The cue was incorrect in 17% of the trials for one group of participants and 100% correct for the second group. Through both analysis and modeling of both search speed and accuracy, the results indicated that the arrow and minimap cues depicting location information were more effective than the icon cue depicting visual appearance, both overall, and when the cue was correct. However, there was a tradeoff on the infrequent occasions when the cue erred. The most effective AR-based cue led to a greater automation bias in which the cue was more often blindly followed without careful examination of the raw images. The results speak to the benefits of AR and the need to examine potential costs when AR-conveyed information may be incorrect because of imperfectly reliable systems.