[Hidden] / [Caution] / [Danger]: How Video Games Can Inform the Design of Sight Cues for Agents

As artificial agents proliferate into society, there will be more and more situations in which they need to communicate their capabilities to humans, including what they can “see.” Humans do this with each other by using mental models of human capabilities coupled with social cues that enable situation awareness. In order to design better agents, we analyze video games, which have been communicating to humans about agent visual perception for decades. We present preliminary findings from a qualitative thematic analysis of sight cues in video games. We focus on three cue properties that warrant further study: whether the cue specifies the perceiver, whether the stimulus’ primary purpose is to be a sight cue, and the communication of non-binary sighting information. We close with an additional call for future work: on the effects of using multiple sight cues in combination.