Nonverbal Communication in Human-AI Interaction: Opportunities and Challenges

In recent years, we have explored the use of gaze—an important nonverbal communication signal and cue in everyday human-human interaction—for use with AI systems. Specifically, our work investigated whether an artificial agent, given the ability to observe human gaze, can make inferences on intentions, and how aspects of these inferences can be communicated to a human collaborator. We leveraged a range of humancomputer interaction techniques to inform the design of a gaze-enabled artificial agent that can predict and communicate predictions. In this paper, we include a snapshot of how AI and HCI can be brought together to inform the design of an explainable interface for an artificial agent. To conclude, we outline the challenges we faced when designing AI systems that incorporate nonverbal communication stemming from our work.