Enhancing Sentient Embodied Conversational Agents with Machine Learning

Abstract Within the area of intelligent User Interfaces, we propose what we call Sentient Embodied Conversational Agents (SECAs): virtual characters able to engage users in complex conversations and to incorporate sentient capabilities similar to the ones humans have. This paper introduces SECAs together with their architecture and a publicly available software library that facilitates their inclusion in applications –such as educational and elder-care– requiring proactive and sensitive agent behaviours. In fact, we illustrate our proposal with a virtual tutor embedded in an educational application for children. The evaluation was performed in two stages: firstly, we tested a version with basic textual processing capabilities; and secondly, we evaluated a SECA with Machine-Learning-enhanced user understanding capabilities. The results show a significant improvement in users’ perception of the agent’s understanding capability. Indeed, the Response Error Rate decreased from 22.31% to 11.46% using ML techniques. Moreover, 99.33% of the participants consider the global experience of talking with the virtual tutor with sentient capabilities to be satisfactory.