IMMERSE: Interactive Mentoring for Multimodal Experiences in Realistic Social Encounters

Abstract : This final report details the IMMERSE effort under the SSIM program. SSIM is designed to address the challenge of training our warfighters to successfully engage with Others (people who are culturally different from oneself) to accomplish a diverse set of military missions. The SSIM IMMERSE effort involves demonstrating the possibility of conducting that training using an immersive, computer-based training system that embodies a mentored practice environment. As part of this effort, we are addressing key technical challenges including, developing interactive, immersive social situations populated by robust social agents, illustrating the keen eye and subtle guidance provided by a human mentor for social competence through intelligent tutoring, and enabling people to interact with simulated characters the way they interact with each other, through speech, gesture, comportment, etc.