Natural Interaction with Video Environments Using Gestures and a Mirror Image Avatar

Video environments are a promising option for a variety of applications such as training, gaming, entertainment, remote collaboration, or user studies. Being able to interact with these environments enables further applications and extends existing application scenarios. In this paper, we propose a novel interaction technique that combines natural gestures with mirror images of the user to allow for immersive interaction with video environments. The technique enables movement inside the 3D space depicted by the video as well as the placement and manipulation of virtual objects within the 3D space. We describe a potential application scenario, where interactive public displays are placed inside a scene by one user and then experienced by another user. We also briefly report on a user study evaluating the gesture set we defined for controlling movement within the video.