Material Recognition for Mixed Reality Scene including Objects’ Physical Characteristics

Mixed Reality (MR) is a technique to represent scenes which make virtual objects exist in the real world. MR is different from Augmented Reality (AR) and Virtual Reality (VR). For example, in MR scenes, a user can put a virtual Computer Graphics (CG) object on a desk of the real world. The virtual object can interact with the real desk physically, and the user can see the virtual object from every direction. However, MR only uses position and shape information of real world objects. Therefore, we present a new MR scene generator considering real world objects’ physical characteristics such as friction, repulsion and so on, by using material recognition based on a deep neural network.