Stereoscopic augmented reality for operating microscopes

Abstract Three-dimensional overlay of objects onto the operating field has big advantages compared to current commercial approaches. There are several problems which can be solved by providing a full color stereoscopic overlay to the surgeon, orienting himself by looking away from the operating field onto a navigation monitor as well as merging the overlaid two-dimensional contours of an object correctly to a three-dimensional structure. This paper introduces a conceptual approach for a full color, stereoscopic overlay of objects onto the operating field using the optics of the operating microscope. Models segmented from preoperatively acquired image data are inserted into the optics of the operating microscope. We use the virtual camera approach to provide the collimation. In the scope of developing a prototype for image overlay, we analyzed current display technologies to determine the best display technique to use. We will present the general process model for our approach, the visualization concept using the virtual camera model, the current workflow, the prototype and a brain phantom which we developed for our needs. The work is supported by the Deutsche Forschungsgemeinschaft (DFG) as a part of the Priority Programme 1124: “Medical Navigation and Robotics”.