Stereo reconstruction and 3D rendering for intraoperative fluorescence lifetime imaging visualization

Robotic and endoscopic surgery is increasingly used in clinical practice and typically relies on stereoscopic vision to enable 3D visualization of the surgical field. We combined this capability with a FLIm acquisition system suitable for the identification of tumor tissue to generate a 3D map of the surgical field that comprises both FLIm and white light image information. This result is achieved using semi-global matching and deep stereo matching neural network. In addition to the generation of a 3D model of the surgical cavity, this approach leads to a more realistic rendering of FLIm maps by including tissue shading.