Augmented Reality in Minimally Invasive Digestive Surgery

Minimally invasive surgery represents one of the main evolutions of surgical techniques. However, minimally invasive surgery adds difficulty that can be reduced by computer technology. Indeed, from a patient’s medical image (US, CT, or MRI), augmented reality (AR) can increase the surgeon’s intraoperative vision by providing virtual transparency of the patient. AR is based on two main processes: 3D visualization of the anatomical or pathological structures appearing in the medical image and registration of this visualization onto the real patient. 3D visualization can be performed directly from the medical image by direct volume rendering or after image processing by 3D surface rendering. Registration can be performed interactively or automatically. Several interactive systems have been developed and applied to humans, demonstrating the benefit of AR in surgical oncology, but also their limits due to tool tracking and organ deformation. Automatic AR (AAR) efficiently overcomes the surgical instrument tracking limitations, but is still currently imperfect due to the high complexity of real-time organ deformation and human movements. However, the first results of AAR show that it will become a system requirement for future computer-assisted surgical oncology. In this article, we will illustrate several applications and results of AR in digestive surgical oncology.