Abstract:

Surgical annotation promotes effective communication between medical personnel during surgical procedures. However, existing approaches to 2D annotations are mostly static with respect to a display. In this study, we propose a method to achieve 3D annotations that anchor rigidly and stably to target structures upon camera movement in a transnasal endoscopic surgery setting. This is accomplished through intra-operative endoscope tracking and monocular depth estimation. In total, 2 deep neural networks are involved in monocular depth estimation. The first one is an adversarial network that transfers image style from the real endoscopic view to a synthetic-like view, while the second one is a fully-supervised depth estimation network that is trained in a virtual endoscopic environment prior to the prediction phase. Accuracy and stability evaluations are performed on an anatomically-accurate nasal airway phantom to demonstrate the feasibility and practicality of our proposed method for achieving 3D annotations.