This project explores the field of Augmented Reality (AR) overlay with a purpose-built surgical microscope (Leica M500-N) that tracked intraoperatively with an optical tracker (NDI Optotrak Certus). It is developed a medical application concept (AROSCOPE – Stereo Augmented Reality in a Navigated Surgical Microscope), which uses the platform independent open-source libraries. For the microscope embedded surgery the patient is registered with landmark-based registration technique and the head of microscope is navigated at the same time with the patient together. Thereby segmented anatomic structures from preoperative radiological images and access pathways can be represented accurately as AR overlays in field of view (FOV) of surgeon, without the attention of surgeons is deflected from the surgical field.

The technical concept and core area of AROSCOPE involves: two dimensional (2D) visualization and spatial viewing of the patient images, calibration of the stereo cameras that mounted on the head of microscope, hand-eye calibration, patient-image registration, three dimensional (3D) segmentation, intraoperative 3D surgical navigation and overlaying of segmented anatomical structures into the ocular of the microscope in real-time.

This project provides the possibility to enhance the surgeon’s ability for a better intraoperative orientation by overlaying 3D information directly in the FOV.

Goal
The aim of this project was developing a surgical navigation and stereoscopic augmented reality concept for the use in minimal invasive microscopic image-guided surgery in ENT operations.

The concept serves user-friendly interactions and all essential steps from planning until overlaying phase and fulfills important requirements for a successfully surgery in minimal invasive interventions with navigated surgical microscope. To enhance the ability of the surgeon during the surgery and facilitate the whole complex workflow, AROSCOPE offers step by step processing of major requirements and clear guideline for the user. Consequently this system enhances the capabilities of the surgeon visual system through the combination of computer generated graphics, computer vision and advanced user interaction technology.

Screenshots

Funded by the Jubilee Fund of the Austrian National Bank, project 13003 (2008 – 2012, ongoing).

Detailed information can be found in Downloads and Publications pages.

Y. Özbek, W. Freysinger

• 
•