Robotic mitral valve repair: 3D hologram to optimize port placement

Precision of ports in robotic surgery: the challenge of subjective expertise

In robot-assisted cardiac surgery, and particularly during mitral valve repairs, the optimal positioning of trocars in the chest wall remains a major technical challenge. Currently, this crucial step relies almost exclusively on the surgeon's experience and intuition, without real-time technological guidance. This inter-operator variability can impact the ergonomics of the instrumentation and, by extension, the outcome of the procedure.

This study, conducted from February to June 2024 in a hospital center in Bergamo (Italy), aims to develop and test a preliminary augmented reality (AR) prototype to address this uncertainty. The specific objective was to integrate the HoloLens 2® headset with the Da Vinci® surgical system, allowing practitioners to use holograms derived from preoperative CT scans superimposed directly onto the patient's thorax to guide port placement.

The authors tested the hypothesis that a specific software pipeline — combining Slicer3D® for 3D cardiac modeling and Unity® for visualization — would allow for reliable co-localization of virtual medical data. This device must validate the technical feasibility of an augmented surgical workflow, transforming static planning into an immersive and precise operative environment.

Methodology for the development of the AR prototype

This technical feasibility study, conducted from February to June 2024 in a hospital center in Bergamo (Italy), details the design of an augmented reality (AR) prototype intended for robot-assisted cardiac surgery, specifically for mitral valve repair. The objective is to assist the practitioner in the positioning of robotic trocars, a step currently depending on the operator's empirical expertise.

The experimental protocol is based on an integrated software architecture combining several cutting-edge technologies:

• Acquisition and modeling: Use of pre-surgical CT scanners to extract data in DICOM format. These data were processed via Slicer3D® software to generate a customized 3D model of the patient's heart.
• Integration pipeline: Development of an interface under the Unity engine to convert 3D models into virtual holograms usable in real time.
• Visualization device: Use of Microsoft HoloLens2® glasses for the co-localization of virtual representations on the patient's thorax in the operating room.

The study evaluated the viability of the workflow by integrating these models into the Da Vinci® surgical system. The analysis focused on the system's ability to accurately overlay internal anatomical structures onto the surface anatomy to guide the insertion of robotic instruments.

Results: A proof of concept for trocar positioning

The development conducted between February and June 2024 in Bergamo validated the technical feasibility of an augmented reality (AR) integration pipeline within a robotic cardiac surgery workflow. The results focus on the creation of a functional composite environment combining medical imaging and intraoperative vision.

3D model development and software integration

The study demonstrated the effectiveness of the DICOM data processing chain for generating patient-specific anatomical models. The validated key steps are as follows:

• Anatomical segmentation: Use of Slicer3D® software to transform pre-surgical CT scans into accurate 3D models of the heart.
• Interoperability: Successful integration of 3D models into the Unity engine to enable holographic visualization.
• Spatial projection: Holograms were successfully superimposed on the patient's thorax via HoloLens 2® glasses, providing a co-localized view of internal structures.

Technological components of the prototype

The table below summarizes the technologies whose integration was validated during this preliminary phase:

Qualitative observations and workflow

Although this preliminary study does not provide p-values (as it is a prototype test and not a comparative clinical trial), the authors report a theoretical improvement in the surgical workflow. The system allows the surgeon to visualize the internal cardiac anatomy directly through the patient's chest wall before the incision. This approach aims to reduce exclusive reliance on the surgeon's subjective expertise for robotic port positioning by providing a visual guide based on the patient's actual anatomy.

Towards holographic navigation of robotic movement

The results of this prototype developed between February and June 2024 mark a technical breakthrough: the successful integration of DICOM data pre-processed via Slicer3D® into an augmented reality environment (HoloLens 2®) synchronized with the Da Vinci® robot. Clinically, this means that the planning of trocar positioning — a critical step previously dependent exclusively on the surgeon's subjective expertise — can now rely on a co-localized 3D visualization of the patient's cardiac anatomy.

This study demonstrates the feasibility of a software pipeline (Unity/Slicer3D) capable of overlaying a precise anatomical hologram onto the actual thorax. While the literature highlights the benefits of robotic mitral surgery in terms of reduced hospital stays, this work specifically addresses the precision of the surgical approach. The contribution of augmented reality (AR) here allows for the transformation of static radiological data into a dynamic spatial guide.

However, the development stage remains preliminary. As it is a prototype, the study does not yet provide comparative data on operative time savings or the millimetric accuracy of registration under real-world conditions. Nevertheless, the implication is major: AR is becoming a concrete spatial assistance tool to secure the insertion of working ports by adapting to each patient's own morphology.

In concrete terms, for the practitioner:

• Optimization of trocar placement: Reduce reliance on empirical expertise alone by using 3D anatomical overlay to define optimal entry points according to the patient's specific morphology.
• Pre-incisional visualization: Access an immersive composite view merging virtual medical data and the real operative field to secure the surgical approach to the mitral valve.
• Integrated digital workflow: Anticipate the evolution towards image-assisted cardiac surgery where the CT scan is no longer used only for diagnosis, but becomes a dynamic intra-operative guidance tool.

Technical lexicon of the study

HoloLens2®: Head-mounted mixed reality device allowing the visualization of 3D medical holograms co-located with the patient's actual anatomy.

Slicer3D®: Medical image processing software used in this study to segment pre-surgical CT scans and generate three-dimensional cardiac models.

Unity: 3D development engine used for software integration and visualization management of anatomical models projected onto the patient's thorax.

DICOM (Digital Imaging and Communications in Medicine): Medical imaging data management standard used as the primary data source for the reconstruction of holographic models.

Trocar positioning (Port positioning): Critical step in robotic surgery consisting of placing instrumental accesses, here guided by the superposition of holograms to increase precision compared to manual expertise alone.

Mitral valve repair: Robot-assisted minimally invasive cardiac surgery procedure (Da Vinci® system) serving as a clinical application model for this augmented reality prototype.

Information intended for healthcare professionals. This content may contain errors or truncated summaries. We recommend always verifying with the original source article. Delynov disclaims any responsibility for the use of this information. This document is not intended for patients or the general public.