Augmented reality enters the operating room

Some of the greatest advances in medicine in recent decades have been in diagnostic imaging: mammography, computed tomography (CT), magnetic resonance imaging (MRI), and so on.

New technologies, built on tiny cameras and ever faster processors, have enabled real-time data streaming. These developments have revolutionised the way surgeons use imaging in operative procedures.

Many surgeries, before moving on to the incision phase, require diagnostic imaging such as ultrasound or CT scans to guide the procedure.

While diagnostic imaging techniques have radically improved and evolved, the way images have been displayed until now has remained essentially the same: visual data projected on a 2D screen.

Surgeons must therefore rely on skill and imagination to mentally project these images into 3D inside the surgical field. Furthermore, radiological information is displayed separately, forcing surgeons to mentally merge multiple image types, such as angiography and CT, into one coherent representation.

Acquiring this ability requires years of training, aptitude, and experience. Now augmented reality is entering the operating room, with a wearable headset that can assist and guide the surgeon’s eyes and hands during procedures.

This goal has been pursued by the European project VOSTARS, coordinated by the Department of Information Engineering at the University of Pisa. The project involves numerous Italian and European partners, including research institutes, companies, and clinical centres.

The EU’s H2020 research and innovation programme, VOSTARS – standing for Video Optical See-Through Augmented Reality Surgical System – aims to deliver a wearable device with a hybrid approach, integrating and optimising decades of augmented reality research and development.

Thanks to this technology, surgeons can keep critical patient data right in front of their eyes, without having to look away from the operating field.

The headset ensures that radiological images, acquired before and during surgery, are perfectly aligned with the patient’s anatomy, offering surgeons a view that goes far beyond the limits of the visible world.

Augmented reality provides a powerful tool to guide the surgeon’s hand with extreme precision.