Augmented reality enters the operating room

Some of the greatest medical advances in recent decades have been in diagnostic imaging : mammography, computed tomography (CT), magnetic resonance imaging (MRI) and so on. New technologies, consisting of tiny cameras and smaller, faster processors, have enabled real-time data streaming. These new realities have revolutionized the way surgeons use images in surgical procedures. Many interventions, before proceeding to the surgical phase of the incision, require an in-depth diagnostic study using a scan such as ultrasound or a CT scan to guide the procedure. While imaging diagnostic processing techniques have radically improved and evolved, the way they have been viewed to date is essentially the same. Visual data is always displayed on a 2D screen. In addition, doctors must use skills and imagination to understand and mentally 3D project images within the operating field. Finally, the different information extrapolated from the radiological images, displayed separately, forcing the surgeons to integrate, mentally merge multiple types of images and information collected, such as angiography and CT, into a single coherent representation. Acquiring this skill requires years of training, preparation and experience. Now augmented reality enters the operating room thanks to a wearable viewer that can assist and guide the surgeon’s eyes and hands during surgery. This goal was pursued by the European VOSTARS project coordinated by the Information Engineering Department of the University of Pisa. Numerous Italian and European partners are involved in this project, including research institutions, companies in the sector and clinical centers. The H2020 innovation and research program of the European Union, VOSTRARS – acronym for Video Optical See-Through Augmented Reality Surgical system – aims, in fact, at the creation of a wearable viewer with a hybrid approach, capable of integrating and optimizing the best has been studied and developed from the dawn of augmented reality to today. Thanks to this technology, the surgeon will be able to have in front of his eyes, without having to look away from the operating field, patient information. The use of the viewer allows to always have available the radiological images acquired before and during the surgery which, perfectly aligned with the patient’s anatomy, can allow the operator a view that goes well beyond the barriers of the visible world. Vision through virtual reality is a great help to guide the surgeon’s hand with extreme precision.