Broadcasting surgeries live for training purposes

The essence

In cooperation with medical device company Becton Dickinson (BD) and Dr. Tollens of the Imelda hospital, the HoloLens will be used for training and educational purposes within the medical field. Upon their request, we developed a HoloLens application that visualizes patient information during surgery. At the same time, both the patient data and camera footage of the smartglasses device can be broadcasted worldwide.

The challenge

Training and educating surgeons

BD, a global medical device company, strongly believes in the potential of augmented reality and wanted to investigate how a smartglasses device such as the HoloLens can bring added value. As part of this project, we were introduced to Dr. Tim Tollens, general and abdominal surgeon at the Imelda Hospital in Bonheiden.

Even long before Covid-19, complex surgeries were often broadcasted. This allows doctors from all over the world to follow along and learn new procedures, without losing valuable time on traveling. Generally, a camera crew is present in the OR to film the entire surgery from a third person view.

With AR technology, we saw the opportunity to broadcast the surgery from a first-person view whilst also visualizing relevant patient data. After all, during surgery, Dr. Tollens often relies on relevant information such as 3D scans, X-rays and even PowerPoint slides containing medical information about the patient.

The Solution

As it’s important for Dr. Tollens to consult patient data without blocking his view, we created an application for the Microsoft HoloLens. The application visualizes a holographic window containing all the required information. Thanks to the hands-free use of the HoloLens, he can consult the data by using his voice.

Because of the built-in cameras of the smartglasses device and its ability to connect to WiFi, live footage of both the real-life environment and the holographic information projected onto it can be shared.

The fact that the HoloLens is being used to film footage from a first-person view allows spectators to look “through the eyes” of Dr. Tollens. This results in a highly detailed view of the patient’s situation. On top of that, viewers have front row seats to follow along with operating actions and procedures.

The result

HoloLens assisted surgery

Dr. Tollens has already performed several complex open surgeries while being assisted by the HoloLens and our custom-made application. One of these surgeries was already broadcasted live as a registered training program to healthcare professionals all over the world.