Bryan Ladd, MD (left)
Neurosurgical Resident, University of Minnesota
Minneapolis, MN
Ann M. Parr, MD, PhD, FAANS, FRCSC (middle)
Assistant Professor, Neurosurgery and Director, Spinal Neurosurgery
University of Minnesota
Minneapolis, MN
Andrew W. Grande, MD, FAANS (right)
Assistant Professor, Neurosurgery and Co-director, Earl Grande Stroke and Stem Cell Laboratory
University of Minnesota
Minneapolis, MN
According to several new sources, virtual reality was the coolest tech gift for Christmas 2016. With holiday shopping fading in our rear view mirrors, the impact of this kind of technology on health care, specifically on neurosurgery, is worth exploring. Neurosurgery has always embraced new technologies, and virtual reality is now being used in new and exciting ways. First a primer on virtual reality (VR).
It was only just a few years ago that the concept of virtual reality was considered science fiction, but predictably it is now a part of our daily lives.
- VR: the immersion of a user in a virtual environment, intended to remove the user from reality and place them in a virtually defined space.
- Mixed reality (MR): More than VR, bringing the artificial world into our own.
- Augmented reality (AR): The most recent iteration is blurring the distinction between the artificial world and reality, and the implications are profound.
From Fun to Function
From its beginnings as an entertainment platform, VR has found usefulness as an adjunct to training programs aimed at developing hands-on skills. The U.S. Air Force has used virtual reality to help train pilots since 1966. Similarly, VR units are available to develop surgical skills. The virtual learning environment allows learners to try and fail, which is associated with improved learning when compared with passive observation. Importantly, young surgeons can learn with zero risk to real patients. When it comes to actual patient care, however, virtual reality often comes up short. This is where mixed reality technologies — the next generation of virtual reality — come into play.
Mixed reality is a term to describe a spectrum of technologies that fall somewhere between reality and virtual reality. This is a relatively broad group of technologies that encompass many different approaches at melding reality with a virtual world. The popular game, Pokémon Go, is an example of mixed reality and falls into the category of location-based games that prompt players to find and capture virtual Pokémon hidden in the real world. When a Pokémon is encountered, the game takes the real-time video input from the mobile device’s camera and interlaces the Pokémon into it. The player sees this mixed reality on their device’s screen.
Within the spectrum of mixed reality falls the fast growing subset known as augmented reality. As the name suggests, this technology’s goal is the augmentation of reality with virtual information. The user still sees their surrounding “real” environment, but with virtual images superimposed. An example of this is Google Glass, which acts as a heads-up display, allowing the user to see virtual information superimposed over their visual field. Neurosurgeons have long labored to bring such systems into use for the benefit of our patients (see Virtual reality neurosurgery: a simulator blueprint). Today, this technology has made its way into the neurosurgical operating room providing surgeons with a heads-up display that is directly injected into the microscope. Analogous to the driver with a heads-up display in the windshield, the surgeon is free to operate without needing to look away.
Several devices have taken augmented reality to the next level by adding sophisticated sensing technology that allows the device to map the user’s surrounding environment. Using clever programming, the devices can then work to merge the virtual environment with reality. So what does this mean exactly? Let’s place a virtual rocket ship on a real table. With the Microsoft HoloLens, a device that can sense the environment, the edges of the table are recognized and act as a reference. As you move your head, the HoloLens “sees” the table reference points move and adjusts the virtual location of the rocket ship to keep the appearance of it being on the table.
Neurosurgery’s Future: VR at Work
How does this spin out within neurosurgery? Beyond the inclusion of VR devices in surgical training programs to enhance patient safety, this exciting next generation technology has amazing potential. Advanced image recognition algorithms are being developed to recognize faces and other landmarks on the human body. With references set, CT or MRI images could be merged with reality to give the user a view of the structures beneath the skin. This would enable a neurosurgeon to have real-time image-guidance based on surface registration (advanced GPS for the brain and spine). This will have a profound impact on everything from positioning in the operating room to surgical planning and execution. This is the future. The next revolution that has been quietly gaining momentum. Neurosurgery continues to lead the way embracing technology to enhance education, patient care and quality outcomes.
