How the healthcare sector is innovating
Whenever we set up at a large conference – as we did at Giant Health in London last month – we’re almost always buoyed by the reaction we receive. When it comes to mixed reality seeing is certainly believing. So, it’s heartening to hear a steady chorus of “wows” and “blimeys” as we place HoloLens 2 devices on the heads of inquisitive delegates.
Those people often go on to tell us they had barely realised this technology existed. Or they seem to believe it’s brand new. However, the HoloLens has been around for five years and it’s currently in its second iteration.
In many ways, this discrepancy is understandable. If clinicians and healthcare professionals haven’t experienced mixed reality for themselves they can easily dismiss HoloLens alongside the litany of over-hyped innovations which eventually go nowhere.
Also, if you’re of a certain generation in the UK, and grew up watching programmes like ‘Tomorrow’s World’ – you could be forgiven for having an almost reflexive scepticism about some technology. After all, wasn’t everyone supposed to have a jetpack by about 1995? And whatever happened to meals in ‘pill form’?
Oddly enough, that scepticism is far less evident in the NHS. There seemed to be a fervent appetite for innovation within several of the trusts we work with. Recently, digital modernisation and innovation has accelerated. That momentum is showing no signs of waning. This enthusiasm is spreading to mixed reality, and HoloLens 2 in particular.
But before we go any further and talk about some of the ways HoloLens is being applied in the NHS – we should probably establish exactly what we mean by ‘mixed reality’.
Mixed reality in focus
“Oh, so this is augmented reality, is it?”
“What’s this, a VR headset?”
Two of the questions we frequently get asked when attending events like Giant Health. And to be fair, the person on the street is far less familiar with the concept of mixed reality, as opposed to augmented reality (AR) and virtual reality (VR).
AR overlays a digital element onto the physical world while VR creates a wholly separate virtual world. Mixed reality is about blending the physical and the digital together. The phrase we often use at Microsoft is: Embedding computing into the real world, and the real world into computing.
In other words, when you’re wearing a HoloLens 2 headset, you can still see the real world. But you’re bringing in interactive digital content. And as you move around the space in the real world, that digital content remains in a fixed position. If you drop a virtual ball, for example, it will hit the ground and bounce back at you – because HoloLens is aware of and constantly mapping out your real environment.
How the NHS innovates and collaborates with the HoloLens
Over the last couple of years, we’ve been consistently impressed by the eagerness on the part of the NHS to experiment with technology and innovate. In fact, when it comes to mixed reality, it’s no exaggeration to say the NHS is at the very forefront.
One of the projects we’re particularly proud of is our collaboration with Imperial College Healthcare NHS Trust.
When the pandemic hit, the trust found itself on the frontline. Not only treating patients but trying to simultaneously protect its frontline staff inside some of the busiest hospitals across the most high-risk areas of London. They began using HoloLens 2 with Dynamics 365 Remote Assist and Microsoft Teams to reduce the number of people needed on ward rounds.
A doctor wearing a HoloLens headset would send a secure live video feed to a computer screen in a nearby room. This allows the rest of the team to see and hear everything on the ward and support the doctor while remaining at a safe distance.
Imperial College Healthcare NHS Trust stated that HoloLens led to as much as an 83 percent reduction in the time staff needed to spend in high-risk areas. It also reduced the need for PPE significantly, because suddenly only the doctor with the headset needed it. Estimates even suggest that using HoloLens saved up to 700 items of PPE per ward, per week.
Using innovation to build skills in students
It’s fair to say the success at Imperial College Healthcare NHS Trust created some momentum. We set up a monthly call with interested NHS trusts, so we could share knowledge and discuss other areas where the technology could be put to work. As we put the technology in the hands of clinicians and educators we saw innovative adoption from patient education to multi-disciplinary teams, surgical planning, to community outreach. In particular, it enables remote access to medical students to complete virtual ward rounds. We have seen development of procedural guides for nurse and clinician training and adoption of patient simulation software.
And one key outcome was our association with Health Education England (HEE), which is the executive body for healthcare training across England. They were unable to effectively train staff on how to treat COVID-19 patients, while medical students were falling behind with their practical tutoring.
So, we assembled a team of top-tier partners, including Insight, GigXR, Fracture Reality and Dimension, to create training resources using HoloLens. This helped HEE address this growing skills gap and cross-train existing staff.
Likewise, Case Western University in the US used HoloLens to develop its ‘HoloAnatomy’ application, which combines cadaver dissections with mixed-reality simulations. Students are, in some cases, improving their overall grades as a result of this initiative. In the longer term, students in the HoloAnatomy lab scored a staggering 50 percent better on their retention tests and required 40 percent less class time than previous years. Medical students can also use the HoloLens for remote education.
Results like this have led us to help found an independent body called the Holomedicine Association. Here, medical professionals can come together, swap ideas and share best practice. The body is also exploring the standards, certifications, policies and research that will soon be vital, if mixed reality continues its current trajectory.
Connecting experts and clinicians together remotely
Of course, nothing beats real life, hands-on experience. Especially in education. We have no intentions to replace in-person interaction with mixed reality. Rather, we aim to complement it where possible. Clinicians can use mixed reality to build muscle memory undertaking mixed reality procedural training in simulation labs such as University of Michigan, Imperial and East Lancashire Hospitals. Or experts can easily oversee student or community carer in the field to give guidance and supervision. Collaborative mixed reality platforms are enabling students to get exposure to training anywhere in the world. At Imperial, Michigan and Alder Hey, they are conducting Grand Ward Rounds using solutions such as JoinXR – Fracture Reality.
However, there may also be times when practical experience is too hard to come by. For example, many surgeons have told us about those extraordinary, ‘once in a career’ procedures – and how useful mixed reality could be in preparing for them.
Likewise, there may be a handful of global experts in a single, rare condition or procedure – and they can’t be in two places at once. Not physically, at least. But with mixed reality, they can consult and even play an active role in treatments on different continents in the same afternoon. Language doesn’t present a barrier either, because the technology already exists to listen and translate voices within HoloLens, in real time. Partners such as ApoQlar are enabling the ability to visualise medical image data in 3D and collaborate immersively.
That, in turn, feeds into a wider point about sustainability. Of course, a world-famous surgeon may have less need to jet around the globe if they can use HoloLens effectively. However, it’s not just in such rarefied contexts that mixed reality can make healthcare greener.
For example, one of our partners operates care homes in an area of roughly eight square-miles around Torbay, Devon. That sounds like a small-level operation, doesn’t it? Until we tell you that their community nursing team clocks over a million miles of travel per year. In that situation, a few dozen HoloLens headsets could help enable clinical experts to cover more patients and increase access to care – a remarkable environmental impact that two organisations, Kendall Care Homes and Torbay Breast Care Unit, are already seeing after adopting HoloLens 2 in 2021.
Now is the time to commit to mixed reality
Innovation in healthcare is a huge topic. And while our special interest lies in mixed reality, that doesn’t mean we’re blind to the other startling breakthroughs in the sector. There are few more impressive than the application of AI and machine learning to diagnostics.
For example, at Microsoft, we’ve partnered with the Mental Health Trust to explore certain models for predicting psychotic episodes. Elsewhere, predicative analytics is being applied to conditions like sepsis and hypertension with impressive results. We are also seeing the use of mixed reality in teaching soft skills through programmable avatars and applications in rehabilitation.
The wider point is that we’re not discussing this technology in the abstract. Mixed reality, AI and machine learning are not the jetpacks or pill meals of Tomorrow’s World. Far from it. Tools like HoloLens 2 are being used by healthcare professionals in the NHS and elsewhere today. And the more we combine them with other capabilities like cognitive services and IoT – the more powerful they will become.
Find out more
Forrester Industry Report: Mixed Reality in Healthcare
Resources to empower your developer team
About the authors
Joe Varrasso is the Europe Mixed Reality Partner Lead, building out an ecosystem of key industry partners helping to innovate with Mixed Reality in key industries. Specifically, Joe has also been focussed on the Healthcare Industry and is a Founding Member of the Holomedicine Association.
Alex leads the IoT and Mixed Reality businesses for Microsoft UK. Prior to Microsoft, Alex led the deployment of HoloLens 2 for McLaren and later worked with Microsoft partner Kognitiv Spark. In his new role at Microsoft, Alex is focussed on accelerating the adoption of Metaverse technologies such as Mixed Reality, IoT and Azure in the UK.
