{"id":303590,"date":"2012-06-07T09:00:11","date_gmt":"2012-06-07T16:00:11","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=303590"},"modified":"2016-10-11T10:35:07","modified_gmt":"2016-10-11T17:35:07","slug":"kinect-launches-surgical-revolution","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/kinect-launches-surgical-revolution\/","title":{"rendered":"Kinect Launches a Surgical Revolution"},"content":{"rendered":"

By Douglas Gantenbein, Senior Writer, Microsoft News Center<\/em><\/p>\n

Medical imaging today gives surgeons an ability to obtain a virtual peek inside the human body in a way that rivals the campy 1966 movie Fantastic Voyage<\/em>, in which a team of physicians is miniaturized and injected into a wounded diplomat to save his life.<\/p>\n

In today\u2019s operating theater, though, those virtual images can be difficult to retrieve and manipulate. A surgeon can\u2019t use a mouse or keyboard, because they are unsterile and pose a risk of infection. A surgeon can give instructions to someone else, who then can operate a mouse or keyboard, but the surgeon must break concentration to say, \u201cRight \u2026 right \u2026 up a little \u2026 OK.\u201d<\/p>\n

Now, a team at Microsoft Research Cambridge<\/a> consisting of social scientists, computer scientists, and designers has used the Kinect for Windows<\/a> hardware and SDK to simplify this process. It enables doctors to use simple hand gestures to change, move, or zoom in on CT scans, MRIs, and other medical images. In initial development, the Kinect for Windows-based system has thrilled surgeons who have seen it and who believe it could help make surgery faster and more accurate. The hope is that these systems will deliver better outcomes to patients when fully field-tested and approved.<\/p>\n

\"Helena

Helena Mentis<\/p><\/div>\n

The project, called Touchless Interaction in Medical Imaging<\/a>, to bring Kinect for Windows into surgical suites has been guided by Helena Mentis, a Philadelphia native and a postdoctoral researcher with the Socio-Digital Systems<\/a> group at the Cambridge lab. She has been intrigued for a long time by how technology can improve medical practices. She earned a Ph.D. in Information Sciences and Technology from The Pennsylvania State University, writing a thesis on how expressions of emotion between health-care workers helped foster collaboration in emergency rooms.<\/p>\n

Microsoft Research Cambridge\u2019s extensive work in exploring new technology attracted Mentis to that facility. The lab\u2019s researchers also were instrumental in delivering many of Kinect for Windows\u2019 capabilities, so when Mentis heard about Kinect, she soon began to investigate how it might be deployed in medical settings.<\/p>\n

\u201cEven before Kinect came on the market, my colleagues and I started to do some fieldwork with an eye on whether something could be developed for surgical theaters,\u201d Mentis says. \u201cThe initial idea was to learn where we would want to use the system and what type of systems could interact with it in the surgical area.\u201d<\/p>\n

When Kinect for Xbox 360<\/a> hit the market in November 2010, Mentis was ready to go. Since then, her team has worked closely with surgeons to test it at several hospitals, particularly vascular surgeons at St. Thomas\u2019 Hospital, located prominently across the Thames River from the British Houses of Parliament.<\/p>\n

Vascular surgeons often can avoid performing open-heart surgery by placing stents to open clogged arteries or to support aortas with an aneurysm. But proper placement of the stent requires a great deal of precision, and medical imaging plays a big role in enabling that precision.<\/p>\n

\"manipulate

Kinect for Windows enables surgeons to use gestures to manipulate medical imagery while in the surgical theater.<\/p><\/div>\n

By using the Kinect for Windows sensor in conjunction with a large monitor, a surgeon easily can change the image view of a patient\u2019s chest area, look at a new image, or zoom in on details\u2014all without touching a screen, a keyboard, or a mouse. That can help the surgeon determine more quickly the proper placement of a stent.<\/p>\n

\u201cThey love it,\u201d Mentis says of surgeons\u2019 reactions to seeing Kinect in action. \u201cThe idea of being able to control almost anything with just a wave of the hand really speaks to how surgeons want to feel like they have control over things. They want to be able to save as much time as possible and to be able to get a patient in and out of anesthesia as quickly as possible. Those all are things that they\u2019re motivated by, so they think this is the greatest thing ever.\u201d<\/p>\n

In the operating room, the setup consists of a Windows PC connected to a fluoroscopy scanner through an external video-capturing device, a Kinect for Windows sensor attached to the PC with a USB cable, and an LCD monitor.\u00a0The Kinect for Windows sensor is placed below or above one of the computer monitors that the surgeons currently use for visualising medical images.<\/p>\n

Key 3-D medical images are acquired before the surgery using a CT (computed tomography) scanner. During surgery, the fluoroscopy scanner captures real-time X-ray images, which are overlaid on one another, registered, and displayed in an image viewer. A surgeon then uses gestures to rotate images, zoom into details, and more.<\/p>\n

Although still being tested, the Kinect for Windows sensor, Mentis said, works well in the surgical suite. It\u2019s able to track and follow multiple people, something other camera-based gestural systems cannot do. It also could be \u201ctrained\u201d to know which individual in a surgical suite wanted to use the system, which can be a challenge, given that everyone essentially is masked and dressed the same.<\/p>\n

To do that, the Kinect sensor is programmed to respond to the command, \u201cKinect, control,\u201d in conjunction with a raised hand. The sensor then \u201cwatches\u201d that individual for instruction, ignoring other people who might themselves be gesturing and talking.<\/p>\n

Surgeons also must avoid swinging their arms in too wide of an arc while working with the Kinect for Windows hardware to avoid touching a contaminated object. The system compensates by supporting gesturing in a limited area in front of the surgeon\u2019s body, using the Near Mode feature in Kinect for Windows.<\/p>\n

\u201cThis is very different than designing for a game, where one can flail about wildly,\u201d Mentis says. \u201cThe surgeon\u2019s sterile area is from chest to waist and from shoulder to shoulder. We worked hard to ensure the surgeon could fully use the system while staying within a safe and sterile zone.\u201d<\/p>\n

But otherwise, sterility isn\u2019t a particular challenge. If the Kinect for Windows sensor and its accompanying monitor are kept clean and left untouched, the risk of infection is substantially reduced.<\/p>\n

In March, Mentis and her team began to test the first iterations of the system with several vascular surgeons in a surgical theater without patients. By the beginning of May, the first field-trial test with patients was under way.<\/p>\n

Surgeons who have worked with the research team are eager to give the system a regular place in surgery, once finalized.<\/p>\n

\u201cAs a surgeon, being able to manipulate information in a touch-free way while operating will have enormous benefits,\u201d says Dr. Neville Dastur, a vascular surgeon at the Brighton and Sussex University Hospitals National Health Service (NHS) Trust. \u201cBreaking the flow of the operation to get someone to do it for you not only distracts from performing a delicate surgery, but it\u2019s also time-consuming.\u201d<\/p>\n

Dr. Tom Carrell, vascular surgeon at Guy\u2019s and St. Thomas\u2019 NHS Foundation Trust, says adding Kinect for Windows to the surgical theater could be a big step forward.<\/p>\n

\u201cWith Kinect, we could revolutionize the way we do complex operations,\u201d Carrell says. \u201cPatients and clinical teams will spend less time in theater, and surgeons will be more in control of the information they need.\u201d<\/p>\n

More Possibilities<\/h2>\n

The Kinect for Windows-based imaging system, Mentis says, already is being evaluated for other new uses. One example is the manipulation of 3-D models of the brain for neurosurgery at Addenbrooke\u2019s Hospital in Cambridge.<\/p>\n

\u201cNeurosurgeons are interested in being able to go deep into the brain and extract a tumor or to even do a biopsy for a tumor,\u201d she says. \u201cUnfortunately, tumors never are right on the outside of the brain\u2014they typically are within the brain tissue. The idea is to get in there with the minimal amount of brain damage possible, and better ability to view imaging can help with that.\u201d<\/p>\n

Another aim is to develop a touchless interaction-in-surgery tool kit that can be used by any hospital or system interested in applying touchless interaction to an imaging system.<\/p>\n

Mentis is working with an extensive team to make the Kinect for Windows-based surgical-suite setup a reality. Partners at Microsoft Research Cambridge included principal researcher Abigail Sellen<\/a>, senior researcher Antonio Criminisi<\/a>, designer Robert Corish<\/a>, and vendor Kenton O\u2019Hara. The team also worked with Mark Rouncefield and Gerardo Gonzalez from Lancaster University\u2019s Computing Department, which is supported by Microsoft Research Connections<\/a>. The team, which also included participation from King\u2019s College London, hopes that innovations in technology using Kinect for Windows that make surgery easier and faster will obtain appropriate governmental approval.<\/p>\n","protected":false},"excerpt":{"rendered":"

By Douglas Gantenbein, Senior Writer, Microsoft News Center Medical imaging today gives surgeons an ability to obtain a virtual peek inside the human body in a way that rivals the campy 1966 movie Fantastic Voyage, in which a team of physicians is miniaturized and injected into a wounded diplomat to save his life. In today\u2019s […]<\/p>\n","protected":false},"author":39507,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"footnotes":""},"categories":[194480,194484],"tags":[213608,213605,193602,213614,196135,186813,213611,204315,213602],"research-area":[13551,13553],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-303590","post","type-post","status-publish","format-standard","hentry","category-graphics-and-multimedia","category-medical-health-and-genomics","tag-ct-scan","tag-fluoroscopy-scanner","tag-kinect-for-windows","tag-kinect-for-windows-based-imaging-system","tag-kinect-for-xbox-360","tag-medical-imaging","tag-near-mode-feature","tag-touchless-interaction-in-medical-imaging","tag-vascular-surgeons","msr-research-area-graphics-and-multimedia","msr-research-area-medical-health-genomics","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[199561],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[],"related-projects":[170869],"related-events":[],"related-researchers":[],"msr_type":"Post","byline":"","formattedDate":"June 7, 2012","formattedExcerpt":"By Douglas Gantenbein, Senior Writer, Microsoft News Center Medical imaging today gives surgeons an ability to obtain a virtual peek inside the human body in a way that rivals the campy 1966 movie Fantastic Voyage, in which a team of physicians is miniaturized and injected…","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/303590"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/39507"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=303590"}],"version-history":[{"count":3,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/303590\/revisions"}],"predecessor-version":[{"id":303614,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/303590\/revisions\/303614"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=303590"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=303590"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=303590"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=303590"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=303590"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=303590"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=303590"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=303590"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=303590"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=303590"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=303590"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}