{"id":236987,"date":"2011-07-26T16:00:32","date_gmt":"2011-07-26T23:00:32","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=236987"},"modified":"2016-07-20T07:33:45","modified_gmt":"2016-07-20T14:33:45","slug":"touch-mouses-tale","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/touch-mouses-tale\/","title":{"rendered":"A Touch Mouse\u2019s Tale"},"content":{"rendered":"

How do you take a concept from research to product? In the case of the Microsoft Touch Mouse (opens in new tab)<\/span><\/a>, it took a collection of prototypes, collaboration between transatlantic teams, and a lot of user testing. It also helps when the research that launched the project (opens in new tab)<\/span><\/a> won the best-paper award during the Association for Computing Machinery\u2019s 22nd Symposium on User Interface Software and Technology (opens in new tab)<\/span><\/a>.<\/p>\n

Mouse 2.0: Multi-Touch Meets the Mouse (opens in new tab)<\/span><\/a><\/i>, a joint effort between Microsoft Research Redmond (opens in new tab)<\/span><\/a>,\u00a0Microsoft Research Cambridge (opens in new tab)<\/span><\/a>, and Microsoft\u2019s Applied Sciences Group (opens in new tab)<\/span><\/a>, introduced five research prototypes, each exploring a different touch-sensing strategy that influenced the design of different mouse form factors and their interaction possibilities. The research featured extensive user feedback, as well as practical comparisons of different techniques for enabling multitouch on the desktop. The prototypes included three camera-imaging approaches, multiple optical sensors, and the use of capacitive sensors on a curved surface.<\/p>\n

Members of the Mouse 2.0 research team expressed the hope that they would be able to refine their prototypes, both ergonomically and in terms of their sensing capabilities, and make a deeper exploration of the interaction techniques specific to this new class of input devices.<\/p>\n

The researchers soon got an opportunity to refine their prototypes. Microsoft Hardware (opens in new tab)<\/span><\/a> decided to get behind the research, and a team was formed to bring a multitouch mouse to market.<\/p>\n

New Possibilities for the Humble Mouse<\/h2>\n

Hrvoje Benko (opens in new tab)<\/span><\/a>, researcher with the Adaptive Systems and Interaction (opens in new tab)<\/span><\/a> group at Microsoft Research Redmond, has worked on both the Mouse 2.0 research and the Microsoft Touch Mouse product-development project. He recalls one of the key product decisions: selecting from five prototypes the one that would be the launching point for the new device.<\/p>\n

\u201cIn the end,\u201d Benko says, \u201cwe selected the prototype using capacitive touch sensing to track the position of multiple fingers on its surface. This approach offered the most consistency and flexibility in terms of how we could mount and integrate the sensor, which is important in a small form factor. Plus, unlike camera-based tracking, there are no issues with ambient light, so you reduce the calibration issues. It\u2019s a much more controllable sensor.\u201d<\/p>\n

\n
\"Microsoft

Microsoft Touch Mouse: the final form factor.<\/p><\/div>\n<\/div>\n

Although the choice of prototype simplified some of the technical issues, there were still plenty of challenges when it came to refining the mouse to the point where it was ready for consumer use. The design of the final form factor required sculpting and testing of hundreds of models. The team also examined user interactions and evaluated the kinds of gestures that made sense, developing an entire gestural set focused on enhancing window manipulation and management. At the same time, core technologies, such as firmware and hardware for capacitive sensing, had to be built and optimized for this specific form factor and device functionality.<\/p>\n

\u201cThe gesture-recognition software is the brains behind all these interactions,\u201d says John Miller (opens in new tab)<\/span><\/a>, software architect with the Cambridge Innovation Development (opens in new tab)<\/span><\/a> team at Microsoft Research Cambridge. \u201cOur gestures are multitouch and designed to amplify your experience with Windows 7 (opens in new tab)<\/span><\/a>. So they are optimized for window management: docking, moving, minimizing and maximizing, going backward and forward on your webpage, switching between tasks, and so on.\u201d<\/p>\n

Getting the Right Touch<\/h2>\n

Benko and Miller agree that one of the toughest problems they tackled was the requirement that users should be able to operate the device using classic point-and-click interactions, as well as the newly developed set of multitouch gestures. The mouse form itself added complications: The shape encourages a user to rest both palm and curled fingers on the entire touch-sensitive surface, creating constant contact.<\/p>\n

\u201cThat made everything much, much harder,\u201d Benko smiles ruefully. \u201cInstead of making palm rejection and other issues easier, it added a few more challenges. But at the end of the day, our goal was to have a comfortable, great-looking mouse that people enjoy using, with a nice look and feel that support the gestures, so it was definitely worth the effort.\u201d<\/p>\n

Unlike touch-screen devices on which one or two clear touches make user commands easy to interpret, a small sensor surface and the nature of mouse usage creates an entirely different set of problems.<\/p>\n

\u201cIf you have a touch-sensitive phone,\u201d Miller explains, \u201cyou interact by touching the screen, and as soon as you\u2019re done, your finger lifts off the screen. We have completely different issues with the mouse. We have a device that not only has to support gesture touches, but also has to deal with times when the user is just holding it.<\/p>\n

\n
\"Gestures

A core technical challenge: developing a gesture set that enables clear differentiation between various types of user contact with the touch surface.<\/p><\/div>\n

\u201cNext, fingers can be very close together when making contact. To the sensor, they can appear as one finger rather than multiple fingers. But if you want to have reliable gesture recognition, you need a way to differentiate between one, two, or three fingers. We had to develop technology that enhances signal processing and reliably tracks contacts.<\/p>\n<\/div>\n

\u201cAnd here is one more example,\u201d Miller continues. \u201cEverybody holds the mouse in a slightly different way. Some people hold their fingers flat on the mouse, and people with very small hands will hold the device differently than people with very large hands. So the mouse does not make contact the same way for all users, and they are all going to be performing these gestures in a slightly different way. As a result, there\u2019s a lot more \u2018noise\u2019 to handle than from a touch-screen phone or a Tablet PC. We had to deal with a lot more data.\u201d<\/p>\n

To mitigate some of these problems, the team set a design goal that gestures should be both intuitive and distinct\u2014the kind that would be hard for a user to perform by accident. This helped simplify the job of the recognition software.<\/p>\n

They also developed a tool that recorded sensor data while human testers were using the mouse for actions such as pointing and clicking, multitouch gesturing, and grabbing and releasing the mouse.<\/p>\n

\u201cWe ended up with data examples of good gestures for mouse usage and unintentional movements,\u201d Benko says, \u201cand this helped us conceive strategies for distinguishing between intentional gestures and incidental movements. It\u2019s what allowed us to develop an engine that\u2019s able to recognize some movements and ignore others.\u201d<\/p>\n

Collaboration Delivers a Quality Product<\/h2>\n

The Microsoft Touch Mouse project is unusual compared with other hardware-development projects, because it is not simply about hardware. Rather, it is a product that combines multiple disciplines in a tightly integrated way, a task that would have been impossible without close collaboration between multiple Microsoft Research and Microsoft hardware-development teams in different locations.<\/p>\n

Decisions about the final product, for example, involved testing and evaluation of different prototypes and features by all parties.<\/p>\n

\n
\"Hrvoje

Touch Mouse collaborators Hrvoje Benko (left) and John Miller.<\/p><\/div>\n<\/div>\n

\u201cThere were a lot of concepts from the original research,\u201d Benko says, \u201cand some of those we decided to leave out. That doesn\u2019t mean they were bad ideas, just that we were being very careful about our choices. It\u2019s how making a product works: You assess the pros and cons of every choice. Both the research and hardware teams were focused on nailing down the core experience, to make sure that everything we included was critical and didn\u2019t distract from the user\u2019s task. Our goal was to deliver a delightful, fluid desktop experience.\u201d<\/p>\n

Even though the multitouch-mouse project officially belonged to the hardware team, Microsoft Research remained integral to the development.<\/p>\n

\u201cThe original Mouse 2.0 paper was just the starting point,\u201d Miller says. \u201cThe research efforts didn\u2019t stop there. They continued in tandem with product development. There was a lot of additional research from different parties before we could turn the multitouch-mouse concept into a device that consumers can buy off the shelf.\u201d<\/p>\n

For Benko and Miller, one of the most rewarding aspects of this project has been the close collaboration between the hardware team and Microsoft Research in both Cambridge and Redmond. It went beyond technology transfer and was absolutely critical to delivering a successful transition from research prototype to consumer product.<\/p>\n

The Microsoft Touch Mouse proves that quality research doesn\u2019t have to address technologies that are many years away from commercialization. Sometimes, it\u2019s about exploring new possibilities. There\u2019s always room for a better mousetrap\u2014make that, a better mouse.<\/p>\n","protected":false},"excerpt":{"rendered":"

How do you take a concept from research to product? In the case of the Microsoft Touch Mouse, it took a collection of prototypes, collaboration between transatlantic teams, and a lot of user testing. It also helps when the research that launched the project won the best-paper award during the Association for Computing Machinery\u2019s 22nd […]<\/p>\n","protected":false},"author":39507,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"categories":[194476],"tags":[202799,204893],"research-area":[13552],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-236987","post","type-post","status-publish","format-standard","hentry","category-devices-and-hardware","tag-microsoft-touch-mouse","tag-product-development","msr-research-area-hardware-devices","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[199561,199565],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[144633],"related-projects":[],"related-events":[],"related-researchers":[],"msr_type":"Post","byline":"","formattedDate":"July 26, 2011","formattedExcerpt":"How do you take a concept from research to product? In the case of the Microsoft Touch Mouse, it took a collection of prototypes, collaboration between transatlantic teams, and a lot of user testing. It also helps when the research that launched the project won…","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/236987"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/39507"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=236987"}],"version-history":[{"count":5,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/236987\/revisions"}],"predecessor-version":[{"id":236997,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/236987\/revisions\/236997"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=236987"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=236987"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=236987"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=236987"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=236987"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=236987"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=236987"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=236987"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=236987"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=236987"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=236987"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}