{"id":602475,"date":"2020-02-22T16:46:32","date_gmt":"2019-08-08T00:16:08","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-research-item&p=602475"},"modified":"2020-02-22T16:46:33","modified_gmt":"2020-02-23T00:46:33","slug":"dueto-accessible-gaze-operated-musical-expression","status":"publish","type":"msr-research-item","link":"https:\/\/www.microsoft.com\/en-us\/research\/publication\/dueto-accessible-gaze-operated-musical-expression\/","title":{"rendered":"Dueto: Accessible, Gaze-Operated Musical Expression"},"content":{"rendered":"

Gaze-tracking technologies can enable computer access for users who are unable to use standard input devices. However, using gaze as input poses challenges for interactions that require visual planning, like playing a digital instrument. We explore how multimodality can support eye-controlled musical expression by designing different multi-modal gaze interactions around a digital instrument we call Dueto. We tackle three design goals: creating an instrument that is explorable, easy to learn, and allows feature controllability. We showcase three different multimodal interactions for music playing such as eye gaze only, gaze + switch, and gaze + partner mode. <\/p>\n","protected":false},"excerpt":{"rendered":"

Gaze-tracking technologies can enable computer access for users who are unable to use standard input devices. However, using gaze as input poses challenges for interactions that require visual planning, like playing a digital instrument. We explore how multimodality can support eye-controlled musical expression by designing different multi-modal gaze interactions around a digital instrument we call […]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"msr-content-type":[3],"msr-research-highlight":[],"research-area":[13554],"msr-publication-type":[193716],"msr-product-type":[],"msr-focus-area":[],"msr-platform":[],"msr-download-source":[],"msr-locale":[268875],"msr-post-option":[],"msr-field-of-study":[],"msr-conference":[],"msr-journal":[],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-602475","msr-research-item","type-msr-research-item","status-publish","hentry","msr-research-area-human-computer-interaction","msr-locale-en_us"],"msr_publishername":"","msr_edition":"","msr_affiliation":"","msr_published_date":"2019-10-28","msr_host":"","msr_duration":"","msr_version":"","msr_speaker":"","msr_other_contributors":"","msr_booktitle":"","msr_pages_string":"","msr_chapter":"","msr_isbn":"","msr_journal":"","msr_volume":"","msr_number":"","msr_editors":"","msr_series":"","msr_issue":"","msr_organization":"ACM","msr_how_published":"","msr_notes":"","msr_highlight_text":"","msr_release_tracker_id":"","msr_original_fields_of_study":"","msr_download_urls":"","msr_external_url":"","msr_secondary_video_url":"","msr_longbiography":"","msr_microsoftintellectualproperty":1,"msr_main_download":"","msr_publicationurl":"","msr_doi":"","msr_publication_uploader":[{"type":"file","viewUrl":"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2019\/08\/dueto.pdf","id":"602478","title":"dueto","label_id":"243109","label":0}],"msr_related_uploader":"","msr_attachments":[{"id":602478,"url":"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2019\/08\/dueto.pdf"}],"msr-author-ordering":[{"type":"text","value":"Stephanie Valencia","user_id":0,"rest_url":false},{"type":"guest","value":"dwayne-lamb","user_id":467226,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=dwayne-lamb"},{"type":"user_nicename","value":"Shane Williams","user_id":33593,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Shane Williams"},{"type":"user_nicename","value":"Harish S. Kulkarni","user_id":31975,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Harish S. Kulkarni"},{"type":"user_nicename","value":"Ann Paradiso","user_id":31045,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Ann Paradiso"},{"type":"user_nicename","value":"Meredith Ringel Morris","user_id":32884,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Meredith Ringel Morris"}],"msr_impact_theme":[],"msr_research_lab":[199565],"msr_event":[],"msr_group":[283244,371909,615681,144928],"msr_project":[638760,438972],"publication":[],"video":[],"download":[],"msr_publication_type":"inproceedings","related_content":{"projects":[{"ID":638760,"post_title":"Augmentative and Alternative Communication","post_name":"augmentative-and-alternative-communication","post_type":"msr-project","post_date":"2020-02-22 16:11:08","post_modified":"2020-09-28 14:51:57","post_status":"publish","permalink":"https:\/\/www.microsoft.com\/en-us\/research\/project\/augmentative-and-alternative-communication\/","post_excerpt":"ALS (amyotrophic lateral sclerosis) is a degenerative neuromuscular disease; people with late-stage ALS typically retain cognitive function, but lose the motor ability to speak, relying on gaze-controlled AAC (augmentative and alternative communication) devices for interpersonal interactions. State-of-the-art AAC technologies used by people with ALS do not facilitate natural communication; gaze-based AAC communication is extremely slow, and the resulting synthesized speech is flat and robotic. Our project on AAC aims to to enhance both the emotional…","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/638760"}]}},{"ID":438972,"post_title":"Microsoft Hands-Free Music","post_name":"microsoft-hands-free-music","post_type":"msr-project","post_date":"2017-11-10 10:00:58","post_modified":"2020-03-13 17:34:10","post_status":"publish","permalink":"https:\/\/www.microsoft.com\/en-us\/research\/project\/microsoft-hands-free-music\/","post_excerpt":"The hands-free music project is a participatory design collaboration with members of the ALS community. Our mission is to restore critical expressive &amp; creative channels for people severely affected by disabling conditions, such as ALS, spinal cord injury, and related conditions.","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/438972"}]}}]},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/602475"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-research-item"}],"version-history":[{"count":1,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/602475\/revisions"}],"predecessor-version":[{"id":602481,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/602475\/revisions\/602481"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=602475"}],"wp:term":[{"taxonomy":"msr-content-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-content-type?post=602475"},{"taxonomy":"msr-research-highlight","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-highlight?post=602475"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=602475"},{"taxonomy":"msr-publication-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-type?post=602475"},{"taxonomy":"msr-product-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-product-type?post=602475"},{"taxonomy":"msr-focus-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-focus-area?post=602475"},{"taxonomy":"msr-platform","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-platform?post=602475"},{"taxonomy":"msr-download-source","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-download-source?post=602475"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=602475"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=602475"},{"taxonomy":"msr-field-of-study","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-field-of-study?post=602475"},{"taxonomy":"msr-conference","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-conference?post=602475"},{"taxonomy":"msr-journal","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-journal?post=602475"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=602475"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=602475"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}