{"id":954351,"date":"2023-07-12T08:28:49","date_gmt":"2023-07-12T15:28:49","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=954351"},"modified":"2024-01-17T12:27:09","modified_gmt":"2024-01-17T20:27:09","slug":"thinking-beyond-audio-augmenting-headphones-for-everyday-digital-interactions","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/thinking-beyond-audio-augmenting-headphones-for-everyday-digital-interactions\/","title":{"rendered":"Thinking beyond audio: Augmenting headphones for everyday digital interactions"},"content":{"rendered":"\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1400\" height=\"788\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/01\/DIS23-BlogHeroFeature-1400x788-1.jpg\" alt=\"Microsoft at DIS 2023: Thinking beyond Audio: Augmenting headphones for everyday digital interactions\" class=\"wp-image-999501\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/01\/DIS23-BlogHeroFeature-1400x788-1.jpg 1400w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/01\/DIS23-BlogHeroFeature-1400x788-1-300x169.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/01\/DIS23-BlogHeroFeature-1400x788-1-1024x576.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/01\/DIS23-BlogHeroFeature-1400x788-1-768x432.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/01\/DIS23-BlogHeroFeature-1400x788-1-1066x600.jpg 1066w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/01\/DIS23-BlogHeroFeature-1400x788-1-655x368.jpg 655w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/01\/DIS23-BlogHeroFeature-1400x788-1-343x193.jpg 343w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/01\/DIS23-BlogHeroFeature-1400x788-1-240x135.jpg 240w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/01\/DIS23-BlogHeroFeature-1400x788-1-640x360.jpg 640w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/01\/DIS23-BlogHeroFeature-1400x788-1-960x540.jpg 960w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/01\/DIS23-BlogHeroFeature-1400x788-1-1280x720.jpg 1280w\" sizes=\"(max-width: 1400px) 100vw, 1400px\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-center\"><strong><em><em>This research was accepted by and received a Best Paper Award during<\/em> <\/em><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/dis.acm.org\/2023\/\" target=\"_blank\" rel=\"noreferrer noopener\"><em>ACM Designing Interactive Systems (DIS) 2023<\/em><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><em>, <em>which is dedicated to advancing the field of user-centered system design<\/em><\/em>.<\/strong><\/p>\n\n\n\n<p>Headphones are traditionally used to provide and manage audio experiences through physical controls and a range of sensors. Nonetheless, these controls and sensors have remained confined to audio input and output functionality, such as adjusting the volume or muting the microphone. Imagine if headphones could transcend their role as mere audio devices.&nbsp;<\/p>\n\n\n\n<p>Because headphones rank among the most popular wearables in the market, we have an exciting opportunity to expand their capabilities through integrating existing sensors with supplementary ones to enable a wide variety of experiences that go beyond traditional audio control. In our paper, \u201c<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/beyond-audio-towards-a-design-space-of-headphones-as-a-site-for-interaction-and-sensing\/\" target=\"_blank\" rel=\"noreferrer noopener\">Beyond Audio: Towards a Design Space of Headphones as a Site for Interaction and Sensing<\/a>,&#8221; we share a vision that explores this potential.<\/p>\n\n\n\n<p>By using sensors such as microphones, proximity sensors, motion sensors, inertial measurement units (IMUs), and LiDARs, headphone designers can explore new avenues of input and interaction. The fact that headphones are worn on a person\u2019s head allows for a wide range of applications, such as following head movements, body postures, and hand gestures. Furthermore, as wearable devices, headphones have the potential to provide wearers with context-rich information and enable more intuitive and immersive interactions with their devices and environment beyond traditional button-based controls.<\/p>\n\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n\t<div class=\"border-bottom border-top border-gray-300 mt-5 mb-5 msr-promo text-center text-md-left alignwide\" data-bi-aN=\"promo\" data-bi-id=\"1085508\">\n\t\t\n\n\t\t<p class=\"msr-promo__label text-gray-800 text-center text-uppercase\">\n\t\t<span class=\"px-4 bg-white display-inline-block font-weight-semibold small\">Spotlight: Blog post<\/span>\n\t<\/p>\n\t\n\t<div class=\"row pt-3 pb-4 align-items-center\">\n\t\t\t\t\t\t<div class=\"msr-promo__media col-12 col-md-5\">\n\t\t\t\t<a class=\"bg-gray-300\" href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/medfuzz-exploring-the-robustness-of-llms-on-medical-challenge-problems\/\" aria-label=\"MedFuzz: Exploring the robustness of LLMs on medical challenge problems\" data-bi-cN=\"MedFuzz: Exploring the robustness of LLMs on medical challenge problems\" target=\"_blank\">\n\t\t\t\t\t<img decoding=\"async\" class=\"w-100 display-block\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2024\/09\/MedFuzz-BlogHeroFeature-1400x788-1.jpg\" alt=\"MedFuzz blog hero (decorative)\" \/>\n\t\t\t\t<\/a>\n\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"msr-promo__content p-3 px-5 col-12 col-md\">\n\n\t\t\t\t\t\t\t\t\t<h2 class=\"h4\">MedFuzz: Exploring the robustness of LLMs on medical challenge problems<\/h2>\n\t\t\t\t\n\t\t\t\t\t\t\t\t<p class=\"large\">Medfuzz tests LLMs by breaking benchmark assumptions, exposing vulnerabilities to bolster real-world accuracy.<\/p>\n\t\t\t\t\n\t\t\t\t\t\t\t\t<div class=\"wp-block-buttons justify-content-center justify-content-md-start\">\n\t\t\t\t\t<div class=\"wp-block-button\">\n\t\t\t\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/medfuzz-exploring-the-robustness-of-llms-on-medical-challenge-problems\/\" class=\"btn btn-brand glyph-append glyph-append-chevron-right\" aria-label=\"Read more\" data-bi-cN=\"MedFuzz: Exploring the robustness of LLMs on medical challenge problems\" target=\"_blank\">\n\t\t\t\t\t\t\tRead more\t\t\t\t\t\t<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div><!--\/.msr-promo__content-->\n\t<\/div><!--\/.msr-promo__inner-wrap-->\n\t<\/div><!--\/.msr-promo-->\n\t\n\n\n<h2 class=\"wp-block-heading\" id=\"potential-scenarios-for-sensor-enhanced-headphones\">Potential scenarios for sensor-enhanced headphones&nbsp;<\/h2>\n\n\n\n<p>To explore this concept further, we propose augmenting headphones with additional sensors and input widgets. These include:&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\" start=\"1\">\n<li>IMUs to sense head orientation<\/li>\n\n\n\n<li>Swappable sets of input controls&nbsp;&nbsp;<\/li>\n\n\n\n<li>A range-sensing LiDAR that enables the sensing of hand gestures<\/li>\n<\/ul>\n\n\n\n<p>By incorporating these capabilities, we envision a wide range of applications where headphone input acts as a bridge between the person wearing it and their environment and enable more efficient and context-aware interactions among multiple devices and tasks. For example, a headphone could assist people with applications like video games or help manage interruptions during a video call.&nbsp;&nbsp;<\/p>\n\n\n\n<p>Let\u2019s explore some scenarios to illustrate the potential of our headphone design concept. Consider a person engaged in a video call with teammates when they are suddenly interrupted by a colleague who approaches in person. In this situation, our headphones would be equipped to detect contextual cues, such as when the wearer rotates their head away from a video call, signaling a shift in attention. In response, the headphones could automatically blur the video feed and mute the microphone to protect the wearer\u2019s privacy, as shown in Figure 1. This feature could also communicate to other participants that the wearer is temporarily engaged in another conversation or activity. When the wearer returns their attention to the call, the system removes the blur and reactivates the microphone.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1200\" height=\"338\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig1-combined.gif\" alt=\"Figure 1: Two videos side-by-side showing the headphones in a context-aware privacy-control scenario. On the left, there is an over-the-shoulder view of a wearer participating in a video call on a laptop. As he looks away from the call, the laptop screen changes color, and the application is muted, depicted by a mute icon overlayed on the video. As the wearer looks back at the screen, it becomes unblurred and a unmute icon is overlaid on the image, indicating the mute has been turned off. On the right, we see the laptop screen previously described.\" class=\"wp-image-954879\"\/><figcaption class=\"wp-element-caption\">Figure 1. These videos illustrate a context-aware privacy control system implemented during a video conference. In this scenario,&nbsp;the wearer temporarily disengages from the video conference to engage in an in-person conversation. After a predefined period, the system detects the wearer\u2019s continued attention directed away from any known device, taking into account the environment context. As a result, privacy measures are triggered, including video blurring, microphone muting, and notifying other participants on the call. Once the wearer re-engages with the screen, their video and microphone settings return to normal, ensuring a seamless experience.<\/figcaption><\/figure>\n\n\n\n<p>In another privacy-focused scenario, imagine a person simultaneously conversing with multiple teammates in separate video call channels. Our headphone design allows the wearer to control to whom their speech is directed by simply looking at their intended audience, as shown in Figure 2. This directed speech interaction can extend beyond video calls and be applied to other contexts, such as sending targeted voice commands to teammates in a multiplayer video game.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1200\" height=\"338\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig2-combined.gif\" alt=\"DIS 2023 - Figure 2: Two videos side-by-side showing the wearer controlling where his input is being sent among a multitude of devices. On the left, a video shows an over-the-shoulder view of a wearer interacting with a monitor and aptop while wearing headphones. There are two separate video calls on each screen. As the wearer turns from one screen to another, a large microphone icon appears on the screen at which the wearer is looking, and a muted microphone icon is shown on the other screen.\n\nThe video on the right shows an over-the-shoulder view of a wearer interacting with a laptop while wearing headphones. The laptop screen shows a video game and four circular icons on each corner depicting the other players. The user looks at the bottom left of the screen, which enlarges the icon of the teammate in that corner, and the wearer starts to speak. The wearer then looks at the top-right of the screen, and the teammate in that corner is highlighted while the wearer speaks.\" class=\"wp-image-954891\"\/><figcaption class=\"wp-element-caption\">Figure 2. Headphones track the wearer\u2019s head pose, seamlessly facilitating the distribution of video and\/or audio across multiple private chats. They effectively communicate the wearer\u2019s availability to other participants, whether in a video conferencing scenario (left) or a gaming scenario (right).<\/figcaption><\/figure>\n\n\n\n<p>In our <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/beyond-audio-towards-a-design-space-of-headphones-as-a-site-for-interaction-and-sensing\/\" target=\"_blank\" rel=\"noreferrer noopener\">paper<\/a>, we also demonstrate how socially recognizable gestures can introduce new forms of audio-visual control instead of relying solely on on-screen controls. For example, wearers could interact with media through gestural actions, such as cupping their ear towards the audio source to increase the volume while simultaneously reducing ambient noise, as shown in Figure 3. These gestures, ingrained in social and cultural contexts, can serve as both control mechanisms and nonverbal communication signals.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"705\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig3_Gestural_Control-1024x705.jpg\" alt=\"DIS 2023 - Fig 3 - image showing gestural controls for volume\" class=\"wp-image-954651\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig3_Gestural_Control-1024x705.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig3_Gestural_Control-300x207.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig3_Gestural_Control-768x529.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig3_Gestural_Control-800x550.jpg 800w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig3_Gestural_Control-240x165.jpg 240w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig3_Gestural_Control.jpg 1400w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\">Figure 3. Top: Raising the earcup, a commonly used gesture to address in-person interruptions, mutes both the sound and the microphone to ensure privacy. Bottom: Cupping the earcup, a gesture indicating difficulty hearing, increases the system volume.<\/figcaption><\/figure>\n\n\n\n<p>Additionally, we can estimate the wearer\u2019s head gaze through the use of an IMU. When combined with the physical location of computing devices in the wearer\u2019s vicinity, it opens up possibilities for seamless interactions across multiple devices. For instance, during a video call, the wearer can share the screen of the device they are actively focusing on. In this scenario, the wearer shifts their attention from an external monitor to a tablet device. Even though this tablet is not directly connected to the main laptop, our system smoothly transitions the screen sharing for the wearer\u2019s audience in the video call, as shown in Figure 4.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1179\" height=\"338\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig4-combined.gif\" alt=\"DIS 2023 - Figure 4: Two videos side-by-side showing a headphone wearer among a multitude of devices controlling which screen is shared in a video call. The video on the left shows an over-the-shoulder view of a person interacting with three screens\u2014a monitor, a laptop, and a tablet\u2014while wearing headphones. A video call is in progress on the laptop, and the wearer is giving a presentation, which appears as a slide on the attached monitor. As the wearer turns from the laptop screen to the monitor, the presentation slide appears on the shared laptop screen. The video on the right shows an over-the-shoulder view of the person interacting with three screens\u2014a monitor, a laptop, and a tablet\u2014while wearing headphones. We see the wearer looking at the monitor with a presentation slide, which is mirrored on the laptop screen. He then turns from the monitor to the tablet, which has a drawing app open. As he does this, the drawing app appears on the shared laptop screen. The wearer uses a pen to draw on the tablet, and this is mirrored on the laptop. Finally, the wearer looks up from the tablet to the laptop, and the laptop screen switches to the video call view with the participants\u2019 videos.\" class=\"wp-image-954894\"\/><figcaption class=\"wp-element-caption\">Figure 4. A wearer delivers a presentation using a video conferencing tool. As the wearer looks at different devices, the streamed video dynamically updates to display the relevant source to participants.<\/figcaption><\/figure>\n\n\n\n<div class=\"annotations \" data-bi-aN=\"margin-callout\">\n\t<ul class=\"annotations__list card depth-16 bg-body p-4 annotations__list--right\">\n\t\t<li class=\"annotations__list-item\">\n\t\t\t\t\t\t<span class=\"annotations__type d-block text-uppercase font-weight-semibold text-neutral-300 small\">Publication<\/span>\n\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/beyond-audio-towards-a-design-space-of-headphones-as-a-site-for-interaction-and-sensing\/\" target=\"_self\" class=\"annotations__link font-weight-semibold text-decoration-none\" data-bi-type=\"annotated-link\" aria-label=\"Beyond Audio: Towards a Design Space of Headphones as a Site for Interaction and Sensing\" data-bi-aN=\"margin-callout\" data-bi-cN=\"Beyond Audio: Towards a Design Space of Headphones as a Site for Interaction and Sensing\">\n\t\t\t\tBeyond Audio: Towards a Design Space of Headphones as a Site for Interaction and Sensing&nbsp;<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t<\/a>\n\t\t\t\t\t<\/li>\n\t<\/ul>\n<\/div>\n\n\n\n<p>Finally, in our paper we also show the use of embodied interactions, where the wearer\u2019s body movements serve to animate a digital representation of themselves, such as an avatar in a video call, as shown in Figure 5. This feature can also be implemented as a gameplay mechanism. Take a racing game for instance, where the wearer\u2019s body movements could control the vehicle\u2019s steering, shown on the left in Figure 6. To extend this capability, these movements could enable a wearer to peek around obstacles in any first-person game, enhancing the immersion and gameplay experience, shown on the right in Figure 6.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1098\" height=\"277\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig5-combined.gif\" alt=\"DIS 2023 - Figure 5: Two videos showing a headphone wearer controlling an avatar in a video call through head movements. The video on the left shows an over-the-shoulder view of a headphones wearer interacting with another participant on the call. The video on the right shows a wearer using a touch control to depict an emotion in his avatar.\" class=\"wp-image-954897\"\/><figcaption class=\"wp-element-caption\">Figure 5. Left: Headphones use an IMU to monitor and capture natural body movements, which are then translated into corresponding avatar movements. Right: Touch controls integrated into headphones enable wearers to evoke a range of emotions on the avatar, enhancing the user experience.<\/figcaption><\/figure>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1197\" height=\"315\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig6-combined.gif\" alt=\"DIS 2023 - Figure 6: Two videos showing a wearer playing a video game while leaning left and right. These movements control his character\u2019s movements, enabling him to duck and peek around walls.\" class=\"wp-image-954888\"\/><figcaption class=\"wp-element-caption\">Figure 6. Leaning while wearing the headphone (with an integrated IMU) has a direct impact on game play action. On the left, it results in swerving the car to the side, while on the right, in enables the player to duck behind a wall.<\/figcaption><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"design-space-for-headphone-interactions\">Design space for headphone interactions&nbsp;<\/h2>\n\n\n\n<p>We define a design space for interactive headphones through an exploration of two distinct concepts, which we discuss in depth in our <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/beyond-audio-towards-a-design-space-of-headphones-as-a-site-for-interaction-and-sensing\/\" target=\"_blank\" rel=\"noreferrer noopener\">paper<\/a>.<\/p>\n\n\n\n<p>First, we look at the <strong>type of input gesture<\/strong> for the interaction, which we further classify into three categories. The gestural input from the wearer might fall under one or more of these categories, which we outline in more detail below and illustrate in Figure 7.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Touch-based gestures that involve tangible inputs on the headphones, such as buttons or knobs, requiring physical contact by the wearer<\/li>\n\n\n\n<li>Mid-air gestures, which the wearer makes with their hands in close proximity to the headphones, detected through LiDAR technology<\/li>\n\n\n\n<li>Head orientation, indicating the direction of the wearer\u2019s attention<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"279\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig7_GestureTypes_1400px-1024x279.png\" alt=\"DIS 2023 - Figure 7: List of three stylized images showing the three main kinds of gestures we look at: touch, head orientation, and mid-air gestures.\" class=\"wp-image-954672\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig7_GestureTypes_1400px-1024x279.png 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig7_GestureTypes_1400px-300x82.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig7_GestureTypes_1400px-768x209.png 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig7_GestureTypes_1400px-240x65.png 240w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig7_GestureTypes_1400px.png 1400w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\">Figure 7. Sensor-enhanced headphones can use touch-based gestures (left), head orientation (middle), or mid-air gestures (right) as types of input.<\/figcaption><\/figure>\n\n\n\n<p>The second way that we define the design space is through the <strong>context within which the wearer executes the action.<\/strong> Here, design considerations for sensor-enhanced headphones go beyond user intentionality and observed motion. Context-awareness enables these headphones to understand the wearer\u2019s activities, the applications they are engaged with, and the devices in their vicinity, as illustrated in Figure 8. This understanding enables the headphones to provide personalized experiences and seamlessly integrate with the wearer\u2019s environment. The four categories that define this context-awareness are comprised of the following:&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Context-free actions, which produce similar results regardless of the active application, the wearer\u2019s activity, or the social or physical environment.&nbsp;&nbsp;<\/li>\n\n\n\n<li>Context that is defined by the application with which the wearer is interacting. For example, are they listening to music, on a video call, or watching a movie?&nbsp;&nbsp;<\/li>\n\n\n\n<li>Context that is defined by the wearer\u2019s body. For example, is the wearer\u2019s gesture close to a body part that has an associated meaning? Eyes might relate to visual functions, ears to audio input, and the mouth to audio output.&nbsp;<\/li>\n\n\n\n<li>Context that is defined by the wearer\u2019s environment. For example, are there other devices or people around the wearer with whom they might want to interact?<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"232\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig8_Headphones_Context_1400px-1024x232.png\" alt=\"DIS 2023 - Figure 8: Diagram showing the different levels of context we look at: context free, application, user's body, and the environment.\" class=\"wp-image-954675\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig8_Headphones_Context_1400px-1024x232.png 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig8_Headphones_Context_1400px-300x68.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig8_Headphones_Context_1400px-768x174.png 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig8_Headphones_Context_1400px-240x54.png 240w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/Fig8_Headphones_Context_1400px.png 1400w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\">Figure 8. The system uses diverse contextual information to enable personalized responses to user input.<\/figcaption><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"looking-ahead-expanding-the-possibilities-of-hci-with-everyday-wearables\">Looking ahead: Expanding the possibilities of HCI with everyday wearables &nbsp;<\/h2>\n\n\n\n<p>Sensor-enhanced headphones offer a promising avenue for designers to create immersive and context-aware user experiences. By incorporating sensors, these headphones can capture subtle user behaviors, facilitating seamless interactions and enhancing the wearer\u2019s overall experience.&nbsp;&nbsp;<\/p>\n\n\n\n<p>From safeguarding privacy to providing intuitive control mechanisms, the potential applications for sensor-enhanced headphones are vast and exciting. This exploration with headphones scratches the surface of what context-aware wearable technology can empower its wearers to achieve. Consider the multitude of wearables we use every day that could benefit from integrating similar sensing and interaction capabilities into these devices. For example, imagine a watch that can track your hand movements and detect gestures. By enabling communication between sensor-enhanced wearables, we can establish a cohesive ecosystem for human-computer interaction that spans across applications, devices, and social contexts.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Because headphones rank among the most popular wearables in the market, we have an exciting opportunity to expand their capabilities through integrating existing sensors with supplementary ones to enable a wide variety of experiences that go beyond traditional audio control.<\/p>\n","protected":false},"author":42735,"featured_media":954699,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"categories":[1],"tags":[],"research-area":[243062,13554],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[261667],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-954351","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-research-blog","msr-research-area-audio-acoustics","msr-research-area-human-computer-interaction","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[199561],"msr_impact_theme":["Empowerment"],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[],"related-projects":[483294],"related-events":[],"related-researchers":[{"type":"user_nicename","value":"Payod Panda","user_id":42171,"display_name":"Payod Panda","author_link":"<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/t-payodpanda\/\" aria-label=\"Visit the profile page for Payod Panda\">Payod Panda<\/a>","is_active":false,"last_first":"Panda, Payod","people_section":0,"alias":"t-payodpanda"}],"msr_type":"Post","featured_image_thumbnail":"<img width=\"960\" height=\"540\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/DIS23-BlogHeroFeature-1400x788-1-960x540.jpg\" class=\"img-object-cover\" alt=\"Microsoft at DIS 2023: Thinking beyond audio: Augmenting headphones for everyday digital interactions\" decoding=\"async\" loading=\"lazy\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/DIS23-BlogHeroFeature-1400x788-1-960x540.jpg 960w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/DIS23-BlogHeroFeature-1400x788-1-300x169.jpg 300w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/DIS23-BlogHeroFeature-1400x788-1-1024x576.jpg 1024w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/DIS23-BlogHeroFeature-1400x788-1-768x432.jpg 768w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/DIS23-BlogHeroFeature-1400x788-1-1066x600.jpg 1066w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/DIS23-BlogHeroFeature-1400x788-1-655x368.jpg 655w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/DIS23-BlogHeroFeature-1400x788-1-343x193.jpg 343w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/DIS23-BlogHeroFeature-1400x788-1-240x135.jpg 240w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/DIS23-BlogHeroFeature-1400x788-1-640x360.jpg 640w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/DIS23-BlogHeroFeature-1400x788-1-1280x720.jpg 1280w, https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2023\/07\/DIS23-BlogHeroFeature-1400x788-1.jpg 1400w\" sizes=\"(max-width: 960px) 100vw, 960px\" \/>","byline":"<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/people\/t-payodpanda\/\" title=\"Go to researcher profile for Payod Panda\" aria-label=\"Go to researcher profile for Payod Panda\" data-bi-type=\"byline author\" data-bi-cN=\"Payod Panda\">Payod Panda<\/a>","formattedDate":"July 12, 2023","formattedExcerpt":"Because headphones rank among the most popular wearables in the market, we have an exciting opportunity to expand their capabilities through integrating existing sensors with supplementary ones to enable a wide variety of experiences that go beyond traditional audio control.","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/954351"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/42735"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=954351"}],"version-history":[{"count":32,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/954351\/revisions"}],"predecessor-version":[{"id":999507,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/954351\/revisions\/999507"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/954699"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=954351"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=954351"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=954351"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=954351"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=954351"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=954351"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=954351"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=954351"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=954351"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=954351"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=954351"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}