{"id":1033068,"date":"2024-05-15T09:00:00","date_gmt":"2024-05-15T16:00:00","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=1033068"},"modified":"2024-05-14T06:29:33","modified_gmt":"2024-05-14T13:29:33","slug":"microsoft-at-chi-2024-innovations-in-human-centered-design","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/microsoft-at-chi-2024-innovations-in-human-centered-design\/","title":{"rendered":"Microsoft at CHI 2024: Innovations in human-centered design"},"content":{"rendered":"\n
\"Microsoft<\/figure>\n\n\n\n

The ways people engage with technology, through its design and functionality, determine its utility and acceptance in everyday use, setting the stage for widespread adoption. When computing tools and services respect the diversity of people\u2019s experiences and abilities, technology is not only functional but also universally accessible. Human-computer interaction (HCI) plays a crucial role in this process, examining how technology integrates into our daily lives and exploring ways digital tools can be shaped to meet individual needs and enhance our interactions with the world.<\/p>\n\n\n\n

The ACM CHI Conference on Human Factors in Computing Systems<\/a> is a premier forum that brings together researchers and experts in the field, and Microsoft is honored to support CHI 2024<\/a> as a returning sponsor. We’re pleased to announce that 33 papers by Microsoft researchers and their collaborators have been accepted this year, with four winning the Best Paper Award and seven receiving honorable mentions.<\/p>\n\n\n\n

This research aims to redefine how people work, collaborate, and play using technology, with a focus on design innovation to create more personalized, engaging, and effective interactions. Several projects emphasize customizing the user experience to better meet individual needs, such as exploring the potential of large language models (LLMs) to help reduce procrastination. Others investigate ways to boost realism in virtual and mixed reality environments, using touch to create a more immersive experience. There are also studies that address the challenges of understanding how people interact with technology. These include applying psychology and cognitive science to examine the use of generative AI and social media, with the goal of using the insights to guide future research and design directions. This post highlights these projects.<\/p>\n\n\n\n\t

\n\t\t\n\n\t\t

\n\t\tSpotlight: On-demand video<\/span>\n\t<\/p>\n\t\n\t

\n\t\t\t\t\t\t
\n\t\t\t\t\n\t\t\t\t\t\"a\n\t\t\t\t<\/a>\n\t\t\t<\/div>\n\t\t\t\n\t\t\t
\n\n\t\t\t\t\t\t\t\t\t

AI Explainer: Foundation models \u200band the next era of AI<\/h2>\n\t\t\t\t\n\t\t\t\t\t\t\t\t

Explore how the transformer architecture, larger models and more data, and in-context learning have helped advance AI from perception to creation.<\/p>\n\t\t\t\t\n\t\t\t\t\t\t\t\t

\n\t\t\t\t\t
\n\t\t\t\t\t\t\n\t\t\t\t\t\t\tWatch video\t\t\t\t\t\t<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t<\/div>\n\t<\/div>\n\t\n\n\n

Best Paper Award recipients<\/h2>\n\n\n\n

DynaVis: Dynamically Synthesized UI Widgets for Visualization Editing<\/strong><\/a> 
Priyan Vaithilingam, Elena L. Glassman, <\/em>Jeevana Priya Inala<\/em><\/a>, <\/em>Chenglong Wang<\/em><\/a> 
GUIs used for editing visualizations can overwhelm users or limit their interactions. To address this, the authors introduce DynaVis, which combines natural language interfaces with dynamically synthesized UI widgets, enabling people to initiate and refine edits using natural language.  <\/p>\n\n\n\n

Generative Echo Chamber? Effects of LLM-Powered Search Systems on Diverse Information Seeking<\/strong><\/a> <\/em> 
Nikhil Sharma, <\/em>Q. Vera Liao<\/em><\/a>, Ziang Xiao<\/em> <\/strong> 
Conversational search systems powered by LLMs potentially improve on traditional search methods, yet their influence on increasing selective exposure and fostering echo chambers remains underexplored. This research suggests that LLM-driven conversational search may enhance biased information querying, particularly when the LLM’s outputs reinforce user views, emphasizing significant implications for the development and regulation of these technologies.  <\/p>\n\n\n\n

Piet: Facilitating Color Authoring for Motion Graphics Video<\/strong><\/a> <\/strong> 
Xinyu Shi, Yinghou Wang, <\/em>Yun Wang<\/em><\/a>, Jian Zhao<\/em> 
Motion graphic (MG) videos use animated visuals and color to effectively communicate complex ideas, yet existing color authoring tools are lacking. This work introduces Piet, a tool prototype that offers an interactive palette and support for quick theme changes and controlled focus, significantly streamlining the color design process.<\/p>\n\n\n\n

The Metacognitive Demands and Opportunities of Generative AI<\/strong><\/a> 
Lev Tankelevitch<\/a>, Viktor Kewenig, Auste Simkute, Ava Elizabeth Scott, <\/em>Advait Sarkar<\/em><\/a>, <\/em>Abigail Sellen<\/em><\/a>, <\/em>Sean Rintel<\/em><\/a> 
Generative AI systems offer unprecedented opportunities for transforming professional and personal work, yet they present challenges around prompting, evaluating and relying on outputs, and optimizing workflows. This paper shows that metacognition\u2014the psychological ability to monitor and control one\u2019s thoughts and behavior\u2014offers a valuable lens through which to understand and design for these usability challenges.  <\/p>\n\n\n\n

\n\n\n\n

Honorable Mentions<\/h2>\n\n\n\n

B<\/a><\/strong>ig or Small, It\u2019s All in Your Head: Visuo-Haptic Illusion of Size-Change Using Finger-Repositioning<\/strong><\/a>
Myung Jin Kim, Eyal Ofek, <\/em>Michel Pahud<\/em><\/a>, Mike J. Sinclair, Andrea Bianchi<\/em> 
This research introduces a fixed-sized VR controller that uses finger repositioning to create a visuo-haptic illusion of dynamic size changes in handheld virtual objects, allowing users to perceive virtual objects as significantly smaller or larger than the actual device. <\/p>\n\n\n\n

LLMR: Real-time Prompting of Interactive Worlds Using Large Language Models<\/strong><\/a> 
Fernanda De La Torre, Cathy Mengying Fang, Han Huang, Andrzej Banburski-Fahey, <\/em>Judith Amores<\/em><\/a>, <\/em>Jaron Lanier<\/em><\/a> 
Large Language Model for Mixed Reality (LLMR) is a framework for the real-time creation and modification of interactive mixed reality experiences using LLMs. It uses novel strategies to tackle difficult cases where ideal training data is scarce or where the design goal requires the synthesis of internal dynamics, intuitive analysis, or advanced interactivity. <\/p>\n\n\n\n

Observer Effect in Social Media Use<\/strong><\/a>\u00a0
Koustuv Saha, Pranshu Gupta, Gloria Mark, <\/em>Emre Kiciman<\/em><\/a>, Munmun De Choudhury<\/em>\u00a0
This work investigates the observer effect in behavioral assessments on social media use. The observer effect is a phenomenon in which individuals alter their behavior due to awareness of being monitored. Conducted over an average of 82 months (about 7 years) retrospectively and five months prospectively using Facebook data, the study found that deviations in expected behavior and language post-enrollment in the study reflected individual psychological traits. The authors recommend ways to mitigate the observer effect in these scenarios.<\/p>\n\n\n\n

Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programming<\/a><\/strong> 
<\/a>Hussein Mozannar, <\/em>Gagan Bansal<\/em><\/a>, <\/em>Adam Fourney<\/em><\/a>, <\/em>Eric Horvitz<\/em><\/a> 
By investigating how developers use GitHub Copilot, the authors created CUPS, a taxonomy of programmer activities during system interaction. This approach not only elucidates interaction patterns and inefficiencies but can also drive more effective metrics and UI design for code-recommendation systems with the goal of improving programmer productivity. <\/p>\n\n\n\n

SharedNeRF: Leveraging Photorealistic and View-dependent Rendering for Real-time and Remote Collaboration<\/strong><\/a> 
Mose Sakashita, Bala Kumaravel, Nicolai Marquardt<\/a>, Andrew D. Wilson<\/a><\/em> 
SharedNeRF, a system for synchronous remote collaboration, utilizes neural radiance field (NeRF) technology to provide photorealistic, viewpoint-specific renderings that are seamlessly integrated with point clouds to capture dynamic movements and changes in a shared space. A preliminary study demonstrated its effectiveness, as participants used this high-fidelity, multi-perspective visualization to successfully complete a flower arrangement task. <\/p>\n\n\n\n

Understanding the Role of Large Language Models in Personalizing and Scaffolding Strategies to Combat Academic Procrastination<\/strong><\/a> 
Ananya Bhattacharjee, Yuchen Zeng, Sarah Yi Xu, Dana Kulzhabayeva, Minyi Ma, Rachel Kornfield, Syed Ishtiaque Ahmed, Alex Mariakakis, <\/em>Mary P. Czerwinski<\/em><\/a>, Anastasia Kuzminykh, Michael Liut, Joseph Jay Williams<\/em> 
In this study, the authors explore the potential of LLMs for customizing academic procrastination interventions, employing a technology probe to generate personalized advice. Their findings emphasize the need for LLMs to offer structured, deadline-oriented advice and adaptive questioning techniques, providing key design insights for LLM-based tools while highlighting cautions against their use for therapeutic guidance.<\/p>\n\n\n\n

Where Are We So Far? Understanding Data Storytelling Tools from the Perspective of Human-AI Collaboration<\/strong><\/a> <\/strong>
Haotian Li, <\/em>Yun Wang<\/em><\/a>, Huamin Qu<\/em>
This paper evaluates data storytelling tools using a dual framework to analyze the stages of the storytelling workflow\u2014analysis, planning, implementation, communication\u2014and the roles of humans and AI in each stage, such as creators, assistants, optimizers, and reviewers. The study identifies common collaboration patterns in existing tools, summarizes lessons from these patterns, and highlights future research opportunities for human-AI collaboration in data storytelling.<\/p>\n\n\n\n

\n\n\n\n

Learn more about our work and contributions to CHI 2024, including our full list of publications<\/a>, on our conference webpage<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

From immersive virtual experiences to interactive design tools, Microsoft Research is at the frontier of exploring how people engage with technology. Discover our latest breakthroughs in human-computer interaction research at CHI 2024.<\/p>\n","protected":false},"author":42735,"featured_media":1033422,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"categories":[1],"tags":[],"research-area":[13554],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[243984],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-1033068","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-research-blog","msr-research-area-human-computer-interaction","msr-locale-en_us","msr-post-option-blog-homepage-featured"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[437514,992148,199560,199561,199565],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[],"related-projects":[1053711],"related-events":[1019022],"related-researchers":[{"type":"user_nicename","value":"Jeevana Priya Inala","user_id":41377,"display_name":"Jeevana Priya Inala","author_link":"Jeevana Priya Inala<\/a>","is_active":false,"last_first":"Inala, Jeevana Priya","people_section":0,"alias":"jinala"},{"type":"user_nicename","value":"Chenglong Wang","user_id":41251,"display_name":"Chenglong Wang","author_link":"Chenglong Wang<\/a>","is_active":false,"last_first":"Wang, Chenglong","people_section":0,"alias":"chenwang"},{"type":"user_nicename","value":"Lev Tankelevitch","user_id":43209,"display_name":"Lev Tankelevitch","author_link":"Lev Tankelevitch<\/a>","is_active":false,"last_first":"Tankelevitch, Lev","people_section":0,"alias":"levt"},{"type":"user_nicename","value":"Advait Sarkar","user_id":37146,"display_name":"Advait Sarkar","author_link":"Advait Sarkar<\/a>","is_active":false,"last_first":"Sarkar, Advait","people_section":0,"alias":"advait"},{"type":"user_nicename","value":"Abigail Sellen","user_id":31112,"display_name":"Abigail Sellen","author_link":"Abigail Sellen<\/a>","is_active":false,"last_first":"Sellen, Abigail","people_section":0,"alias":"asellen"},{"type":"user_nicename","value":"Sean Rintel","user_id":33579,"display_name":"Sean Rintel","author_link":"Sean Rintel<\/a>","is_active":false,"last_first":"Rintel, Sean","people_section":0,"alias":"serintel"},{"type":"user_nicename","value":"Q. Vera Liao","user_id":40912,"display_name":"Q. Vera Liao","author_link":"Q. Vera Liao<\/a>","is_active":false,"last_first":"Liao, Q. Vera","people_section":0,"alias":"veraliao"},{"type":"user_nicename","value":"Yun Wang","user_id":37827,"display_name":"Yun Wang","author_link":"Yun Wang<\/a>","is_active":false,"last_first":"Wang, Yun","people_section":0,"alias":"wangyun"},{"type":"user_nicename","value":"Michel Pahud","user_id":33007,"display_name":"Michel Pahud","author_link":"Michel Pahud<\/a>","is_active":false,"last_first":"Pahud, Michel","people_section":0,"alias":"mpahud"},{"type":"user_nicename","value":"Judith Amores","user_id":42003,"display_name":"Judith Amores","author_link":"Judith Amores<\/a>","is_active":false,"last_first":"Amores, Judith","people_section":0,"alias":"judithamores"},{"type":"user_nicename","value":"Jaron Lanier","user_id":32148,"display_name":"Jaron Lanier","author_link":"Jaron Lanier<\/a>","is_active":false,"last_first":"Lanier, Jaron","people_section":0,"alias":"jalani"},{"type":"user_nicename","value":"Emre Kiciman","user_id":31739,"display_name":"Emre Kiciman","author_link":"Emre Kiciman<\/a>","is_active":false,"last_first":"Kiciman, Emre","people_section":0,"alias":"emrek"},{"type":"user_nicename","value":"Gagan Bansal","user_id":41707,"display_name":"Gagan Bansal","author_link":"Gagan Bansal<\/a>","is_active":false,"last_first":"Bansal, Gagan","people_section":0,"alias":"gaganbansal"},{"type":"user_nicename","value":"Adam Fourney","user_id":30820,"display_name":"Adam Fourney","author_link":"Adam Fourney<\/a>","is_active":false,"last_first":"Fourney, Adam","people_section":0,"alias":"adamfo"},{"type":"user_nicename","value":"Eric Horvitz","user_id":32033,"display_name":"Eric Horvitz","author_link":"Eric Horvitz<\/a>","is_active":false,"last_first":"Horvitz, Eric","people_section":0,"alias":"horvitz"},{"type":"user_nicename","value":"Nicolai Marquardt","user_id":42630,"display_name":"Nicolai Marquardt","author_link":"Nicolai Marquardt<\/a>","is_active":false,"last_first":"Marquardt, Nicolai","people_section":0,"alias":"nicmarquardt"},{"type":"user_nicename","value":"Andy Wilson","user_id":31159,"display_name":"Andy Wilson","author_link":"Andy Wilson<\/a>","is_active":false,"last_first":"Wilson, Andy","people_section":0,"alias":"awilson"},{"type":"user_nicename","value":"Mary Czerwinski","user_id":32824,"display_name":"Mary Czerwinski","author_link":"Mary Czerwinski<\/a>","is_active":false,"last_first":"Czerwinski, Mary","people_section":0,"alias":"marycz"}],"msr_type":"Post","featured_image_thumbnail":"\"Microsoft","byline":"","formattedDate":"May 15, 2024","formattedExcerpt":"From immersive virtual experiences to interactive design tools, Microsoft Research is at the frontier of exploring how people engage with technology. Discover our latest breakthroughs in human-computer interaction research at CHI 2024.","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/1033068"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/42735"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=1033068"}],"version-history":[{"count":20,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/1033068\/revisions"}],"predecessor-version":[{"id":1033785,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/1033068\/revisions\/1033785"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/1033422"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=1033068"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=1033068"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=1033068"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=1033068"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=1033068"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=1033068"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=1033068"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=1033068"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=1033068"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=1033068"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=1033068"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}