Due to recent events, millions of office workers have needed to rapidly adjust to working from home—learning new collaboration tools and best practices, re-thinking how to stay connected with colleagues outside the office, and adapting to new social norms around meetings. Working remotely presents both technical and social challenges, and researchers at Microsoft have been working across disciplines to understand and support both aspects of this challenge for decades.
Below is just a small sample of the work researchers at Microsoft and their colleagues have produced to improve the remote work experience. For those seeking to build better remote work products and services—or for anyone who wants to be more productive at home—we hope this research can provide some guidance, insight, and inspiration.
Although these are truly challenging times, we can benefit from a strong foundation of interdisciplinary research that can help us all stay productive and connected—with the hope of emerging from this crisis better-equipped to work together.
- Paying attention can be harder in remote meetings. Sean Rintel at Microsoft Research Cambridge recently published two papers that can inform the design of features to support remote participants’ attention. One paper models how we ‘see’ attention in meetings. It suggests that machine perception may help us gather, signal, and follow attention when remote. The second paper suggests that low engagement in meetings may not always be a problem. Not every meeting requires our full engagement, but until we develop technological support for more nuanced roles, it is good practice to be up front about your engagement level. Together, these papers suggest that AI supported attention personalization could make future remote meetings more inclusive and effective by helping us overcome constraints and assumptions.
- One benefit to everyone attending a meeting virtually is that it can be easier to review missed content if you show up to a meeting late or have to step out for a moment. For instance, Kori Inkpen, Sasa Junuzovic, and John Tang from Microsoft Research Redmond have explored using “accelerated instant replay” (AIR) to help people catch up quickly and then jump (back) into the real-time meeting.
- In a world without business travel, negotiating time zone constraints becomes even more important. John Tang at Microsoft Research Redmond and Kori Inkpen at the Microsoft Research AI Lab have catalogued strategies for mitigating time zone-related obstacles to productivity and provided guidelines to help overcome these obstacles. Tang and Inkpen also worked with Asta Roseway, Mary Czerwinski, and Paul Johns from Microsoft Research Redmond to explore novel uses of asynchronous video to support collaboration across time zones and developed two prototype systems, Time Travel Proxy and Video Threads.
- Some features in Microsoft Teams were inspired by work led by Kenton O’Hara at Microsoft Research Cambridge on utilizing multiple devices to support better presentation and collaboration over video; researchers’ participation in an internal hackathon led to close collaboration on development of the product. Their previous research into ad hoc adaptability in video calling, wireless smartphone mirroring in video calls, and shared slideware control helped enhance users’ ability to join meetings on multiple devices and use a phone as a companion device. Learn the inside story of the hackathon and product collaboration in this article on companion experiences for Microsoft Teams.
- Video calling that is “just like being there” isn’t always ideal for people with autism, who have different sensory, cognitive and social needs. Researchers at Microsoft are seeking to better understand the experiences of people with autism when they use current video communication technologies. These researchers found that, when participating in video calls, people with autism often have to use coping strategies such as moderating sensory inputs, creating mental models of conversation partners, and adopting neurotypical behaviors. These coping strategies produce a great deal of stress, increase cognitive load, and leave people with autism feeling less empowered to participate in conversations. Using their results, the research team at Microsoft was able to outline improvements to video conference technologies that can make these technologies more comfortable and empowering for individuals with autism.
Looking forward, there will be an unprecedented opportunity for researchers to learn from the current situation to figure out not only how to manage future disruption, but also to incorporate new ways of working at home or in the office. Microsoft is committed to investing in research internally and externally to make this happen. For example, in more typical times, remote work often involves meetings with both remote and co-located colleagues. Better understanding and supporting productivity in these hybrid meetings is the subject of one of the academic projects Microsoft funds through the Microsoft Productivity Research program in collaboration with Dr. Mirjam Augstein and Dr. Thomas Neumayr at the University of Applied Sciences Upper Austria.
If you’d like to do a deeper dive into the literature on remote work, below is a selection of additional papers from Microsoft researchers on the subject.
Remote Meetings
Researchers at Microsoft have been working across disciplines to address the unique and complex challenges of meeting remotely, such as improving the quality, fidelity, and utility of meetings; addressing design issues; merging physical and virtual collaboration; and exploring the use of avatars:
“Hybrid meetings” (meetings with remote and co-located participants)
- “Embodied social proxy: mediating interpersonal connection in hub-and-satellite teams,” ACM CHI 2010
- Managing multiple technical and social asymmetries is key to a successful hybrid meeting: “Hybrid Meetings in the Modern Workplace: Stories of Success and Failure,” CollabTech 2019
- Enabling extreme remote autonomy using virtual reality, mixed reality, and mobile telepresence: “VROOM: Virtual Robot Overlay for Online Meetings,” late-breaking work at ACM CHI 2020
Spatialized audio and video for video calling
- “Exploring Spatialized Audio & Video for Distributed Conversations,” ACM CSCW 2010
- “Realistic audio in immersive video conferencing,” IEEE ICME 2011
- “Spatiality in videoconferencing: trade-offs between efficiency and social presence,” ACM CSCW 2006
- Window layouts for videoconference meetings: “Towards ideal window layouts for multi-party, gaze-aware desktop videoconferencing,” Graphics Interface 2011
Spotlight: AI-POWERED EXPERIENCE
Microsoft research copilot experience
Discover more about research at Microsoft through our AI-powered experience
“Accelerated instant replay” during a video call
- “What did I miss? In-Meeting Review using Multimodal Accelerated Instant Replay (AIR) Conferencing,” ACM CHI 2011
- “AIR conferencing: accelerated instant replay for in-meeting multimodal review,” ACM Multimedia 2010
Avatars in remote meetings
- “To see or not to see: A study comparing four-way avatar, video, and audio conferencing for work,” ACM Group 2012
- “Me and My Avatar: Exploring Users’ Comfort with Avatars for Workplace Communication,” ACM CSCW 2011
- An exploration of how users’ comfort with video impacts their use of videoconferencing: “Image, Appearance and Vanity in the Use of Media Spaces and Videoconference Systems,” ACM Group 2009
Remotely collaborating when completing tasks in the physical world
- Make any passive surface shareable during a videoconference: “IllumiShare: Sharing Any Surface,” ACM CHI 2012
- “Hands-Free Remote Collaboration Over Video: Exploring Viewer and Streamer Reactions,” ACM ISS 2018
- “Three Challenges for Designing Better Videoconferences,” 2014: A brief opinion piece highlighting past research into interactional challenges in videoconferencing systems.
Remote Teambuilding
Researchers are also addressing the challenges of remote team building by helping people maintain connections across time zones, welcome new remote team members, and e engage in shared experiences:
Working across time zones
- “Your Time Zone or Mine? A Study of Globally Time Zone-Shifted Collaboration,” ACM CSCW 2011
- “Time Travel Proxy: Using Lightweight Video Recordings to Create Asynchronous, Interactive Meetings” ACM SIGCHI 2012
- “Video Threads: Asynchronous Video Sharing for Temporally Distributed Teams,” ACM CSCW 2012
Remote employee onboarding
Microsoft research podcast
What’s Your Story: Lex Story
Model maker and fabricator Lex Story helps bring research to life through prototyping. He discusses his take on failure; the encouragement and advice that has supported his pursuit of art and science; and the sabbatical that might inspire his next career move.
Co-watching video
- “Sharing Experiences over Video: Watching Video Programs Together at a Distance,” INTERACT 2013
- “Crowdcasting: Remotely Participating in Live Events Through Multiple Live Streams,” ACM CSCW 2017
- “Rivulet: Exploring Participation in Live Events through Multi-Stream Experiences,” TVX 2016
- “Eventstreaming: Connecting streamers and viewers at the Silicon Valley Comic Con,” TVX-in-Industry 2016
Early research on remote work
Long before remote work became a way of life, researchers at Microsoft exploring the social and technical aspects of collaborating remotely. Below is a selection of work on the subject dating back nearly thirty years:
- This compendium of the latest research (circa 1997) in video-mediated communication (VMC) considers the true utility of video, versus audio calls or face-to-face communication: Video-Mediated Communication (Book, 1997)
- An early experimental study looking at how speech patterns in video calls, such as turn-taking, are different from in-person conversations: “Speech Patterns in Video-Mediated Conversations,” ACM CHI 1992
- An older but highly relevant study that highlights the problems people have in establishing a shared frame of reference when using video technology: “One is not Enough: Multiple Views in a Media Space,” ACM CHI 1993
- Case studies of different workplaces show the limitations of VMC tech for supporting organizational work: “From Video-Mediated Communication to Technologies for Collaboration,” ACM CSCW 1995
Work in related areas
Microsoft researchers have also made substantial contributions to researcher areas adjacent to remote work that are increasingly relevant in today’s context—in how remote work technologies can support family life and play. For instance:
Connecting family across distance
- How families stay close using Skype when geographically distributed: “Understanding family communication across time zones,” ACM CSCW 2010
- “Experiences2Go: Sharing Kids’ Activities Outside the Home with Remote Family Members,” ACM CSCW 2013
- Connecting Families is a book-length collection of research articles on how technology can be used to connect families, including several contributions from Microsoft researchers
Remote Play
Microsoft research podcast
What’s Your Story: Lex Story
Model maker and fabricator Lex Story helps bring research to life through prototyping. He discusses his take on failure; the encouragement and advice that has supported his pursuit of art and science; and the sabbatical that might inspire his next career move.
- “Video Playdate: Toward Free Play across Distance,” ACM CHI 2010
- Shopping with a remote friend: “Remote Shopping Advice: Enhancing In-Store Shopping with Social Technologies,” ACM CSCW 2014
- “Video Kids: Augmenting Close Friendships with Asynchronous Video Conversations in VideoPal,” ACM CHI 2012
Video Communication for First Responders
Thanks to Kori Inkpen, Sean Rintel, Abi Sellen, and John Tang, who also contributed to this post.
Artificial intelligence 
Human-computer interaction 
Systems and networking