The Appropriate Reliance research initiative focuses on advancing research and creating practical solutions for fostering appropriate reliance on AI. 

Through appropriate reliance (opens in new tab), we aim to help people who use AI systems find a balance between over-trusting AI outputs and accepting them even when they are incorrect (aka overreliance on AI (opens in new tab)) and under-trusting—not accepting, or not using AI systems even when they perform well (aka underreliance on AI). 

Both over- and under-reliance on AI can have negative consequences such as harms resulting from poor performance of the human+AI team, or product abandonment. 

The research initiative brings together researchers and practitioners from across the company, with the goal of delivering:

• Applied research on the nature of these phenomena and ways to mitigate them;
• Guidance for AI practitioners;
• Specific solutions such as UX/UI patterns. See guidelines, patterns, and examples for effective human-AI interaction, including appropriate reliance, in the HAX Toolkit Design Library.

Leads

Mihaela Vorvoreanu

Director, UX Research and Education, Aether

Researchers

There are dozens of researchers and practitioners across Microsoft working on this initiative, including:

The post Appropriate Reliance Research Initiative appeared first on Microsoft Research.

Large language model-powered tools like Copilot have the potential to increase labor productivity more than any technology in a generation. Motivated by the significance of this moment, researchers from across Microsoft have come together to measure and improve the productivity impacts of Copilot, as well as understand and mitigate externalities. 

This group of researchers is part of Microsoft’s broader New Future of Work Initiative (opens in new tab), which is our research effort to make work better as work rapidly changes, from hybrid work to AI.

Leads

Alexia Cambon

Senior Director, Future of Work Research

Brent Hecht

Director of Applied Science

Michael Schwarz

Chief Economist

Jaime Teevan

Chief Scientist

Researchers

There are dozens of researchers from across Microsoft working on this initiative. The authors of the above papers are a great place to start. We’ll be posting more work from more researchers as it becomes available!

The post AI and Productivity Research Initiative appeared first on Microsoft Research.

AI is not only software-based but also transforms the way we create software. Tools like GitHub Copilot affect the processes of building, testing, and delivering software. This workstream aims to examine how AI can and should influence software engineering – including its effects on developers, how it improves developer efficiency, how it can assist in keeping software safe, and what the potential risks are.

Leads

Jenna Butler

Principal Applied Research Scientist

Ben Hanrahan

Senior UX Researcher

Researchers

There are many researchers from across Microsoft working on this initiative:

The post AI and Software Engineering Research Initiative appeared first on Microsoft Research.

By Brent Hecht (opens in new tab), Jaime Teevan (opens in new tab), and Abi Sellen (opens in new tab)

In the past year, leaf blowers, barking dogs, and fussy children have all become staples of our online meetings. This is a case where technology has seemed to come to the rescue, and one of the more impressive AI features in Teams and online meeting software is background noise reduction (opens in new tab). Noise reduction capabilities, however, have become so good that they have introduced a new issue, which we call the “Leaf Blower Problem.” And the Leaf Blower Problem is about more than just leaf blowers (or dogs or children): it highlights the importance of designing for “common ground”, one of the more critical design challenges in developing technologies for remote and hybrid work.

Put simply, the Leaf Blower Problem occurs during online meetings when there is a very loud background noise for one participant and a background noise reduction system successfully eliminates that noise for the other participants. Problem solved, right? Nope! While the noise reduction minimizes distraction for other participants, it does not do so for the person collocated with loud noise. This creates a gap in the shared experience of the participants. The people who can’t hear the noise may wonder why the person who can is distracted or not paying attention. And the person who can hear the noise may not know that the noise has been eliminated for the others, causing that person to apologize into the silence for “All that noise that’s coming through!”

The Leaf Blower Problem is a new manifestation of an old problem in human-centered AI and UX design. Specifically, the problem is a version of a grounding error, a well-known issue among the class of those in human-centered AI and UX trained in psycholinguistics. Grounding errors get in the way of a key requirement for effective conversations: common ground.

Prof. Darren Gergle at Northwestern University and his colleagues provide a good summary of the role of common ground in communication.

“Successful communication relies on a foundation of mutual knowledge or common ground. Conversational grounding is the process of establishing that common ground. Speakers form utterances based on an expectation of what a listener is likely to know and then monitor that the utterances are understood, whereas listeners have a responsibility to demonstrate their level of understanding.” (Gergle et al. 2013 (opens in new tab))

Through the lens of Gergle’s description, we can understand a ground error as something that prevents all the people in a conversation from having the same understanding of what’s going on, and – critically – knowing that they all have that same understanding. In other words, a grounding error is something that negatively affects a group’s ability to achieve common ground. Without common ground, significant miscommunications, team-wide failures, and even serious conflict can occur, in addition to other problems like the actor-observer effect (opens in new tab).

Some examples are illustrative. If Maria, a programmer, uses an acronym like “RESTful” with Nell, who is not a programmer, and Nell doesn’t ask for clarification, a grounding error has occurred. Maria and Nell don’t have the same understanding of the key entity being discussed, and they don’t know they don’t have that same understanding. The conversation may be derailed until common ground around the entity Maria is talking about can be established. A related example comes from AI systems that translate between languages: if Maria says the word “red”, but an AI system translates that to “purple” in Chinese, it will be very difficult for all participants in the conversation to have the same understanding of what’s being discussed and know that each is having the same understanding. The conversation will be disrupted until all participants are able to reestablish a common ground around the color that Maria is discussing.

The exact same dynamic occurs in the Leaf Blower Problem. Maria might hear a horrendously loud noise, but Maria doesn’t know if the others in the meeting hear that noise as well. In other words, Maria doesn’t know if the noise is common ground. Similarly, if the noise is not audible to Maria’s colleagues, Maria’s colleagues do not know why Maria might be acting distracted; the reason Maria is acting distracted is another piece of information that is not the common ground of the meeting. The desire to re-establish common ground in any conversation is the reason why Maria feels the urge to explain that she can hear a leaf blower (or crying baby, or vacuum cleaner…) and to tell folks that is why she is feeling distracted.

Grounding errors that are caused by technology can usually be mitigated or addressed with better technology design that prioritizes common ground. How can meeting software support the participants in a call in achieving common ground in the presence of background noise reduction? One approach might be to use an icon similar to the mute indicator to let others on a call know that one person is experiencing significant background noise, and thus may be distracted. Another design might detect whether a loud background sound is present on the other ends of a call and provide a pop-up notifying the person who is experiencing the background sound, e.g., “Noise reduction successfully eliminated the background noise you are experiencing” (or the inverse). This could appear in a banner at the top of the screen similar to recording notifications.

More generally, as we increasingly build systems for hybrid settings, we have to ensure that we minimize the amount of time it takes for all participants to successfully achieve common ground. This means either designing common ground into meeting software (by making sure, for example, visual and auditory experiences are shared). When that is not possible – e.g., when there is a leaf blower present – every effort should be made to scaffold the grounding process so that each participant is aware of the other’s situation.

People will have many other grounding challenges as we enter a world of hybrid work. Some examples include:

• We already see people wanting to know if others can see a shared document after they’ve shared it. The common “Can you see my screen?” utterance emerges out of the desire to ensure common ground in a conversation.
• Common ground is undermined in the default “Brady Bunch” video call layout because the layout is different for each person. An obvious manifestation of this occurs when we “go around the table” to introduce people and there is no shared ordering to allow us all to know who should go next. Likewise, pointing or head turning towards someone doesn’t work (unlike in the opening credits of the Brady Bunch) due to the lack of this mutually shared frame of reference. This is one reason why Together Mode in Teams has benefits: everyone knows that everyone else see the same view.
• When a person sends an emoji on Teams or Slack in Windows and it shows up on Teams or Slack in iOS, the emoji will be rendered differently (i.e. using Apple emoji instead of Microsoft emoji). This can create grounding challenges similar to the machine translation example above (see Miller et al. 2016).
• In hybrid settings, collocated meeting participants will struggle to know if remote participants have heard side conversations, and the inverse is true with parallel chat. This also extends to shared artifacts: remote people will often not see or understand how in-room participants are viewing shared whiteboards, documents, or other objects, including gestures around them.
• Grounding is a key reason why self-view has become a mandatory feature for video calls. People want to see what others see when they look at their box. In hybrid work, this will be more difficult for collocated participants.

Grounding covers a huge range of issues from “in-the-moment” shared understanding, through to shared history and knowledge, to shared culture. All of this is fundamental to our ability to communicate and collaborate together. Those who are particularly interested in learning more about grounding in communication can read Prof. Herb Clark’s foundational book (opens in new tab) on the topic. If you’d like a quicker summary, Gergle and colleagues’ 2013 paper on grounding in video calls (opens in new tab) (read to the end of Section 2) and Miller et al.’s 2017 paper on grounding and emoji (opens in new tab) (read to the end of the introduction) are great places to start.Opens in a new tab

The post The “Leaf Blower Problem” and the importance of common ground appeared first on Microsoft Research.

Hybrid work “is the future” for Microsoft and many of our customers. Hybrid work also presents new challenges not present in remote work. Fortunately, we can draw on pre-pandemic research on hybrid work, extrapolation from our research on all-remote work, and evidence from some sites that are already experiencing the early phases of post-COVID hybrid work.

This hybrid work guidance is based on the existing research, with the understanding that there is still much to learn and that recommendations may change as new evidence comes to light. This document focuses on general guidance for hybrid workplaces. We also have a guide for hybrid meetings specifically.

A good summary of this general guidance comes from one of the best literature reviews of all the research on remote and hybrid work that had been written prior to 2015:

“Similar to the general notion regarding the appropriate dosage for medication, finding the right amount of time to telecommute may be the key to producing desired outcomes, because too little or too much might not have the intended effect.” – Allen et al. 2015

How much time should I spend in the office?

Assuming you have a good (i.e., ergonomic and focus-friendly) workspace at home, the best data available suggests that you should spend approximately half of your time in the office. Here are some reasons why:

• Some research has found that co-worker relationships are not seriously affected by low-intensity remote work, but are affected when remote work is more than ~2.5 days a week.
• Too much time spent working remote undermines both knowledge sharing and innovation within teams.
• One of the foremost academics on the topic of hybrid work, Dr. Nick Bloom (Stanford GSB), advocates for three days a week in the office.
• There is some research that suggests that the job satisfaction benefits of WFH max out at about 15 hours per week.

If you do not have a good setup at home, you should probably spend more time in the office (and in fact this was the top reason that employees in Microsoft’s China offices went back to work in summer 2020).

What types of work should we do in the office and what types of work should we do at home?

The research suggests that the following types of activities benefit from being in-office:

• Work that requires brainstorming or idea generation.
• Work with high “task interdependence” (e.g., you’d be chatting back and forth with colleagues on Teams quite often).
• New workstreams: spinning up new projects and setting the strategic direction for a team are easier in face-to-face meetings.
• Work that seeks to strengthen and build new relationships. Reduced interaction on WFH days can be balanced with increased interaction on days at office.
• Work with colleagues that are new to the group or organization.
• Work that requires tools or hardware that can only be used in the office (e.g., specialized tools that are too large, confidential, or expensive to be used at home).
• Serendipitous and spontaneous encounters that benefit long-term success of many different workstreams. ​​​​​​​

And the following types of activities are best suited for work at home:

• Work with low task interdependence.
• Work on projects that are mostly in a distributed, execution-focused phase.
• Work that requires significant uninterrupted focus (for people who do have an environment at home that is suitable for focus).
  ​​​​​​​

Should my team be in the office on the same days or different days?

Since many of the benefits of in-office work listed above come from co-location, you should seek to be in the office for at least some of the same days as the people you work with closely. In addition to more routine kinds of collaborative activities, in-office days are the time for special events such as customer meetings, brainstorming sessions, strategic planning, and team bonding.

How should the in-office time be distributed? Fixed days of the week? Fixed weeks of the month? On demand?

This is a critical question for which there is unfortunately little research. Most of the research that informs the above best-practices used a fixed- or flexible-days-of-the-week model, in which people came into the office for a certain amount of time each week.

Early data from our China offices found that the majority of employees were given the freedom to choose their own arrangements.  The result was that about half went into the office on a fixed schedule, and half on demand. Fixed days may make it easier to coordinate with close collaborators, but make serendipitous encounters with other colleagues less likely. We do not yet know whether one system – or a combination – works better.

People with disabilities may especially appreciate scheduling flexibility that allows them to work at the times when they can be most productive.

How should we think about changing our office workspaces?

Team Work and Socializing

Given the importance of spending time at the office for creative collaboration, interdependent teamwork, and serendipitous social encounters, spaces should be optimized to support:

• Brainstorming and idea generation: spaces that teams can “own” for some period of time, with plenty of display space (both physical and digital) and where teams can have extended discussions when they are together.
• Team integration: Spaces where teams who are working together in tightly integrated teams can be aware of and available to each other, sitting nearby, but with other places they can retreat to for focused work.
• Informal interactions: places where people can informally meet each other, such as communal seating areas, places for food and drink, and spaces designed for social activities.

Individual work

There are two important dimensions to consider for individual workspaces: dedicated space and private vs. open space.

Dedicated space:
It might be tempting to assume that because people will spend less time in the office, they will no longer need dedicated workspaces. For employees that do almost all their focus work at home, that may be true. However, for more hybrid employees, ‘hotdesking’, where employees either share a desk with others or are not assigned a permanent desk, may present challenges. Pre-pandemic research shows that hotdesking can lead to increased demands arising from distractions, a rise in uncooperative behaviours amongst colleagues, more distrust, and a perception of less supervisor support. Additionally, research has also shown the importance of at least two screens to support of knowledge work, which is possible, but less common with hotdesking.  Specialized ergonomic equipment can also be challenging with hotdesking.

Private vs Open space:
Research suggests open office spaces with dedicated desks can serve similar needs to traditional full offices when properly designed and even promote positive productivity when held to wellness certification standards. Alternatively “activity based” workspaces are another approach, where the workplace provides a range of different spaces for different activities and employees work flexibly across them as needed.

However, hybrid work means more video calls to remote colleagues, with more potential to disrupt others in an open office space. While “phone booths” that provide some audio privacy may help mitigate this issue, moving to a phone booth often leaves behind the work spread across screens in the workspace that is needed for a meeting.

How can we have better hybrid meetings?

See our Guide for Hybrid MeetingsOpens in a new tab

The post Suggestions for hybrid work appeared first on Microsoft Research.

What does hybrid mean?

Generally, ‘hybrid’ refers to a mix of co-located (in office or facility) and non-co-located work or workers. That mix can be within a single person or job, or it can be across individuals in a team, workforce, or group of people meeting.

 

• A hybrid worker: a person who spends some of their regular working hours in an office or other location shared with colleagues and some of their regular working hours working from their home or other location that is not shared with their work colleagues
• A hybrid team or workforce: a group where some people spend some of their time working from the same location. A team of hybrid workers is a hybrid team, but a team can also be hybrid without any hybrid workers if some of the workers are in-office and some are all-remote, or if everyone works in an office, but the team is split across multiple geographies.
• A hybrid meeting: a meeting where some attendees are co-located in one place and others somewhere else (either at home or another office)

For individuals, it is not obvious what cutoff to choose – someone who goes into the office twice a week is hybrid and someone who goes in twice a year is work-from-home, what about someone who goes in twice a month? The right threshold may depend on the context, but a useful starting point is to think of someone who works an average of at least one day a week in the office and at least one day a week at home as ‘hybrid.’ If someone works from home less than once a week, they are ‘mostly in-office’ and if someone goes into the office less than once a week, they are ‘mostly work-from-home.’

Microsoft’s Hybrid Workplace Flexibility Guide (opens in new tab) refers to a mix of workstyles across work site, work location (geography), and work hours. Microsoft’s policy uses a 50% threshold for needing manager approval and not having assigned on-campus office space, but that is a cutoff between ‘more-in-office hybrid’ and ‘more-at-home hybrid,’ not between hybrid and other modes.

The simple definitions and terminology can help us talk about hybrid work but should not mask the diverse implications and impacts it will have: on the timing of our workdays, formats of our meetings, the mix of tools we use, boundaries between our home and work lives and the nature of our relationships, teams, and social networks.

Other useful terms related to ‘hybrid’

Pre-Covid, work was often defined as working in a shared, consistent, company-owned space for a set number of hours. Earlier papers on ‘telecommuting’ (or hybrid work) such as that by Allen et al. (opens in new tab) were specifically looking at deviations from that pattern. The Covid pandemic caused work to shift drastically, and the new world of work is often referred to as ‘hybrid.’ However, the new world of work involves many facets such as: flexibility, non-standard employee/employer relationships, working-from-home, etc. Discussions will be clearer if we use different terms for the different aspects of work rather than lump them all under a ‘hybrid’ umbrella.

All-remote/remote

‘Remote’ is often used as the opposite of ‘co-located,’ but it is not a useful term unless it is clear what the person is remote from. Employees that never go into the office are remote from their co-workers, but an employee may also be remote from their collaborators if they work in a satellite office (either type of employee may be part of a hybrid team). ‘All-remote’ refers to companies that do not have office space, so all employees are remote from each other (at least most of the time, they may get together in-person yearly or quarterly).

Flexibility

For some people, hybrid work comes with flexibility – employee choice in location or hours or maybe other things like how work is done, whether to wear pajamas, etc. For instance, some people may now have the flexibility to go for a walk while listening to a meeting or do a yoga class between meetings and show up sweaty, or put in some hours on Saturday in order to go to the grocery store at 9am on Tuesday.  Hybrid work does not have to be flexible and flexible work does not have to be hybrid (but flexibility is core to Microsoft’s own Hybrid Workplace Flexibility Guidelines (opens in new tab)).

Non-standard employee types (e.g. ‘gig work’)

The future of work may involve more people working for multiple employers, either through traditional contract work, gig work, or other evolutions of worker-employee relationships.

Geographic and time-zone dispersion

Hybrid or all-remote workforces allow for more dispersion in the geographic location of employees. If they do not have to go into the same office, employees may live in different cities, states or even countries. This can be particularly relevant for collaboration when people live in very different time zones and synchronous collaboration becomes challenging or impossible within a standard 9-5 day.

Working from home

Hybrid tends to involve some amount of working from home, but not everyone who works from home is hybrid: in addition to all-remote jobs that we discussed earlier, someone who goes to the office five days a week and then does more work in the evenings or weekends from home should not be considered hybrid.Opens in a new tab

The post What is hybrid? appeared first on Microsoft Research.

Hybrid meetings – meetings in which there is a mix of remote and in-person participants – aren’t a new phenomenon. Microsoft’s approach to hybrid meetings is informed by our long history of videoconferencing and workplace research (opens in new tab), especially that on hybrid meeting successes and failures (opens in new tab) and configurations (opens in new tab), as well as what we have learned during the pandemic (opens in new tab).

Hybrid meetings create imbalances in knowing how one is perceived, visibility of conversational cues, access to technology and resources, and distribution of power.

• It is hard for remote people to hear concurrent talk and see individuals in a local room.
• It is hard for remote people to engage with physical resources in a local room.
• It is hard for local groups to do things in parallel with remote people.
• It is hard to know how engaged remote people are in the meeting.

Remote participation in hybrid meetings is more disadvantageous for new or junior attendees and minorities of culture, geography, identity, physical ability, health, neurodiversity etc.

This guide focuses on the trade-offs between in-person, remote, and hybrid meetings and how to make hybrid meetings as effective and inclusive as possible.​​​​​​​​​​​​​​​​​​​​​ There is a separate set of tips for hybrid workplaces more broadly (opens in new tab).

Remember, not all collaboration requires a meeting (opens in new tab). Have a conversation about your shared goals and then choose and balance synchronous and asynchronous collaboration.

The post Hybrid meetings guide appeared first on Microsoft Research.

Most videoconferencing platforms enable attendees to post text, images, files, links etc. in a meeting chat pane or window simultaneously with audio/visual (A/V) modalities. remote This parallel chat is typically open to all meeting invitees and flows concurrently with the A/V focus of the meeting (for example a presentation or discussion).  ​​​​​​​Our analysis of parallel chat in work meetings (opens in new tab) found that while parallel chat is considered a net positive, it can also be distracting, especially in meetings that involve more than a small group of people. Parallel chat may also not be accessible or inclusive.

Our general position is that organizations should exploring more intentional collaboration (opens in new tab). These guidelines should help organizers/moderators and participants to be more intentional about using parallel chat in meetings. We cover what makes parallel chat useful or distracting, and guidelines for before, during, and after meetings.

​​​​​​​What makes parallel chat useful versus distracting?

Parallel chat is used for at least seven distinct types of messages

• Questions for the speaker or someone else in the meeting
• Links to resources such as documents and webpages
• Unrelated conversation held in the same chat
• Voicing agreement with the speaker, or sending messages of praise/congratulations (‘kudos’).
• Adding information to what is being said, or starting a conversation about a related topic
• Responses to previous messages
• Humour and casual conversation

Positive impacts of parallel chat

• Inclusion and managing the flow of the primary conversation: A key advantage of parallel chat is participation without interrupting the flow of the A/V conversation. Being able to ask a question or make a comment in parallel chat may reduce the competition for the floor. Moreover, parallel chat gives participants a way to engage if they are otherwise unable to get a chance to speak
• Coordination of action and collaboration: Another key function of parallel chat is to share links to relevant resources and documents. Such sharing might have otherwise been follow-up actions.If parallel chat persists beyond the end of the A/V meeting, it can act as both a record and a means of enabling post-meeting discussion. Parallel chat also enables coordination during technical issues, coping with language barriers, and written precision (e.g. for technical terms).
• Social connection: Casual conversation and humor can provide social support and connection.

Negative impacts of parallel chat

• Distraction and division of attention: Parallel chat provides room for unrelated topics to emerge, distracting meeting participants who wish to focus on the meeting topic. Participants may feel obliged to divide their attention, which is difficult to maintain.
• Differing expectations on how chat should be used: Informality and side conversations may be perceived negatively because the make it difficult to find specific points in chat due to message volume or topic irrelevancy (16 participants). Some participants may want more concrete norms and expectations around parallel chat use, while others like having designated moderators to ensure professionalism and respectful behavior, or to monitor the flow of the meeting and ensure voices are heard.
• Information asymmetries: Presenters find presenting and following parallel chat difficult.

Considerations for participants

Here are some questions that will help guide you to maximizing the value of chat and help it from becoming distracting.

Check and act accordingly:

• Are there standing guidelines (formal or informal) for using parallel chat in this meeting?
• Are there participants with accessibility needs?
• How do presenters feel about parallel chat?

Post when:

• you will be advancing the goals of the main meeting.
• you have listened before posting (to ensure that you are not being repetitive).
• you are asking a question relevant to the main meeting.
• you will be clarifying others’ confusion or providing useful background material (including links).
• your post promotes inclusion – even your own if you have not had the opportunity to speak or don’t feel comfortable speaking.
• if you are posting an aside, even if humorous, try to ensure that it will be interesting to most participants

Refrain from posting when:

• you will be not be advancing the goals of the main meeting.
• your post will be of interest only to a small subset of participants.
• you will create or add to a forked conversation that lasts more than a few turns (make a plan to take forked conversations into other/later forums).
• you have posted a lot (number or length of posts) relative to the size of the group (i.e. try not to dominate the chat, even just because you are excited).
• you are simply having trouble attending to the speaker (consider whether unconscious bias is an issue).

Consider accessibility:

• If there are blind/low-vision attendees, be prepared to describe images in text or check for alt-text.
• Is the amount of chat potentially overwhelming for other attendees with multi-tasking challenges?
• Am I clearly communicating sentiment, for example when using emoji (e.g., ), acronyms, or  non-literal talk?
• Does someone else’s chat contain ambiguous sentiment? If so – ask about ambiguities rather than assuming intent.

Considerations for organizers/moderators

Parallel chat can be very effective when it is planned for and expectations are communicated among team members and meeting attendees. In larger meetings it also benefits from light moderation.

Before the meeting

Set the stage

• For recurring meetings, develop and circulate a standing set of guidelines for use of parallel chat (feel free to adapt from this document). Ideally, recurring meetings involving more than a few people already have a set of written guidelines, laying out goals and standing agenda items, to which any parallel chat guidance can be appended.
• For any meeting, consider putting parallel chat guidance into the chat at the start of each meeting, as a reminder. This can reduce the need for a moderator to “police” chat during the meeting.
• For a one-off meeting, or meetings with a mix of agenda item types (e.g., both informational and discussion items), tailor parallel chat guidance to the agenda goals.  For example, in a discussion or brainstorm section of the meeting, parallel chat will be useful for ideas, disagreement, questions, etc. For getting through an update efficiently, parallel chat could be reserved for clarifying questions.
• If new people are joining the meeting, moderators should make a practice of checking in with them beforehand, for example to understand how they prefer to be addressed (including name pronunciation), to identify any accessibility requests, etc. This function may already be performed by a general meeting moderator, in which case chat-specific questions could simply be added to the check-in (for example, are @mentions by first name only okay with the person?).
• Depending on the meeting, chat moderation can be hard work.  To ensure that those who take on this role also get their say, consider rotating the moderator role across different meetings, and/or distributing some aspects of the role across multiple people for more complex meeting.

Consider accessibility needs of meeting members

• Compile and share best practices with your group by:
  • Prior to the meeting, asking participants for any accessibility accommodation requests.
  • Based on any requests, compiling, and sharing a list of best practices for your group.
  • Make this list part of onboarding for new group members.
• Familiarize yourself with how to include participants who may face the following challenges:
  • Processing parallel sources in multiple modalities: Participating in a meeting with text-based chat requires monitoring two parallel sources, which may be difficult (e.g., due to difficulties navigating between sources while using a screen reader or magnification). Simultaneous presentation of information through auditory and visual channels may also present barriers (e.g., blind participants must listen to both sources via screen reader, or deaf participants must watch two sources via transcription/interpreter).
  • Consuming and generating text: Reading may present challenges (e.g., for people with low vision who must zoom and pan, and people with dyslexia). Writing may also present challenges (e.g., for people with hand mobility problems or who do not touch type).
  • Understanding non-text chat: Non-text content may not be accessible, particularly for participants using screen readers (e.g., GIFs and emoticons without or with inadequate alternative text).
  • Understanding sentiment: It can be difficult to understand the sentiment behind written and non-written content (e.g., GIFs), particularly for participants with autism.

During the meeting

Parallel chat to encourage

• Chat that amplifies the meeting’s main A/V includes:
  • Links to items mentioned in or related to the main A/V.
  • Questions/requests for clarification.
• When solicited: discussion contributions, including disagreement.
• Requests for follow-up.
• Enabling more voices to be heard includes chats from:
  • People who rarely speak or chat in meetings.
  • People who haven’t already spoken or chatted a lot in this meeting.
  • People who may generally have a hard time getting a word in for many possible reasons, including low power or seniority status, minority status, or disability.

Best practices for encouraging useful parallel chat

• Bring useful chat to the speaker’s attention, to encourage transfer of relevant ideas into the main conversation; highlight key points and clarifying questions.
• Ask participants to speak key written chat content out loud, both so the content enters any meeting recording and/or transcript, and for participants who may miss the chat; examples include participants such as presenters, or others for whom monitoring multiple sources simultaneously is difficult or impossible (as with certain disabilities).  If participants are unable to speak their own content, speak it for them.
• Remind people of the option to raise their hands.
• Apply inclusive behavior guidelines to parallel chat. For example:
  • When contributions are requested, encourage people who haven’t yet spoken or chatted.
  • Acknowledge when an idea advanced by one person gets ignored, then later credited to someone else.
  • Follow up privately with specific participants if you have concerns that certain chat entries may have affected them negatively.  Follow up can happen during or after the meeting, as appropriate.
• Align the chat with the accessibility requests of the group, for example by:
  • Providing text descriptions of non-text chat.
  • Providing text descriptions of visual content shared in the meeting (e.g., through screen share, or in a participant’s video feed) that has not been described.
  • Leaving appropriate time for participants with disabilities to respond (e.g., if using an interpreter or screen reader).

Parallel chat to discourage

• Chat that is likely of interest to only a small subset of attendees.
• Chat that diverges from the main presentation or discussion.
  • This can happen when a post generates a chat-only discussion that was either off-topic to begin with or becomes off-topic as the main AV meeting agenda moves on.
  • Such discussions may raise useful topics, but if they persist “too long” they risk distracting not only the chatters involved, but also everyone else in the meeting. How long is too long is exactly the kind of judgment call moderators are asked to make.
• Chat that overwhelms the main conversation
  • On-point chats can still become more intensive than the main discussion, especially in large meetings.  When chats start flashing fast and furious on the screen, participants may perceive them as “higher volume” than the main meeting.
  • On the one hand, the norms of in-person meetings would label side conversations as rude. On the other hand, a major benefit of parallel chat is its potential for greater inclusivity.
  • One possible way to distinguish these two scenarios is to notice whether the high volume is a result of the chat engaging a large fraction of meeting participants (more likely to be a marker of inclusion) or engaging a few participants intensely (more likely to be a marker of distraction).
• Chat that results from multi-segment meeting structure
• In meetings with multiple segments of interest to different attendees, the chat window may frequently get out of sync with the main AV agenda. For example, a speaker in one segment may respond to questions during another segment; or a chat discussion may become divergent (as discussed above) when it starts during one segment then persists into another segment. This latter scenario is especially likely if the different segments are aimed at separate sub-groups of attendees.
• Chat that is inaccessible to meeting participants. For example, depending on the participant group, inaccessible chat could entail:
  • Visual content that lacks text description (which may be inaccessible to blind or low-vision participants)
  • Subtle sentiment implications that are difficult to perceive
  • Any other accessibility requirements participants have shared.

Best practices for discouraging distracting parallel chat

• If you need to ask an individual to refrain from further chats on a topic, use a private backchannel to avoid public shaming.
• For divergent topics or intense small group discussions that have gone on “too long,” ask the participants to shift to backchannel so as not to distract others.  If the topic is important but not appropriate for further discussion in the main AV meeting, suggest a follow-up meeting.
• For distracting chats within multi-segment meeting structure: A Project Tahiti journal entry suggests that pausing between segments to handle that segment’s chat and establishing a norm of out-of-band communication to address questions or discussion beyond the pause, can be an effective practice for parallel chat for this type of meeting. (See first verbatim in the Appendix for the full entry.)

After the meeting

• Follow up on the parallel chat. If items in the meeting chat text deserve more attention than the main meeting AV was able to give them, consider sharing such items in a short follow-up email to the meeting participants. If a meeting recap is already part of the meeting’s regular practice, the chat recap can be bundled into it. Examples of useful items include:
  • Relevant ideas that did not get attention during the meeting
  • Links to relevant information
• This practice can also reduce help reduce distraction by reducing the pressure on participants to split their focus between the main meeting and the chat, knowing that chat highlights will be shared later.  Reducing this pressure may be particularly helpful to participants who find it challenging to multitask in this way, possibly (but not only) because of a disability.

More information on accessibility

For more information on making remote interactions more accessible, especially for large events such as conferences, see the ACM Guide to Accessible Remote Attendance (opens in new tab). For more information on inclusive practices in general, see Microsoft’s 10 inclusive behaviors (opens in new tab).

The authors thank Jaime Teevan and Priscilla Wong for their ideas and pointers to other sources that contributed to the original “Guide for Moderators” framing. We also thank Irene Money and Phil Rosenfield who reviewed an earlier version of this document, and others who provided feedback.Opens in a new tab

The post Suggestions for parallel chat in meetings appeared first on Microsoft Research.

Here are some suggestions for remote onboarding of new hires based on a survey and interviews of Microsoft employees (details are in the research paper). Page numbers refer to where the findings are discussed in the synthesis report.

Promote communication and asking for help

Remote work often makes asking questions harder. Without being able to ‘drop by’, communication may feel less casual—and emails and IMs generate a record, which can make people hesitant to ask potentially ‘stupid’ questions (page 25). Everyone should share their communication preferences with their colleagues, but it is particularly important that new hires be encouraged to ask questions, and managers need to communicate what medium is preferred for what types of communication.

Help new hires connect with the team

Encourage teams to turn on cameras while onboarding new hires

People like being able to put a face to a name, so, if possible, use a camera when in meetings.  Managers that turn their cameras on can act as role models for team members.

Schedule 1:1 meetings

Of new hires surveyed, 40% said they did not have at least one 1:1 communication from someone on their team daily. For employees in general, manager 1:1 time is associated with less of an increase in work hours during WFH.

Provide information about the organization early on

In large organizations, working remotely can make it particularly hard for new hires to understand how their individual role and their team’s role fit in the broader context.

Emphasize team building activities

Social and collaborative isolation is a major challenge for remote work (page 9). New hires need extra help connecting with their teammates. Team connection is essential for productivity and job satisfaction. Managers should host virtual events and encourage all team members to join in. These events can vary from a coffee hour to a team game night.

Assign an onboarding buddy

An onboarding buddy, often from a different team in the organization, helps a new hire find the resources they need, connects them with others within the company, and helps make the new hire feel welcome. By making the mentor role explicit, managers can alleviate the anxiety around sending multiple and frequent questions.

For developers / programmers—help new hires get started on work

Assign a technical mentor

A technical mentor—someone on the same team and who has worked with the code—helps a new hire with the technical details (e.g., code questions) of their work, that the onboarding buddy may not be able to help with.

Assign a simple first task

In one Microsoft group, 24% of new hires that started during WFH did not complete any pull requests in their first 90 days (page 31). Until new hires have gone through the process of creating their first pull request, they may not understand the workflow and be lost during some of the discussions in meetings. Having new hires complete a simple task during their first week—such as fixing a spelling error—gives them the opportunity to quickly go through the process of making changes, creating a pull request, submitting it, and having it reviewed.

Provide up-to-date documentation

New hires that had ample documentation reported that they had a smooth onboarding process. Up-to-date documentation can be particularly valuable during remote work when it is harder to ask colleagues for help. However, new hires also need context to understand where they fit into the project in relation to the documentation. In addition to technical guidance, providing documentation on workflow information, can also be helpful in giving new hires a guide to the team’s day-to-day language.

Much of the research motivating this guidance is in Please Turn Your Cameras On: Remote Onboarding of Software Developers during a Pandemic.Opens in a new tab

The post Advice for remote onboarding of new hires appeared first on Microsoft Research.

Remote meetings can be really valuable for togetherness and immediacy among colleagues. However, the COVID-19 pandemic has taught us that the artificiality of videocalls, exacerbated by the increase in meetings, is leading to fatigue. How can you preserve the value while reducing the problems? Our position is that thoughtless collaboration will always be poor collaboration no matter how good or bad the technology is.

This guide to better remote meetings draws principles from the Collaboration and Meetings section of The New Future of Work synthesis report. Page numbers in the report are provided if you want to dig into the findings.

The guide covers three aspects of intentionality.

1. Choose and balance asynchronous and synchronous collaboration.
2. Be more intentional about behavior throughout the remote meeting lifecycle.
3. Reflect on how intentionality has affected your actions as individuals, teams, and organizations.

Choose and balance synchronous and asynchronous collaboration

Instead of just having meetings, choose when meetings or asynchronous modes will suit your organizational goals and communicative needs.

Meetings are fast. For many organizations, meetings have become convenient container in which we hope work gets done because of the façade of easy scheduling and dynamism in the moment. The best meetings, especially when remote, promote interactivity over information sharing.

• Meetings are important for being connected with people, for building relationships, for getting to know each other.
• Meetings can be useful when the subject of discussion is complex or uncertain enough to warrant conversation to avoid potential miscommunication, but only if everyone is prepared.

Asynchronous modes scale from fast to slow.

• Asynchronous modalities overall are more searchable, they give people more time to consider the relevant issues, and may be more time-efficient because composing a message and reading a message take very different amounts of time.
• A team that has a place that everyone is expected to update regularly and is considered the ground truth should be able to reserve meetings for critical decisions and team members who find asynchronous work hard.
• You can reduce the burden of spontaneous meetings by first checking if your answer exists in some documentation and check recipient’s presence and working hours before reaching out.

Asynchronous modes in the Microsoft ecosystem:

• Microsoft Teams channels and chat for quick engagement of variable length and detail, and can upgrade to a meeting.
• OneNote and document collaboration allow more thoughtful long-form explorations, explanations, and planning.
• Yammer suits community building, which is rapid in the moment but with a distinctly longer term objective.

Be more intentional about behavior throughout the remote meeting lifecycle

Be intentional about scheduling meetings

Do you need a meeting?

• The primary burden is on the meeting organizer to explain the need for the meeting. Needing people’s attention on something should not be conflated with taking the time to meet about it.
• An empty time slot is not simply an open invitation for a meeting. Ask people whether they prefer meetings in a row or time between meetings, and if your meeting is very important, ask if they will need time for preparation or a break before.
• Check everyone’s timezone, including holidays, and share the burden of early/late meetings.
• Don’t double or triple book without asking.
• Provide people with a way to catch up with the content if they can’t attend.

What is this meeting about?

A meeting should have clear goals and an agenda to meet them.

• What is the project goal and how does this meeting fit?
• The meeting agenda should guide:
  • Who is invited/attends and in what capacity. It should be clear who is expected to fully engage, who could monitor a meeting, and generally keep attendance as small as possible.  As a potential attendee, you should choose the meetings you attend carefully.
  • When the meeting is. How much time do people need to prepare? How long should the meeting last? Recurring meetings need more reasons to exist, not less.
  • Where the meeting is. If some attendees are remote, consider having everyone join the meeting from their own device instead together in a conference room. This levels the playing field for all attendees. If you intentionally schedule a hybrid meeting, set everyone’s expectations for remote participation.

Be intentional about conduct in meetings

As a meeting owner

At the beginning of the meeting start with introductions and set expectations (verbally or in the invite/chat). We suggest using the CHARM framework:

• Chat: How should we use meeting chat (see more on this below)?
• Hand raising: Should we use hand raising (a feature or even actual hands!) or just jump in?
• Agenda: What are the meeting goals and agenda?
• Recording: Should we record and has everyone agreed?
• Moderator: Who will moderate the agenda, introductions, and questions.
• Support: How are we supporting one another?

At the end of the meeting, ensure follow-up:

• Leave time for summarizing action-items at the end of your meeting.
• When people say things like “we’ll take this offline”, ensure that there is follow-up and if possible, someone from that offline discussion should report back on it somewhere.

As an attendee

Be intentional about your role in the meeting:

• Do you want to participate? Let the organizer know if you just want to ‘follow’ the meeting. There may be a way for you to get updates without attending and they will know whether to solicit your input. You should be able to decline a meeting, but it’s also helpful if you tell people why you’ve said no.
• Ask: why and how am I contributing now? Speak up and encourage others to do so, but have a reason.  ‘Air time’ is limited, but parallel chat and shared documents can help make contributions both inclusive and rich.
• Listen: What is everyone saying? Meetings need to be as much about listening as they are about contributing. Take turns to listen. Help others listen, too, by actively seeking clarity.

Taking turns is one of the biggest challenges in all-remote meetings (page 11). Some things you can do to help are:

• Be clear about when you have reached the end of your thought.
• Be deliberate about selecting next speakers. Be on the lookout to include people who have not yet participated.
• Wait longer for responses than you would for an all in-person meeting.
• Share re-starts. If you re-started last time, let someone else re-start this time. If it keeps happening, pause the meeting and actively establish a way to deal with it.

As a moderator

To help meetings achieve their goals, active moderators should:

• Be energetic, to encourage meeting attendees to be engaged.
• Be up front on how the meeting will be run, including expectations for video on/off, audio muting, and other technical issues.
• Use private chat as well as public announcements to improve inclusion.
• Consider asking remote attendees to speak first, as they will have the hardest time getting a word in edgewise to in-room discussions.
• Delegate tasks to other people if they are overloaded.
• Others should help the moderator speak, if they have been caught up helping others.

Intentional use of tools for meetings

Video and visual representations of audio participants

• Showing video should not be a one-size-fits-all requirement (page 12).
• Consider inviting people to turn on video by explaining why it will help. Not everyone may turn on their video, but a proportion will.
• Individuals and teams should decide on and communicate their video preferences. This may be especially important to ensure inclusion of people with diverse abilities.
• In presentation scenarios, don’t leave presenters speaking into a total void of no-video and muted-audio. If not everyone can or should show video, choose a few representative people to turn their video on, to help the presenter ‘read the room’.
• Video has generally been found to make meetings more comfortable, but not necessarily more effective in every situation. It can have great value in certain situations, such as helping the comprehension of non-native speakers.

Parallel chat in meetings

Parallel chat has become much more common in meetings (page 13). It can improve inclusion but can also be distracting. We have a longer guide to parallel chat in meetings for moderators and participants (based on a research paper), but the basics are as follows:

• Encourage parallel chat that allows more voices to be heard and that amplifies the meeting’s main audio/video. For example, links, on-point questions, clarification, solicited contributions including disagreement, requests for follow-up.
• Consider discouraging parallel chat that is likely of interest to only a small subset of attendees, diverges from the main audio/video, or overwhelms the main conversation.
• Parallel chat can benefit from active moderation.
• Always check on accessibility accommodation if parallel chat will be important.
• Use a follow-up communication to share chat items that deserve special attention.
• Ensure that tabled items from chat are followed-up.

Active agendas in working session meetings

An active agenda is a live shared document that all attendees use in parallel with the meeting itself. It is most useful when the meeting itself is a working session.

• Create a shared document, pre-populate it with the agenda, and encourage contributions.
• The value is both a permanent record and richer interaction because it allows threaded/organised responses. While audio requires a single focus, several text conversations can be occurring in parallel in relevant sections of the document.
• It helps if there is a meeting secretary taking notes and surfacing relevant points to the meeting.

Reflect

A culture of intentionality is also a culture of experimentation.

• Writing down a line or two about meetings afterwards will help individuals and teams reflect later.
• We think that meetings are at their most valuable when being together (space) harnesses the immediacy (time) of the presence of people as skilled and emotional beings (sociality) to enable collective reasoning (purpose). Notes about space, time, sociality, and purpose will quickly build into a valuable archive.
• Discuss the results of intentionality at the end of some period you decide (e.g. two weeks, a month, a quarter).

The key to better meetings is to be more intentional throughout the meeting lifecycle and to consider how meetings fit into our larger workflow. When we add technology into the mix, this means being more intentional about how technology shapes our experiences in and around meetings.Opens in a new tab

The post A guide to having better remote meetings by being more intentional appeared first on Microsoft Research.