{"id":307076,"date":"2009-06-10T16:00:00","date_gmt":"2009-06-10T23:00:00","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=307076"},"modified":"2016-10-17T15:28:47","modified_gmt":"2016-10-17T22:28:47","slug":"remote-meetings-thinking-inside-box","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/remote-meetings-thinking-inside-box\/","title":{"rendered":"Remote Meetings: Thinking Inside the Box"},"content":{"rendered":"

By Rob Knies, Managing Editor, Microsoft Research<\/em><\/p>\n

George Robertson is taking this meeting seriously. He focuses intently on other participants in the room, making eye contact, noting posture and visual cues, interjecting comments when appropriate. He studies diagrams scrawled onto a whiteboard, and, on occasion, uses a laser pointer to call attention to something he wants to address.<\/p>\n

Sound like just another productive business meeting? Well, you\u2019re right\u2014except for one little detail. The meeting is being held in Redmond, and Robertson is in Northeast Harbor, Maine, more than 3,000 miles away.<\/p>\n

How is this possible? That\u2019s a question for the researchers behind the Embodied Social Proxies project, a joint effort between the Human Interactions in Programming<\/a> (HIP) and Visualization and Interaction for Business and Entertainment<\/a> (VIBE) groups within Microsoft Research Redmond<\/a>.<\/p>\n

\u201cFor a team that\u2019s mostly collocated, with one person somewhere else, that person is often out-of-sight, out-of-mind,\u201d says Gina Venolia<\/a>, a senior researcher in the HIP group. \u201cThey\u2019re left out of the loop on ad hoc conversations. It\u2019s hard to pull them in for planned meetings, because you put them on a speaker phone or on a projector and they get forgotten. They just aren\u2019t as real as the person next door.<\/p>\n

\u201cIt\u2019s very common for somebody to be looking at a problem that they\u2019re having and walk down the hall to talk to somebody about it. They\u2019re going to talk with somebody collocated more than they\u2019re going to talk with somebody who\u2019s remote.\u201d<\/p>\n

Such a scenario became underscored for Venolia and colleagues in autumn 2008, when Robertson, a principal researcher in the VIBE group, suddenly found himself thousands of miles from his Microsoft Research colleagues. His wife had seen her work take her from the West Coast to the East Coast, and her husband used the opportunity to examine a research question.<\/p>\n

\"Gina

Gina Venolia meets with George Robertson.<\/p><\/div>\n

\u201cMicrosoft was glad to let me do this remotely,\u201d he said, \u201cand shift some research toward finding ways to make remote or distributed teams work more effectively.\u201d<\/p>\n

As it turns out, Venolia had been thinking along the same lines.<\/p>\n

\u201cCould we build a device that always represented that remote person, to give that person some physical weight in the team\u2019s presence, to make it easier to be aware of what that person is up to, to make it easier to initiate ad hoc communication, and to be able to represent them physically in a meeting?\u201d<\/p>\n

While mulling the possibilities, Venolia recalled a scenario from her past.<\/p>\n

\u201cFifteen years ago or so,\u201d she explains, \u201cI was at [Silicon Valley-based] Silicon Graphics. I was in a group, and two of our senior people were located elsewhere. They worked out of their homes. Jim was in Columbus, Ohio, and Helga was in Reykjavik, Iceland. And these were important people.<\/p>\n

\u201cMy manager at the time had the brilliant idea to put up two clocks, one representing the local time at each place. And rather than labeling them \u2018Columbus\u2019 and \u2018Reykjavik,\u2019 they were labeled \u2018Jim\u2019 and \u2018Helga.\u2019 Over time, photos they had sent or postcards of their locales, Christmas Cards, vacation postcards\u2014whatever reminded us of Jim and Helga\u2014went up on the wall next to the clocks. That part of the hallway became a physical embodiment of Jim and Helga and made it hard to forget about them.\u201d<\/p>\n

Reflecting on that lesson, it didn\u2019t take long for Venolia to begin imagining a nascent research project. Add more information\u2014live information. Add a screen and provide information about the remote worker\u2019s activities. Analyze the person\u2019s communications. Even\u2014consider this\u2014combine it all into a package that could be transported into a real-life meeting room.<\/p>\n

Welcome to the concept of Embodied Social Proxies\u2014or, as the project inevitably came to be known, George-in-a-Box.<\/p>\n

In a nutshell, the researchers proceeded to connect a pair of computers to an array of cameras, enabling both hub and satellite presences to gain visibility into the other\u2019s activities. And when the monitor is not actually being used by team members during a meeting, it displays information about the remote user\u2019s calendar, activities, and availability.<\/p>\n

\u201cRather than trying to build the perfect system and see what was wrong with it,\u201d Venolia says, \u201cwe did a philosophy of underdesign: What is the least we could do to use this? We got a 15-inch laptop with a Webcam and quickly found out that there was a problem with the camera\u2019s field of view. We quickly found out that we needed a speakerphone.\u201d<\/p>\n

The team cobbled the necessary elements together, along the way borrowing a camera array from colleague Cha Zhang from the Communications and Collaboration Systems group at Microsoft Research Redmond. Put it all on a cart, roll it into a conference room, and there\u2019s George, with a seat at the table and ready to engage.<\/p>\n

\u201cMy moving to Maine was a catalyst for exploring these various things,\u201d Robertson says, \u201cso Gina and I got together to figure out how to make this happen.\u201d<\/p>\n

Robertson had concerns.<\/p>\n

\u201cHe was worried,\u201d Venolia recalls, \u201cabout: \u2018How do I stay aware of what\u2019s going on in my team? How do they stay aware of me? How do we not get out-of-sight, out-of-mind?\u2019 \u201d<\/p>\n

The two formulated a set of research questions for which they wanted to find answers: Was it possible to create a meeting experience almost as good as face-to-face? Could awareness displays increase ad hoc communication with a satellite teammate? Can physical presence of a satellite teammate increase his or her social presence in a hub team?\u201d<\/p>\n

It was time, Robertson says, to figure things out.<\/p>\n

\u201cYou\u2019d take that laptop to the meeting I was attending and have it occupy a physical space at the table. That did part of what we wanted. When people interacted with me, they turned and looked at me in the laptop.\u201d<\/p>\n

Then came a set of refinements. One problem was that while the hub-based team could view the remote member\u2019s reactions, gaze direction, and attention, thanks to a Webcam on the satellite computer, the experience, from the remote perspective, was more problematic.<\/p>\n

\u201cIt didn\u2019t work very well in terms of me being able to see what was going on in the meeting,\u201d Robertson says. \u201cThe Webcam view just was not wide-enough-angle, and I had no control over what I was looking at.\u201d<\/p>\n

So the project members outfitted George-in-a-Box with two more cameras. One, stationed above the monitor, is a pan-tilt-zoom camera operated via remote control, enabling the remote participant to focus on a whiteboard, an individual speaker, slides being projected during a meeting, or team members at a conference-room table.<\/p>\n

A third camera, affixed with a 140-degree, fish-eye lens, is stationed below the George-in-a-Box monitor, enabling the remote user to view practically everybody in the room, see their responses, watch who\u2019s looking at whom, and gain environmental awareness at what is occurring in the meeting room.<\/p>\n

\"George

George Robertson gets a pair of views of the meeting he is attending from a remote location. At left is a customizable look that includes Kori Inkpen Quinn (left) and Gina Venolia. At right is a fish-eye view of everybody in the meeting, including (from left) John Tang, Quinn, Venolia, and Bongshin Lee.<\/p><\/div>\n

\u201cThis setup works really well in meetings,\u201d Robertson says. \u201cI\u2019m able to, with all of these cameras, see how people are reacting to what\u2019s being said. I can follow pretty much everything going on in the meeting, if it\u2019s on the whiteboard or on the screen or interactions between people. Sometimes, I\u2019m actually leading the meeting, and if somebody\u2019s on one side of the table, and somebody else on the other side says something and I see that the first person doesn\u2019t agree with it, I will ask what her response is.<\/p>\n

\u201cThat combination of these two cameras, which I\u2019m controlling, turns out to be really effective.\u201d<\/p>\n

For Venolia, the cart presence just makes everything much more lifelike.<\/p>\n

\u201cGeorge can see us, and we can see George at all times,\u201d she says. \u201cGeorge can see very clearly when we\u2019re looking at him, so that enables really natural stuff like turning to him and saying, \u2018What do you think?\u2019<\/p>\n

\u201cThese cameras are actually little embedded Web servers, and they serve up a page that allows George to pan and tilt and zoom. The wide-angle camera is also a solid-state pan-tilt-zoom, but generally he keeps it so he can see everybody. Sometimes, he\u2019ll look down one side of the table with one camera and at the other side with the other. He wants to be able to quickly glance between who\u2019s talking and who\u2019s listening. You can\u2019t do that with manual camera control.\u201d<\/p>\n

Venolia, Robertson, and colleagues are not the first to head down this road. Other teams have placed a stuffed animal in a chair to represent an absent teammate. And, of course, we all have family mementoes to remind us of people who aren\u2019t present.<\/p>\n

But make no mistake: The Embodied Social Proxies project is unique\u2014in part because of its ingenious model.<\/p>\n

\u201cThis is really around this hub-and-satellite case,\u201d Venolia says. \u201cFrom my surveys of software engineers at Microsoft, about 16 percent have no collocated coworkers.\u201d<\/p>\n

That\u2019s a lot of disenfranchised satellites.<\/p>\n

In addition to the conversational benefits offered by the George-in-a-Box concept, there are also its awareness capabilities, although, Venolia and Robertson indicate, more work needs to be done on that front.<\/p>\n

\u201cWhen I walk past a colleague\u2019s office, I can see if he\u2019s around, if he\u2019s busy, if he\u2019s reading a magazine, if he\u2019s looking irritated,\u201d Venolia says. \u201cI can see what\u2019s going on his whiteboard, if there\u2019s new stuff there. I can see his printouts \u2026<\/p>\n

\u201cThere are two kinds of ad hoc communication. One is I need to talk to somebody: \u2018George is around. I\u2019ll call him.\u2019 For that, I need to know if George is there. Is now a good time to talk with him? What\u2019s he working on? What\u2019s his calendar look like? What\u2019s his recent and predicted availability?\u201d<\/p>\n

A second kind of awareness built into the project regards activity awareness: What is George working on?<\/p>\n

\u201cI might hear a couple of coworkers talking about revising a slide deck for project X that we\u2019re working on together,\u201d Venolia continues. \u201cI might jump in and say, \u2018Oh, you know, I meant to change this in the slide deck,\u2019 or \u2018You know, what I think really needs to be fixed is \u2026\u2019 \u201c<\/p>\n

\"The

The Embodied Social Proxies team enjoys a business meeting that includes “George-in-a-Box.”<\/p><\/div>\n

Those questions and scenarios are what the information offered on George-in-the-Box\u2019s awareness screen is designed to convey. His calendar is displayed, as is the time in his time zone, his busy\/free status, and a description of his current activity. Also listed is critical information for long-distance colleagues: the quality of service between hub and satellite.<\/p>\n

\u201cThe intention,\u201d Robertson says, \u201cwas to make it possible for somebody walking by the office to notice that I was available and have impromptu discussions.<\/p>\n

\u201cThe availability awareness display could be improved. The activity-awareness dashboard we have now doesn\u2019t capture all of the information we want to capture. The part that\u2019s worked the best is improving meeting attendance.\u201d<\/p>\n

Still, says Venolia, the awareness work holds plenty of promise.<\/p>\n

\u201cThis is maybe more aspirational than current,\u201d she says, \u201cbut the degree to which we\u2019re going to be pushing on work-activity awareness is very distinct. This concept of \u2018this device is George and always George\u2019 is definitely a novel thing.\u201d<\/p>\n

In addition to improving the awareness display, the team is also working on a set of challenges. Networking reliability is crucial for this kind of work, as is dependable wireless access, insignificant network latency, and the ability to support more than one remote user.<\/p>\n

In recent moves, John Tang<\/a>, a researcher based in Silicon Valley, also has joined the project as a satellite user. His cart sits alongside George\u2019s in their own office in Microsoft Research\u2019s Redmond headquarters. Work on multiple remote users is necessary for the project\u2019s next step: a six-week summer trial with a handful of Microsoft product teams including remote workers to see how the technology changes communication patterns.<\/p>\n

\u201cI\u2019ve made a bet here,\u201d Venolia says. \u201cI\u2019ve made a bet that having a hotline to George is a good thing. I don\u2019t know how that bet is going to play out. Should it be a hotline, or should it be a telephone? If it\u2019s a hotline, you could imagine adding semi-autonomous robotics so that George could just go to his next meeting.<\/p>\n

\u201cOn the other hand, rather than going robotic and drive the price through the ceiling, what if this was just something like a tea tray that was easily carryable and you could just place it at the end of a table?\u201d<\/p>\n

There are many tantalizing possibilities inherent in the technology.<\/p>\n

\u201cI would love to see either of these tried out in the home,\u201d Venolia says. \u201cWhat about Grandma? You\u2019ve got an elder living independently, but you want here there for dinner. You want to hang with her and watch American Idol<\/em>, because she loves that. That\u2019s what you do together.\u201d<\/p>\n

Meanwhile, Robertson, lacking robotics, must count on the kindness of colleagues to transport his cart from room to room.<\/p>\n

\u201cHaving somebody take on that role,\u201d he says, \u201cis critical to making this work.\u201d<\/p>\n

Among those happy to assist in helping George-in-a-Box traverse from office to meeting and back is Bongshin Lee<\/a>, a VIBE researcher particularly interested in the awareness-display component of the project.<\/p>\n

\u201cOne of the most important things,\u201d Lee says, \u201cis to have a dedicated office for him. \u201cBefore, we put him in one of the labs, and people didn\u2019t interact with him much. But now, even though they don\u2019t yet initiate an impromptu call with George often in this space, sometimes they check to see if George is available. And if the cart is not here, they wonder where George is.\u201d<\/p>\n

Having a human colleague represented in such a manner has its lighthearted moments. One of George\u2019s handlers has talked about leaving the cart behind and feeling as if she had left a baby in the back seat of a car.<\/p>\n

\u201cThese carts are still somewhat the person,\u201d Venolia says, \u201ceven though they aren\u2019t live. \u201cWhat\u2019s super-cool is that, when it works, it\u2019s just transparent. You stop thinking about the technology, and you\u2019re just there with these people. To me an interface is successful when you don\u2019t know it\u2019s there.\u201d<\/p>\n

Lee agrees.<\/p>\n

\u201cIt helps us interact with George better and have George interact with us better,\u201d she says. \u201cIt\u2019s fair to say I feel this is more like a George, even though it\u2019s a machine. It doesn\u2019t feel awkward. It\u2019s very natural.\u201d<\/p>\n

Team members\u2014including Kori Inkpen Quinn<\/a>, an expert on computer-supported cooperative work who has helped design the system and who is helping to evaluate it\u2014meet regularly to discuss the Embodied Social Proxies project. Refinements continue.<\/p>\n

For George Robertson, he with a bi-coastal daily presence, albeit it sometimes \u201cin a box,\u201d it\u2019s all about the bottom line.<\/p>\n

\u201cWhen I\u2019m in a meeting with the cart,\u201d he says, \u201cI feel much more engaged in the meeting, and I think people are more engaged with me. When they talk to me, they will turn and look at me in the cart, and I can tell that they\u2019re doing that.<\/p>\n

\u201cThese little social cues and being able to see how everybody in the room is reacting to what\u2019s being said \u2026 Those are the things that make meetings work effectively, and the cart really does make that possible.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"

By Rob Knies, Managing Editor, Microsoft Research George Robertson is taking this meeting seriously. He focuses intently on other participants in the room, making eye contact, noting posture and visual cues, interjecting comments when appropriate. He studies diagrams scrawled onto a whiteboard, and, on occasion, uses a laser pointer to call attention to something he […]<\/p>\n","protected":false},"author":39507,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"categories":[194476,194481,194461],"tags":[215105,215096,215081,215111,215090,215099,215093,215108,215102],"research-area":[13552,13554,13559],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-307076","post","type-post","status-publish","format-standard","hentry","category-devices-and-hardware","category-human-centered-computing","category-social-sciences","tag-awareness-displays","tag-distributed-teams","tag-embodied-social-proxies","tag-fish-eye-lens","tag-george-in-a-box","tag-live-information","tag-remote-teams","tag-satellite-teammate","tag-underdesign","msr-research-area-hardware-devices","msr-research-area-human-computer-interaction","msr-research-area-social-sciences","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[199565],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[144794,578422],"related-projects":[],"related-events":[],"related-researchers":[],"msr_type":"Post","byline":"","formattedDate":"June 10, 2009","formattedExcerpt":"By Rob Knies, Managing Editor, Microsoft Research George Robertson is taking this meeting seriously. He focuses intently on other participants in the room, making eye contact, noting posture and visual cues, interjecting comments when appropriate. He studies diagrams scrawled onto a whiteboard, and, on occasion,…","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/307076"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/39507"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=307076"}],"version-history":[{"count":3,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/307076\/revisions"}],"predecessor-version":[{"id":307109,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/307076\/revisions\/307109"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=307076"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=307076"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=307076"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=307076"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=307076"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=307076"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=307076"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=307076"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=307076"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=307076"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=307076"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}