{"id":189634,"date":"2013-07-19T00:00:00","date_gmt":"2013-07-22T12:20:23","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/msr-research-item\/integrating-algorithmic-and-behavioral-approaches-to-crowdsourcing\/"},"modified":"2016-08-02T06:11:53","modified_gmt":"2016-08-02T13:11:53","slug":"integrating-algorithmic-and-behavioral-approaches-to-crowdsourcing","status":"publish","type":"msr-video","link":"https:\/\/www.microsoft.com\/en-us\/research\/video\/integrating-algorithmic-and-behavioral-approaches-to-crowdsourcing\/","title":{"rendered":"Integrating Algorithmic and Behavioral Approaches to Crowdsourcing"},"content":{"rendered":"<div class=\"asset-content\">\n<p>With the advent of crowdsourcing and crowd work, human intelligence can be brought to bear on a great variety of useful tasks and on short notice. However, because of the varying attention span and inevitable errors made by human contributors, much research has focused on algorithmic and AI-based approaches to optimizing the quality of results aggregated from human input. In this talk, we present several projects providing examples of how human behavior in crowdsourcing can be not only modeled, but also managed. We show how the attention of crowd workers can be accurately predicted by the combination of certain features, allowing for targeted interventions. We present an experiment showing how different financial incentives, even at the same overall level of payment, can result in a trade-off between quality and speed. Finally, we provide some interesting results comparing paid workers to volunteers and the attention span of workers on Amazon Mechanical Turk. Our work sets the stage for richer crowdsourcing algorithms that can adapt to and even take advantage of differences in human behavior.<\/p>\n<p>Joint work with Ece Kamar, Eric Horvitz, and Yiling Chen.<\/p>\n<\/div>\n<p><!-- .asset-content --><\/p>\n","protected":false},"excerpt":{"rendered":"<p>With the advent of crowdsourcing and crowd work, human intelligence can be brought to bear on a great variety of useful tasks and on short notice. However, because of the varying attention span and inevitable errors made by human contributors, much research has focused on algorithmic and AI-based approaches to optimizing the quality of results [&hellip;]<\/p>\n","protected":false},"featured_media":197751,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"footnotes":""},"research-area":[],"msr-video-type":[206954],"msr-locale":[268875],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-189634","msr-video","type-msr-video","status-publish","has-post-thumbnail","hentry","msr-video-type-microsoft-research-talks","msr-locale-en_us"],"msr_download_urls":"","msr_external_url":"https:\/\/youtu.be\/n2Y8RWkhH8\/","msr_secondary_video_url":"","msr_video_file":"","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video\/189634"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-video"}],"version-history":[{"count":0,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video\/189634\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/197751"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=189634"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=189634"},{"taxonomy":"msr-video-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video-type?post=189634"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=189634"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=189634"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=189634"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}