{"id":328679,"date":"2016-12-01T08:00:46","date_gmt":"2016-12-01T16:00:46","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=328679"},"modified":"2016-12-02T07:50:45","modified_gmt":"2016-12-02T15:50:45","slug":"making-better-use-of-the-crowd","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/making-better-use-of-the-crowd\/","title":{"rendered":"Making better use of the crowd"},"content":{"rendered":"

By Jennifer Wortman Vaughan, Senior Researcher, Microsoft Research<\/em><\/p>\n

\"Jennifer

Jennifer Wortman Vaughan, Senior Researcher, Microsoft Research
Photography: John Brecher<\/em><\/p><\/div>\n

Over the last decade, computer scientists have harnessed crowds of Internet users to solve tasks that are notoriously difficult to crack with computers alone, such as determining whether an image contains a tree, rating the relevance of websites, and verifying phone numbers.<\/p>\n

The machine learning community was early to embrace so-called crowdsourcing to quickly and inexpensively obtain the vast quantities of labeled data needed to train machine learning systems how to classify images or recognize speech, for example. Labeled data are essentially sets of teaching examples, such as pictures of cats that are tagged with the word \u201ccat.\u201d<\/p>\n

Usually this handoff of labeled data is where interaction with the crowd ends. Are there better ways to make use of the crowd?<\/p>\n

On December 5, I will showcase innovative uses of crowdsourcing that go beyond data collection during a crowdsourcing tutorial<\/a> at NIPS<\/a>, the premier international machine learning conference, held this year in Barcelona. The tutorial will also dive into recent research aimed at understanding who crowdworkers are and how they behave, which could inform best practices for interacting with the crowd.<\/p>\n

Innovative uses of crowdsourcing that go beyond the collection of data include:<\/p>\n