{"id":301163,"date":"2016-09-26T00:00:16","date_gmt":"2016-09-26T07:00:16","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-research-item&#038;p=301163"},"modified":"2017-07-11T15:02:05","modified_gmt":"2017-07-11T22:02:05","slug":"counterfactual-evaluation-learning-logged-user-feedback","status":"publish","type":"msr-video","link":"https:\/\/www.microsoft.com\/en-us\/research\/video\/counterfactual-evaluation-learning-logged-user-feedback\/","title":{"rendered":"Counterfactual Evaluation and Learning from Logged User Feedback"},"content":{"rendered":"<p>Interactive systems like search engines, recommender systems, and ad placement platforms are ubiquitous. Evaluating and optimizing these systems is hard &#8211; users&#8217; feedback governs system performance, and gathering their feedback in repeated randomized experiments is costly. I study how we can use logs collected from deployed systems to perform offline evaluation and learning. I will outline two projects (Evaluation: Recommendations as Treatments, ICML&#8217;16 and Learning: Counterfactual Risk Minimization, ICML&#8217;15) that advance the state of the art for these problems. I will also briefly reference the tutorial Thorsten Joachims and I taught at SIGIR&#8217;16 (<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"http:\/\/www.cs.cornell.edu\/~adith\/CfactSIGIR2016\/\">http:\/\/www.cs.cornell.edu\/~adith\/CfactSIGIR2016\/<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>) that explores these counterfactual evaluation and learning problems in more detail.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Interactive systems like search engines, recommender systems, and ad placement platforms are ubiquitous. Evaluating and optimizing these systems is hard &#8211; users&#8217; feedback governs system performance, and gathering their feedback in repeated randomized experiments is costly. I study how we can use logs collected from deployed systems to perform offline evaluation and learning. I will [&hellip;]<\/p>\n","protected":false},"featured_media":301181,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"footnotes":""},"research-area":[13556,13555],"msr-video-type":[],"msr-locale":[268875],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-301163","msr-video","type-msr-video","status-publish","has-post-thumbnail","hentry","msr-research-area-artificial-intelligence","msr-research-area-search-information-retrieval","msr-locale-en_us"],"msr_download_urls":"","msr_external_url":"https:\/\/youtu.be\/4I0zsPTZyP4","msr_secondary_video_url":"","msr_video_file":"","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video\/301163"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-video"}],"version-history":[{"count":1,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video\/301163\/revisions"}],"predecessor-version":[{"id":398666,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video\/301163\/revisions\/398666"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/301181"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=301163"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=301163"},{"taxonomy":"msr-video-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video-type?post=301163"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=301163"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=301163"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=301163"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}