{"id":192758,"date":"2015-09-16T00:00:00","date_gmt":"2015-09-16T15:00:16","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/msr-research-item\/big-data-and-bayesian-nonparametrics\/"},"modified":"2016-07-15T15:26:34","modified_gmt":"2016-07-15T22:26:34","slug":"big-data-and-bayesian-nonparametrics","status":"publish","type":"msr-video","link":"https:\/\/www.microsoft.com\/en-us\/research\/video\/big-data-and-bayesian-nonparametrics\/","title":{"rendered":"Big Data and Bayesian Nonparametrics"},"content":{"rendered":"<div class=\"asset-content\">\n<p>Big Data is often characterized by large sample sizes, high dimensions, and strange variable distributions.  For example, an e-commerce website has 10-100s million observations weekly on a huge number of variables with density spikes at zero and elsewhere and very fat tails.   These properties \u2013 big and strange \u2013 beg for nonparametric analysis.  We revisit a flavor of distribution-free Bayesian nonparametrics that approximates the data generating process (DGP) with a multinomial sampling model.  This model then serves as the basis for analysis of statistics \u2013 functionals of the DGP \u2013 that are useful for decision making regardless of the true DGP.  The ideas will be illustrated in the indexing of treatment effect heterogeneity onto user characteristics in digital experiments, and in analysis of decision trees employed in fraud prediction.  The result is a framework for scalable nonparametric Bayesian decision making on massive data.<\/p>\n<\/div>\n<p><!-- .asset-content --><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Big Data is often characterized by large sample sizes, high dimensions, and strange variable distributions. For example, an e-commerce website has 10-100s million observations weekly on a huge number of variables with density spikes at zero and elsewhere and very fat tails. These properties \u2013 big and strange \u2013 beg for nonparametric analysis. We revisit [&hellip;]<\/p>\n","protected":false},"featured_media":199267,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"footnotes":""},"research-area":[13548],"msr-video-type":[],"msr-locale":[268875],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-192758","msr-video","type-msr-video","status-publish","has-post-thumbnail","hentry","msr-research-area-economics","msr-locale-en_us"],"msr_download_urls":"","msr_external_url":"https:\/\/youtu.be\/XaGHXx9SJ4A","msr_secondary_video_url":"","msr_video_file":"","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video\/192758"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-video"}],"version-history":[{"count":0,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video\/192758\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/199267"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=192758"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=192758"},{"taxonomy":"msr-video-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video-type?post=192758"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=192758"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=192758"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=192758"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}