{"id":806446,"date":"2021-12-17T16:07:14","date_gmt":"2021-12-18T00:07:14","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-blog-post&p=806446"},"modified":"2022-03-02T14:04:58","modified_gmt":"2022-03-02T22:04:58","slug":"intro-to-machine-learning-in-customer-insights_","status":"publish","type":"msr-blog-post","link":"https:\/\/www.microsoft.com\/en-us\/research\/articles\/intro-to-machine-learning-in-customer-insights_\/","title":{"rendered":"Intro to Machine Learning in Customer Insights"},"content":{"rendered":"

Authors:
\nTommy Guy<\/a>, Sally Kellaway, Zachary Cook, Julie Koesmarno<\/p>\n

Microsoft Dynamics 365 Customer Insights (opens in new tab)<\/span><\/a> accelerates time to value with Machine Learning-based predictions covering Product recommendations, Churn risk, Sentiment analysis and Customer lifetime value scenarios. These features were developed using vast data sets and advanced analytics to provide a comprehensive and timely understanding of customers. The complexity of these artificial intelligence features also introduces new questions about the scope and meaning of the results they produce. This article introduces some key concepts in Customer Insights\u2019 Machine Learning capabilities. Our goal is to demystify Machine Learning and help Customer Insights users gain more confidence in the predictions they can generate with these features.<\/p>\n

Machine Learning is a Generalization of Patterns in Data<\/strong>
\nFor sellers and marketers, growing a business requires a wealth of domain knowledge. This expertise is necessary to identify campaigns and activities that will help drive revenue. One activity might be to recommend products that will have a high purchase rate to recommend to customers in marketing campaigns. For instance, Cameron is a marketing specialist for Contoso Coffee. Cameron\u2019s experience may indicate that people who buy a new grinder are likely to also buy whole bean coffee. Cameron has also found that these customers are less likely to purchase ground coffee in the future.<\/p>\n

Simple rules like this are called heuristics. Heuristics don\u2019t scale well because each rule created requires the domain expert (in this case, Cameron) to encode their personal experience and direct observations from the data into the rule. Human-generated heuristics may encode biases (e.g., Cameron might have a subconscious preference for whole bean coffee and may look for confirming evidence) and may also suffer from the human expert\u2019s limited breadth of experience (e.g., Cameron might have only worked for Contoso for 3 months).<\/p>\n

Nevertheless, formulating good human-generated heuristics and good Machine Learning generated predictions share the same methods for reducing bias:<\/p>\n

    \n
  1. Cameron should check their intuition by assessing whether the patterns exist across longer time periods.<\/strong><\/li>\n
  2. \u00a0When Cameron recommends whole bean coffee, they understand why<\/strong> that recommendation was made by being able to provide an explanation<\/strong> to justify the recommendation.<\/li>\n
  3. Cameron should continue to reassess<\/strong> and re-validate their intuition to make sure the heuristic still holds. For instance, a shift to WFH could change coffee consumption habits in ways\u00a0 \u00a0 \u00a0 \u00a0that are hard to predict.<\/li>\n<\/ol>\n

    What are the components of a Machine Learning Prediction?<\/strong>
    \nLike all Machine Learning products, the     product recommendation model (opens in new tab)<\/span><\/a> offered in Customer Insights improves on human-generated heuristic rules. It has a few key components:<\/p>\n

      \n
    1. A business objective<\/strong>, which is the thing we are trying to optimize. If we are trying to predict future purchases by customers, the objective is to identify products or services that have the highest chance of being purchased.<\/li>\n
    2. An algorithm<\/strong>, which is the \u2018program\u2019 that will consume input data and make a prediction. Machine Learning algorithms contain variables that can be optimized during training to produce the most accurate predictions. The values of variables that optimize the predictions are called a model. For example, every Customer Insights product recommendation model starts with the same algorithm, but the model is trained to give optimal recommendations based on the product set and the purchase patterns of the dataset that the user inputs. We train machine learning models by \u201creplaying history\u201d to see if the model accurately predicts what happened in the data and update it to make better predictions. A recommendation algorithm may train a model by taking every customer\u2019s purchase history and using all previously purchased items to predict the last item that each customer purchased.<\/li>\n
    3. Machine Learning algorithms have features<\/strong> that describe aspects of the data. As an example, the product recommendation algorithm may create features by categorizing items in our catalogue (ground vs whole bean coffee, or beans vs grinders vs espresso machines). Other features may be manufacturers, colors, etc. Features are important for Machine Learning algorithms because they represent facts about input data that can be used to generate the predictions. Some features are constructed by the algorithm (as opposed to just extracted directly from the data). For instance, the Customer Lifetime Value prediction model computes the time between purchases from the total list of transactions because the frequency of purchase may be an important predictor of lifetime value in some scenarios.<\/li>\n<\/ol>\n
      <\/h6>\n
      \"Figure

      Figure 1- High level workflow from business objective to model predictions<\/em><\/p><\/div>\n

      How do Machine Learning Predictions differ from Human Generated Rules?<\/strong>
      \nDifferent models have different components and details that can be tuned to help generate a wide array of business predictions. We\u2019ll dive into these details in future blog posts and interviews. In this introduction, we\u2019ll discuss two of the key differences between Machine Learning model and human generated rules.<\/p>\n

        \n
      1. Machine Learning can consume vastly more data than a human can read and understand. This makes it impossible for a human to fully simulate every possible decision the model will make (e.g., it is impossible to list all possible categories of items that customers may have previously purchased for all of time). In fact, good models can generalize<\/strong> well to inputs that never occurred in the training data. In Contoso Coffee\u2019s transaction history, no human has ever bought a specific coffee grinder and French press together, but our product recommendation algorithm should be able to generalize that the user may be interested in an electric kettle. However, if we cannot \u201ccheck the model\u2019s homework\u201d, this can make it more difficult to know if we trust a Machine Learning model. Below, we discuss how Customer Insights validates our prediction models.<\/li>\n
      2. Machine Learning algorithms are only successful if the dataset contains features that describe the incoming data<\/strong>. In the example above, Cameron used their past knowledge to recommend whole bean coffee instead of ground coffee. Cameron implicitly used features of those products: they understood that coffee grinders are only useful when the coffee isn\u2019t already ground. Unless we include useful features like product category in our training data, the Machine Learning algorithm can\u2019t use that \u2018fact\u2019 to learn the best model. Unless \u201cground vs whole\u201d is a feature on bean products, a model couldn\u2019t learn to suggest one over the other. Our predictions will only be as good as the data used to train the model.<\/li>\n<\/ol>\n

        Is My Model Trustworthy?<\/strong>
        \nCustomer Insights applies several techniques to help you trust a model in the process of making business decisions. This section describes some of the quality and performance testing that Machine Learning engineers conduct when they build a new model. Stay tuned for more details about them in subsequent posts.<\/p>\n

        Is my model \u201coverfitting\u201d to the training data?<\/em><\/strong>
        \nOverfitting occurs when a Machine Learning model works well on its training data but produces poor results when it is applied to other data. Consider the 8 user purchase logs below. Three customers buy a kettle and of those, two buy a French Press. It\u2019s reasonable to suggest a French Press to anyone who buys a kettle. But One customer (Purchase 5) bought a coffee machine and a kettle. If the model recommends a kettle for Purchase 1, it\u2019s likely overfitting based on one observation.<\/p>\n

        \"Figure

        Figure 2- 8 purchase logs from Contoso Coffee transactions.<\/em><\/p><\/div>\n

        Customer Insights uses an automated procedure called cross validation (opens in new tab)<\/span><\/a> to detect overfitting during training. During model training, the algorithm divides the training dataset into 10 subsets. Then, it trains 10 models that each use 9\/10 of the data. Each model is used to predict data in the remaining (1\/10) data, also called the holdout set. If the predictions on the holdout set are accurate then we know we can trust the final model to work on new data in the future. If the algorithm overfits, then Machine Learning engineers will modify the algorithm or its features to generalize more effectively.<\/p>\n

        How do you know what a model uses to make decisions ?<\/strong><\/em>
        \nCameron can clearly explain why they recommend whole bean coffee to grinder buyers. It can be more challenging to understand what feature a Machine Learning model uses to make predictions.<\/p>\n

        Machine Learning engineers talk about feature importance<\/strong> when explaining how a model works. A feature\u2019s importance is the amount of improvement in the model\u2019s accuracy that we gain from including that one feature. So, a feature has no importance if removing it from the data set has no impact on the model\u2019s decisions. A feature has high importance if removing it from the data makes the model significantly worse.<\/p>\n

        <\/h6>\n
        \"Figure

        Figure 3- Feature importance table that is available with Business-to-Business Churn prediction. High impact features increase the risk of a business account churning, with the % impact listed as the importance of that one feature.<\/em><\/p><\/div>\n

        The Customer Insights Business-to-Business Churn model generates detailed information about the importance of features used to generate its predictions. In this example, features like Customer Service support activities were important in indicating high churn, as well as customer traits like what city the customer lives in. Stay tuned for a deeper dive in to feature importance in future blog posts.<\/p>\n

        Machine Learning in Customer Insights<\/strong>
        \nIn Customer Insights, you can use Machine Learning to identify advanced patterns across comprehensive data sets. It\u2019s important to understand how those patterns emerge: algorithms examine historical data to identify patterns that can be used on new data in the future. Customer Insights predictions features are built with industry best practices to ensure that models are accurate and trustworthy. You can help improve the accuracy of the predictions you use them for by providing training data with features that help the model identify trends and patterns for your business.<\/p>\n

        In future posts, we will share deep dives about how we create accurate and trustworthy Machine Learning models in Customer Insights and about how you can help improve the predictions you generate with those models.<\/p>\n","protected":false},"excerpt":{"rendered":"

        Authors: Tommy Guy, Sally Kellaway, Zachary Cook, Julie Koesmarno Microsoft Dynamics 365 Customer Insights accelerates time to value with Machine Learning-based predictions covering Product recommendations, Churn risk, Sentiment analysis and Customer lifetime value scenarios. These features were developed using vast data sets and advanced analytics to provide a comprehensive and timely understanding of customers. The […]<\/p>\n","protected":false},"author":41161,"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-content-parent":804652,"msr_hide_image_in_river":0,"footnotes":""},"research-area":[],"msr-locale":[268875],"msr-post-option":[],"class_list":["post-806446","msr-blog-post","type-msr-blog-post","status-publish","hentry","msr-locale-en_us"],"msr_assoc_parent":{"id":804652,"type":"group"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-blog-post\/806446","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-blog-post"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-blog-post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/41161"}],"version-history":[{"count":6,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-blog-post\/806446\/revisions"}],"predecessor-version":[{"id":823687,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-blog-post\/806446\/revisions\/823687"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=806446"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=806446"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=806446"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=806446"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}