{"id":711,"date":"2021-05-13T20:15:20","date_gmt":"2021-05-13T20:15:20","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/?post_type=pattern&#038;p=711"},"modified":"2023-10-25T07:39:38","modified_gmt":"2023-10-25T14:39:38","slug":"g14-a-comprehensive-updates","status":"publish","type":"pattern","link":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g14-a-comprehensive-updates\/","title":{"rendered":"Pattern 14A: Comprehensive updates"},"content":{"rendered":"\n\n<div class=\"toolkit-heading-with-image-block\">\n\t\n\n<h2 class=\"wp-block-heading\">Pattern 14A: Comprehensive updates<\/h2>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"10\" src=\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/prod\/2023\/05\/green-header-bar-1536x14-5-23-1024x10.png\" alt=\"green header bar\" class=\"wp-image-8814\" srcset=\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/prod\/2023\/05\/green-header-bar-1536x14-5-23-1024x10.png 1024w, https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/prod\/2023\/05\/green-header-bar-1536x14-5-23-300x3.png 300w, https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/prod\/2023\/05\/green-header-bar-1536x14-5-23-768x7.png 768w, https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/prod\/2023\/05\/green-header-bar-1536x14-5-23-1536x14.png 1536w, https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/prod\/2023\/05\/green-header-bar-1536x14-5-23-2048x19.png 2048w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n<\/div>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"problem\">Problem<\/h4>\n\n\n\n<p>The system adapts as it learns over time and it needs to update its outputs without disrupting or disorienting the user.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"solution\">Solution<\/h4>\n\n\n\n<p>The system makes a controlled and deliberate comprehensive update in response to user behaviors or other additional data.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"use-when\">Use when<\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li>The user would benefit from an updated set of outputs.<\/li>\n\n\n\n<li>The model has sufficient additional information for updating.<\/li>\n\n\n\n<li>If updating in response to user behavior: The model is capable of accepting additional information from user interactions, interpreting user intent, and adapting over time.<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"how\">How<\/h4>\n\n\n\n<p>Update system outputs in a way that permeates the whole user experience, not just an isolated part.<\/p>\n\n\n\n<p>Make the update in a controlled and deliberate manner by matching the scale and the rate of the change to user expectations.<\/p>\n\n\n\n<p><strong>Scale<\/strong>: Balance the proportion of new, updated outputs to previous ones.<\/p>\n\n\n\n<p><strong>Rate<\/strong>: Control how quickly the update is reflected in the user experience. For example, the updated outputs can be released gradually or all at once, depending on user expectations and context.<\/p>\n\n\n\n<p>Collaborate with an AI\/ML practitioner to decide how to infer user intent from user behaviors (if applicable) and to adjust the nature, scale, and rate of the update.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"user-benefits\">User benefits<\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Provides a better user experience through improved or new functionality,&nbsp;and\/or personalization.<\/li>\n\n\n\n<li>Provides value to users by improving relevance of content.<\/li>\n\n\n\n<li>Enables the user to discover new relevant content.<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"common-pitfalls\">Common pitfalls<\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li>The system triggers the update based on insufficient or misinterpreted information and mischaracterizes user intent.<\/li>\n\n\n\n<li>The scale and\/or rate of change is mismatched with user expectations.<\/li>\n\n\n\n<li>For example, delivering a very high proportion of updated to previous outputs quickly can be disrupting and disorienting if it floods the user with new outputs and makes it difficult to maintain the previous context.<\/li>\n\n\n\n<li>Conversely, if the user expects the system to have learned something (e.g., a name), a slow and gradual change can be disorienting because the user would be unsure whether the system learned.<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"references\">References<\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Bansal, G., Nushi, B., Kamar, E., Weld, D.S., Lasecki, W.S. and Horvitz, E. 2019.&nbsp;<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/updates-in-human-ai-teams-understanding-and-addressing-the-performance-compatibility-tradeoff\/\" target=\"_blank\" rel=\"noreferrer noopener\">Updates in Human-AI Teams: Understanding and Addressing the Performance\/Compatibility Tradeoff<\/a>.&nbsp;<em>Proceedings of the AAAI Conference on Artificial Intelligence<\/em>. 33, 01 (Jul. 2019), 2429-2437. DOI: https:\/\/doi.org\/10.1609\/aaai.v33i01.33012429.<\/li>\n<\/ul>\n","protected":false},"menu_order":141,"template":"","meta":{"toolkit_pattern_number":"14A","ep_exclude_from_search":false},"application-type":[68,61,57,96,60,99,56,55,69,29,58,30,59],"goal":[120,126],"guideline-term":[16],"product-category":[71,70,23,64,62,26,66,28,65,31,32,63],"class_list":["post-711","pattern","type-pattern","status-publish","hentry","application-type-classification","application-type-facial-recognition","application-type-filtering-and-ranking","application-type-gen-ai","application-type-image-recognition","application-type-llm","application-type-natural-language-processing-text","application-type-natural-language-processing-voice","application-type-prediction","application-type-recommendation","application-type-route-planning","application-type-search","application-type-text-generation","goal-personalization","goal-reliability","guideline-term-update-and-adapt-cautiously","product-category-advertising","product-category-chatbot","product-category-e-commerce","product-category-email","product-category-health-and-wellness","product-category-navigation","product-category-media-entertainment","product-category-productivity","product-category-search-engine","product-category-social-networks","product-category-voice-assistants","product-category-writing-and-editing"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v24.8.1 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Pattern 14A: Comprehensive updates - Microsoft HAX Toolkit<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g14-a-comprehensive-updates\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Pattern 14A: Comprehensive updates - Microsoft HAX Toolkit\" \/>\n<meta property=\"og:description\" content=\"Problem The system adapts as it learns over time and it needs to update its outputs without disrupting or disorienting the user. Solution The system makes a controlled and deliberate comprehensive update in response to user behaviors or other additional data. Use when How Update system outputs in a way that permeates the whole user [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g14-a-comprehensive-updates\/\" \/>\n<meta property=\"og:site_name\" content=\"Microsoft HAX Toolkit\" \/>\n<meta property=\"article:modified_time\" content=\"2023-10-25T14:39:38+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/prod\/2023\/05\/green-header-bar-1536x14-5-23-1024x10.png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n\t<meta name=\"twitter:label2\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data2\" content=\"Shipi Dhanorkar\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g14-a-comprehensive-updates\/\",\"url\":\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g14-a-comprehensive-updates\/\",\"name\":\"Pattern 14A: Comprehensive updates - Microsoft HAX Toolkit\",\"isPartOf\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g14-a-comprehensive-updates\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g14-a-comprehensive-updates\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/prod\/2023\/05\/green-header-bar-1536x14-5-23-1024x10.png\",\"datePublished\":\"2021-05-13T20:15:20+00:00\",\"dateModified\":\"2023-10-25T14:39:38+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g14-a-comprehensive-updates\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g14-a-comprehensive-updates\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g14-a-comprehensive-updates\/#primaryimage\",\"url\":\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/prod\/2023\/05\/green-header-bar-1536x14-5-23-1024x10.png\",\"contentUrl\":\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/prod\/2023\/05\/green-header-bar-1536x14-5-23-1024x10.png\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g14-a-comprehensive-updates\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"G14: Update and adapt cautiously.\",\"item\":\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/guideline\/update-and-adapt-cautiously\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Pattern 14A: Comprehensive updates\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/#website\",\"url\":\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/\",\"name\":\"Microsoft HAX Toolkit\",\"description\":\"Collaborative tools to help you create more effective and responsible human-AI experiences\",\"publisher\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/#organization\",\"name\":\"Microsoft Corporation\",\"url\":\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/prod\/2022\/07\/msft-logo.png\",\"contentUrl\":\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/prod\/2022\/07\/msft-logo.png\",\"width\":1536,\"height\":1536,\"caption\":\"Microsoft Corporation\"},\"image\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/#\/schema\/logo\/image\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Pattern 14A: Comprehensive updates - Microsoft HAX Toolkit","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g14-a-comprehensive-updates\/","og_locale":"en_US","og_type":"article","og_title":"Pattern 14A: Comprehensive updates - Microsoft HAX Toolkit","og_description":"Problem The system adapts as it learns over time and it needs to update its outputs without disrupting or disorienting the user. Solution The system makes a controlled and deliberate comprehensive update in response to user behaviors or other additional data. Use when How Update system outputs in a way that permeates the whole user [&hellip;]","og_url":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g14-a-comprehensive-updates\/","og_site_name":"Microsoft HAX Toolkit","article_modified_time":"2023-10-25T14:39:38+00:00","og_image":[{"url":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/prod\/2023\/05\/green-header-bar-1536x14-5-23-1024x10.png","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes","Written by":"Shipi Dhanorkar"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g14-a-comprehensive-updates\/","url":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g14-a-comprehensive-updates\/","name":"Pattern 14A: Comprehensive updates - Microsoft HAX Toolkit","isPartOf":{"@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g14-a-comprehensive-updates\/#primaryimage"},"image":{"@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g14-a-comprehensive-updates\/#primaryimage"},"thumbnailUrl":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/prod\/2023\/05\/green-header-bar-1536x14-5-23-1024x10.png","datePublished":"2021-05-13T20:15:20+00:00","dateModified":"2023-10-25T14:39:38+00:00","breadcrumb":{"@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g14-a-comprehensive-updates\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g14-a-comprehensive-updates\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g14-a-comprehensive-updates\/#primaryimage","url":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/prod\/2023\/05\/green-header-bar-1536x14-5-23-1024x10.png","contentUrl":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/prod\/2023\/05\/green-header-bar-1536x14-5-23-1024x10.png"},{"@type":"BreadcrumbList","@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g14-a-comprehensive-updates\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/"},{"@type":"ListItem","position":2,"name":"G14: Update and adapt cautiously.","item":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/guideline\/update-and-adapt-cautiously\/"},{"@type":"ListItem","position":3,"name":"Pattern 14A: Comprehensive updates"}]},{"@type":"WebSite","@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/#website","url":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/","name":"Microsoft HAX Toolkit","description":"Collaborative tools to help you create more effective and responsible human-AI experiences","publisher":{"@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/#organization","name":"Microsoft Corporation","url":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/#\/schema\/logo\/image\/","url":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/prod\/2022\/07\/msft-logo.png","contentUrl":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/prod\/2022\/07\/msft-logo.png","width":1536,"height":1536,"caption":"Microsoft Corporation"},"image":{"@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/#\/schema\/logo\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/wp-json\/wp\/v2\/pattern\/711","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/wp-json\/wp\/v2\/pattern"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/wp-json\/wp\/v2\/types\/pattern"}],"version-history":[{"count":10,"href":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/wp-json\/wp\/v2\/pattern\/711\/revisions"}],"predecessor-version":[{"id":10986,"href":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/wp-json\/wp\/v2\/pattern\/711\/revisions\/10986"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/wp-json\/wp\/v2\/media?parent=711"}],"wp:term":[{"taxonomy":"application-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/wp-json\/wp\/v2\/application-type?post=711"},{"taxonomy":"goal","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/wp-json\/wp\/v2\/goal?post=711"},{"taxonomy":"guideline-term","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/wp-json\/wp\/v2\/guideline-term?post=711"},{"taxonomy":"product-category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/wp-json\/wp\/v2\/product-category?post=711"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}