{"id":124,"date":"2021-04-05T15:37:37","date_gmt":"2021-04-05T15:37:37","guid":{"rendered":"https:\/\/aitoolkit.test\/pattern\/g2-c-report-system-performance-information\/"},"modified":"2023-10-25T07:30:36","modified_gmt":"2023-10-25T14:30:36","slug":"g2-c-report-system-performance-information","status":"publish","type":"pattern","link":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g2-c-report-system-performance-information\/","title":{"rendered":"Pattern 2C: Report system performance information"},"content":{"rendered":"\n\n<div class=\"toolkit-heading-with-image-block\">\n\t\n\n<h2 class=\"wp-block-heading\">Pattern 2C: Report system performance information<\/h2>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/2023\/05\/blue-header-bar-1536x14-5-23.png\" alt=\"\" \/><\/figure>\n\n<\/div>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"problem\">Problem<\/h4>\n\n\n\n<p>The user needs to form accurate expectations about how well the system can do what it can do.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"solution\">Solution<\/h4>\n\n\n\n<p>Provide grounded information about how well the system can do what it can do.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"use-when\">Use when<\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li>The user needs system performance information to make or justify decisions about adopting the system.<\/li>\n\n\n\n<li>The user needs system performance information to decide how much to rely on the system.<\/li>\n\n\n\n<li>There are established benchmarks for the system type.<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"how\">How<\/h4>\n\n\n\n<p>Collaborate with an AI\/ML practitioner to collect information about:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>What information about system performance is relevant to the user.<\/li>\n\n\n\n<li>How to retrieve system performance indicators such as false positive\/negative rates, accuracy, etc. Performance indicators vary across types of applications.<\/li>\n<\/ul>\n\n\n\n<p>Performance information may cover overall system performance as well as system performance under certain conditions or for certain user groups where the system is suspected to perform less well.<\/p>\n\n\n\n<p>Enable the user to understand practical implications for system performance by clearly relating system performance to system capabilities, and reporting information in units the user understands.\u200b\u200b\u200b\u200b\u200b\u200b\u200b<\/p>\n\n\n\n<p>Performance information may be presented in documentation or as an interactive tool for inspecting model performance.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"user-benefits\">User benefits<\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Enables the user to gain insights into overall system performance.<\/li>\n\n\n\n<li>Enables the user to assess system performance independent of system use.<\/li>\n\n\n\n<li>Enables the user to compare across similar systems.<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"common-pitfalls\">Common pitfalls<\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li>System performance information does not translate into practical implications for system use.<\/li>\n\n\n\n<li>System performance information is not discoverable.<\/li>\n\n\n\n<li>System performance information is limited to overall performance and does not cover specific conditions.<\/li>\n\n\n\n<li>Using this pattern as a substitute for&nbsp;<a href=\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g2-d-provide-low-performance-alerts\/\">G2-D: Provide low performance alerts.<\/a><\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"references\">References<\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li>The&nbsp;<a href=\"http:\/\/gendershades.org\/overview.html\" target=\"_blank\" rel=\"noreferrer noopener\">Gender shades project<\/a>&nbsp;reports performance for specific groups.&nbsp;<\/li>\n\n\n\n<li><a href=\"https:\/\/modelcards.withgoogle.com\/about\" target=\"_blank\" rel=\"noreferrer noopener\">Model cards<\/a>&nbsp;for reporting model performance<\/li>\n\n\n\n<li>Nushi, Besmira, Ece Kamar, and Eric Horvitz. 2018.&nbsp;<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/towards-accountable-ai-hybrid-human-machine-analyses-for-characterizing-system-failure\/\" target=\"_blank\" rel=\"noreferrer noopener\">Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure<\/a>. HCOMP, 126\u201335.<\/li>\n\n\n\n<li>Partnership for AI &#8211;&nbsp;<a href=\"https:\/\/www.partnershiponai.org\/about-ml\/\" target=\"_blank\" rel=\"noreferrer noopener\">About ML:&nbsp; Annotation and Benchmarking on Understanding and Transparency of Machine learning Lifecycles<\/a><\/li>\n<\/ul>\n","protected":false},"menu_order":23,"template":"","meta":{"toolkit_pattern_number":"2C","ep_exclude_from_search":false},"application-type":[68,61,57,96,60,99,56,55,69,29,58,30,59],"goal":[114,126,123],"guideline-term":[4],"product-category":[71,70,23,64,62,26,66,28,65,31,32,63],"class_list":["post-124","pattern","type-pattern","status-publish","hentry","application-type-classification","application-type-facial-recognition","application-type-filtering-and-ranking","application-type-gen-ai","application-type-image-recognition","application-type-llm","application-type-natural-language-processing-text","application-type-natural-language-processing-voice","application-type-prediction","application-type-recommendation","application-type-route-planning","application-type-search","application-type-text-generation","goal-appropriate-reliance","goal-reliability","goal-transparency","guideline-term-make-clear-how-well-the-system-can-do-what-it-can-do","product-category-advertising","product-category-chatbot","product-category-e-commerce","product-category-email","product-category-health-and-wellness","product-category-navigation","product-category-media-entertainment","product-category-productivity","product-category-search-engine","product-category-social-networks","product-category-voice-assistants","product-category-writing-and-editing"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.3 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Pattern 2C: Report system performance information - Microsoft HAX Toolkit<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g2-c-report-system-performance-information\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Pattern 2C: Report system performance information - Microsoft HAX Toolkit\" \/>\n<meta property=\"og:description\" content=\"Problem The user needs to form accurate expectations about how well the system can do what it can do. Solution Provide grounded information about how well the system can do what it can do. Use when How Collaborate with an AI\/ML practitioner to collect information about: Performance information may cover overall system performance as well [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g2-c-report-system-performance-information\/\" \/>\n<meta property=\"og:site_name\" content=\"Microsoft HAX Toolkit\" \/>\n<meta property=\"article:modified_time\" content=\"2023-10-25T14:30:36+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/prod\/2023\/05\/blue-header-bar-1536x14-5-23.png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n\t<meta name=\"twitter:label2\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data2\" content=\"Shipi Dhanorkar\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/pattern\\\/g2-c-report-system-performance-information\\\/\",\"url\":\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/pattern\\\/g2-c-report-system-performance-information\\\/\",\"name\":\"Pattern 2C: Report system performance information - Microsoft HAX Toolkit\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/pattern\\\/g2-c-report-system-performance-information\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/pattern\\\/g2-c-report-system-performance-information\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/uploads\\\/2023\\\/05\\\/blue-header-bar-1536x14-5-23.png\",\"datePublished\":\"2021-04-05T15:37:37+00:00\",\"dateModified\":\"2023-10-25T14:30:36+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/pattern\\\/g2-c-report-system-performance-information\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/pattern\\\/g2-c-report-system-performance-information\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/pattern\\\/g2-c-report-system-performance-information\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/uploads\\\/2023\\\/05\\\/blue-header-bar-1536x14-5-23.png\",\"contentUrl\":\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/uploads\\\/2023\\\/05\\\/blue-header-bar-1536x14-5-23.png\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/pattern\\\/g2-c-report-system-performance-information\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"G2: Make clear how well the system can do what it can do.\",\"item\":\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/guideline\\\/make-clear-how-well-the-system-can-do-what-it-can-do\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Pattern 2C: Report system performance information\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/#website\",\"url\":\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/\",\"name\":\"Microsoft HAX Toolkit\",\"description\":\"Collaborative tools to help you create more effective and responsible human-AI experiences\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/#organization\",\"name\":\"Microsoft Corporation\",\"url\":\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/uploads\\\/2022\\\/07\\\/msft-logo.png\",\"contentUrl\":\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/uploads\\\/2022\\\/07\\\/msft-logo.png\",\"width\":1536,\"height\":1536,\"caption\":\"Microsoft Corporation\"},\"image\":{\"@id\":\"https:\\\/\\\/www.microsoft.com\\\/en-us\\\/haxtoolkit\\\/#\\\/schema\\\/logo\\\/image\\\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Pattern 2C: Report system performance information - Microsoft HAX Toolkit","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g2-c-report-system-performance-information\/","og_locale":"en_US","og_type":"article","og_title":"Pattern 2C: Report system performance information - Microsoft HAX Toolkit","og_description":"Problem The user needs to form accurate expectations about how well the system can do what it can do. Solution Provide grounded information about how well the system can do what it can do. Use when How Collaborate with an AI\/ML practitioner to collect information about: Performance information may cover overall system performance as well [&hellip;]","og_url":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g2-c-report-system-performance-information\/","og_site_name":"Microsoft HAX Toolkit","article_modified_time":"2023-10-25T14:30:36+00:00","og_image":[{"url":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/prod\/2023\/05\/blue-header-bar-1536x14-5-23.png","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes","Written by":"Shipi Dhanorkar"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g2-c-report-system-performance-information\/","url":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g2-c-report-system-performance-information\/","name":"Pattern 2C: Report system performance information - Microsoft HAX Toolkit","isPartOf":{"@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g2-c-report-system-performance-information\/#primaryimage"},"image":{"@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g2-c-report-system-performance-information\/#primaryimage"},"thumbnailUrl":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/2023\/05\/blue-header-bar-1536x14-5-23.png","datePublished":"2021-04-05T15:37:37+00:00","dateModified":"2023-10-25T14:30:36+00:00","breadcrumb":{"@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g2-c-report-system-performance-information\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g2-c-report-system-performance-information\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g2-c-report-system-performance-information\/#primaryimage","url":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/2023\/05\/blue-header-bar-1536x14-5-23.png","contentUrl":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/2023\/05\/blue-header-bar-1536x14-5-23.png"},{"@type":"BreadcrumbList","@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/pattern\/g2-c-report-system-performance-information\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/"},{"@type":"ListItem","position":2,"name":"G2: Make clear how well the system can do what it can do.","item":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/guideline\/make-clear-how-well-the-system-can-do-what-it-can-do\/"},{"@type":"ListItem","position":3,"name":"Pattern 2C: Report system performance information"}]},{"@type":"WebSite","@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/#website","url":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/","name":"Microsoft HAX Toolkit","description":"Collaborative tools to help you create more effective and responsible human-AI experiences","publisher":{"@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/#organization","name":"Microsoft Corporation","url":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/#\/schema\/logo\/image\/","url":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/2022\/07\/msft-logo.png","contentUrl":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/uploads\/2022\/07\/msft-logo.png","width":1536,"height":1536,"caption":"Microsoft Corporation"},"image":{"@id":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/#\/schema\/logo\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/wp-json\/wp\/v2\/pattern\/124","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/wp-json\/wp\/v2\/pattern"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/wp-json\/wp\/v2\/types\/pattern"}],"version-history":[{"count":9,"href":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/wp-json\/wp\/v2\/pattern\/124\/revisions"}],"predecessor-version":[{"id":10950,"href":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/wp-json\/wp\/v2\/pattern\/124\/revisions\/10950"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/wp-json\/wp\/v2\/media?parent=124"}],"wp:term":[{"taxonomy":"application-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/wp-json\/wp\/v2\/application-type?post=124"},{"taxonomy":"goal","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/wp-json\/wp\/v2\/goal?post=124"},{"taxonomy":"guideline-term","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/wp-json\/wp\/v2\/guideline-term?post=124"},{"taxonomy":"product-category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/haxtoolkit\/wp-json\/wp\/v2\/product-category?post=124"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}