{"id":967926,"date":"2023-09-20T09:45:35","date_gmt":"2023-09-20T16:45:35","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=967926"},"modified":"2023-09-20T09:45:37","modified_gmt":"2023-09-20T16:45:37","slug":"neural-graphical-models","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/neural-graphical-models\/","title":{"rendered":"Neural Graphical Models"},"content":{"rendered":"\n

This research paper was presented at the 17th<\/sup> European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty (opens in new tab)<\/span><\/a>, a premier forum for advances in the theory and practice of reasoning under uncertainty.<\/em><\/strong><\/p>\n\n\n\n

\"ECSQARU<\/figure>\n\n\n\n

In the field of reasoning under uncertainty, probabilistic graphical models (PGMs) stand out as a powerful tool for analyzing data. They can represent relationships between features and learn underlying distributions that model functional dependencies between them. Learning, inference, and sampling are operations that make graphical models useful for domain exploration.  <\/p>\n\n\n\n

In a broad sense, learning involves fitting the distribution function parameters from data, and inference is the procedure of answering queries in the form of conditional distributions with one or more observed variables. Sampling entails the ability to extract samples from the underlying distribution as defined by the graphical model. A common challenge with graphical model representations lies in the high computational complexity of one or more of these operations.   <\/p>\n\n\n\n

Various graphical models impose restrictions on the set of distributions or types of variables in the domain. Some graphical models work with continuous variables only (or categorical variables only) or place restrictions on the graph structure, for example, the constraint that continuous variables cannot be parents of categorical variables in a directed acyclic graph (DAG). Other restrictions affect the set of distributions the models can represent, for example, only multivariate Gaussian distributions.<\/p>\n\n\n\n

<\/div>\n\n\n\n\t
\n\t\t\n\n\t\n\t
\n\t\t\t\t\t\t
\n\t\t\t\t\n\t\t\t\t\t\"Digital\n\t\t\t\t<\/a>\n\t\t\t<\/div>\n\t\t\t\n\t\t\t
\n\n\t\t\t\t\t\t\t\t\t

GigaPath: Whole-Slide Foundation Model for Digital Pathology<\/h2>\n\t\t\t\t\n\t\t\t\t\t\t\t\t

Digital pathology helps decode tumor microenvironments for precision immunotherapy. In joint work with Providence and UW, we\u2019re sharing Prov-GigaPath, the first whole-slide pathology foundation model, for advancing clinical research.<\/p>\n\t\t\t\t\n\t\t\t\t\t\t\t\t

\n\t\t\t\t\t
\n\t\t\t\t\t\t\n\t\t\t\t\t\t\tRead more\t\t\t\t\t\t<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t<\/div>\n\t<\/div>\n\t\n\n\n

In our paper, \u201cNeural Graphical Models (opens in new tab)<\/span><\/a>,\u201d presented at ECSQARU 2023 (opens in new tab)<\/span><\/a>, we propose Neural Graphical Models (NGMs), a new type of PGM that learns to represent the probability function over the domain using a deep neural network. The parameterization of such a network can be learned from data efficiently, with a loss function that jointly optimizes adherence to the dependency structure, given as input in the form of a directed or undirected graph, and fit to the data. Probability functions represented by NGMs are unrestricted by any of the common restrictions inherent in other PGMs. NGMs can handle various input types: categorical, continuous, images and embedding representations. They also support efficient inference and sampling.<\/p>\n\n\n\n

\"Figure
Figure 1: Graphical view of NGMs: The input graph G (undirected) for given input data X. Each feature \\( x_i=f_i(\\text{Nbrs}(x_i))\\) is a function of the neighboring features. For a DAG, the functions between features will be defined by the Markov Blanket relationship \\( x_i=f_i(\\text{MB}(x_i))\\). On the right, the adjacency matrix represents the associated dependency structure S.<\/figcaption><\/figure>\n\n\n\n
Figure 2: Neural view of NGMs: This is a neural network as a multitask learning architecture capturing nonlinear dependencies for the features of the undirected graph in Figure 1. The presence of a path from the input to the output features indicates a dependency between them. The dependency matrix between the input and output of the NN reduces to matrix product operation \\(S_{nn}=\\Pi_i|W_i|=|W_1|\\times|W_2|\\). Note that not all the zeroed-out weights of the MLP (in black-dashed lines) are shown for the sake of clarity.<\/figcaption><\/figure>\n\n\n\n

Experimental validations for NGMs<\/h2>\n\n\n\n

In our paper (opens in new tab)<\/span><\/a>, we evaluate NGMs\u2019 performance, inference accuracy, sensitivity to the input graph, and ability to recover the input dependency structure when trained on both real and synthetic data: Infant mortality data (opens in new tab)<\/span><\/a> from the Centers for Disease Control and Prevention (CDC), synthetic Gaussian Graphical model data, and lung cancer data from Kaggle. <\/p>\n\n\n\n

The infant mortality dataset (opens in new tab)<\/span><\/a> describes pregnancy and birth variables for all live births in the US and, in instances of infant death before the first birthday, the cause of death. We used the latest available data, which includes information about 3,988,733 live births in the US during 2015. It was particularly challenging to evaluate the inference accuracy of NGMs using this dataset due to the (thankfully) rare occurrence of infant deaths during the first year of life, making queries concerning such low probability events hard to accurately estimate.  <\/p>\n\n\n\n

We used the CDC data to evaluate the NGMs\u2019 inference accuracy. We compared their prediction for four variables of various types: gestational age (ordinal, expressed in weeks), birth weight (continuous, specified in grams), survival until the first birthday (binary) and the cause of death. We used the categories of \u201calive,\u201d the 10 most common causes of death, or \u201cother\u201d for the less common causes. Here, “alive” was indicated for 99.48% of infants. We also compared the performance of logistic regression, Bayesian networks, Explainable Boosting Machines (EBM), and NGMs. In case of NGMs, we trained two models: one using the Bayesian network graph and one using the uGLAD<\/a> graph.<\/p>\n\n\n\n

Our results demonstrate that NGM are significantly more accurate than logistic regression, more accurate than Bayesian networks, and on par with EBM models for categorical and ordinal variables. They particularly shine when predicting very low probability categories for multi-valued variable cause of death, where, in contrast most models (such as both PGMs and classification models) typically struggle. Note that while we need to train a separate LR and EBM model for each outcome variable evaluated, all variables can be predicted within one trained NGM model. Interestingly, the two NGM models show similar accuracy results despite the differences in the two dependency structures used in training. <\/p>\n\n\n\n

We believe that NGMs are an interesting amalgam of the deep learning architectures’ expressivity, and PGMs\u2019 representation capabilities and can be applied in many domains, given that they place no restrictions on input types and distributions. We encourage you to explore NGMs and take advantage of the ability to work with a wider range of distributions and inputs. You can access the code for Neural Graphical Models on GitHub (opens in new tab)<\/span><\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

This research paper was presented at the 17th European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty (opens in new tab), a premier forum for advances in the theory and practice of reasoning under uncertainty. In the field of reasoning under uncertainty, probabilistic graphical models (PGMs) stand out as a powerful tool for […]<\/p>\n","protected":false},"author":42735,"featured_media":968850,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"footnotes":""},"categories":[1],"tags":[],"research-area":[13556],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[264846],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-967926","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-research-blog","msr-research-area-artificial-intelligence","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[849856],"msr_impact_theme":["Computing foundations"],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[],"related-projects":[],"related-events":[],"related-researchers":[{"type":"user_nicename","value":"Harsh Shrivastava","user_id":41299,"display_name":"Harsh Shrivastava","author_link":"Harsh Shrivastava<\/a>","is_active":false,"last_first":"Shrivastava, Harsh","people_section":0,"alias":"hshrivastava"},{"type":"user_nicename","value":"Urszula Chajewska","user_id":38853,"display_name":"Urszula Chajewska","author_link":"Urszula Chajewska<\/a>","is_active":false,"last_first":"Chajewska, Urszula","people_section":0,"alias":"urszc"}],"msr_type":"Post","featured_image_thumbnail":"\"ECSQARU","byline":"Harsh Shrivastava<\/a> and Urszula Chajewska<\/a>","formattedDate":"September 20, 2023","formattedExcerpt":"This research paper was presented at the 17th European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty (opens in new tab), a premier forum for advances in the theory and practice of reasoning under uncertainty. In the field of reasoning under uncertainty, probabilistic…","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/967926"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/42735"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=967926"}],"version-history":[{"count":32,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/967926\/revisions"}],"predecessor-version":[{"id":969291,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/967926\/revisions\/969291"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/968850"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=967926"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=967926"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=967926"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=967926"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=967926"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=967926"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=967926"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=967926"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=967926"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=967926"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=967926"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}