{"id":979650,"date":"2023-10-26T00:51:47","date_gmt":"2023-10-26T07:51:47","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-project&#038;p=979650"},"modified":"2023-10-26T00:51:49","modified_gmt":"2023-10-26T07:51:49","slug":"science-of-ai","status":"publish","type":"msr-project","link":"https:\/\/www.microsoft.com\/en-us\/research\/project\/science-of-ai\/","title":{"rendered":"Science of AI"},"content":{"rendered":"<section class=\"mb-3 moray-highlight\">\n\t<div class=\"card-img-overlay mx-lg-0\">\n\t\t<div class=\"card-background  has-background-purple card-background--full-bleed\">\n\t\t\t\t\t<\/div>\n\t\t<!-- Foreground -->\n\t\t<div class=\"card-foreground d-flex mt-md-n5 my-lg-5 px-g px-lg-0\">\n\t\t\t<!-- Container -->\n\t\t\t<div class=\"container d-flex mt-md-n5 my-lg-5 \">\n\t\t\t\t<!-- Card wrapper -->\n\t\t\t\t<div class=\"w-100 w-lg-col-5\">\n\t\t\t\t\t<!-- Card -->\n\t\t\t\t\t<div class=\"card material-md-card py-5 px-md-5\">\n\t\t\t\t\t\t<div class=\"card-body \">\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\n<h1 class=\"wp-block-heading\" id=\"science-of-ai\">Science of AI<\/h1>\n\n\n\n<p>Design, analysis and interpretability of large language models<\/p>\n\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t<\/div>\n\t\t<\/div>\n\t<\/div>\n<\/section>\n\n\n\n\n\n<p>Transformers and large language models (LLMs) have had enormous success in recent years. Yet they remain poorly understood, in particular why and how they work. We are trying to answer such questions using tools such as mathematical analysis and mechanistic interpretability. One area where these models perform poorly is Continual Learning. We are working on alternate solutions, for example based on models such as biological neural networks. <\/p>\n\n\n","protected":false},"excerpt":{"rendered":"<p>Design, analysis and interpretability of large language models Transformers and large language models (LLMs) have had enormous success in recent years. Yet they remain poorly understood, in particular why and how they work. We are trying to answer such questions using tools such as mathematical analysis and mechanistic interpretability. One area where these models perform [&hellip;]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"footnotes":""},"research-area":[13561],"msr-locale":[268875],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-979650","msr-project","type-msr-project","status-publish","hentry","msr-research-area-algorithms","msr-locale-en_us","msr-archive-status-active"],"msr_project_start":"","related-publications":[],"related-downloads":[],"related-videos":[],"related-groups":[],"related-events":[],"related-opportunities":[],"related-posts":[],"related-articles":[],"tab-content":[],"slides":[],"related-researchers":[{"type":"user_nicename","display_name":"Amit Deshpande","user_id":30988,"people_section":"Reseachers","alias":"amitdesh"},{"type":"user_nicename","display_name":"Ankit Garg","user_id":36107,"people_section":"Reseachers","alias":"garga"},{"type":"user_nicename","display_name":"Navin Goyal","user_id":33063,"people_section":"Reseachers","alias":"navingo"},{"type":"user_nicename","display_name":"Neeraj Kayal","user_id":33076,"people_section":"Reseachers","alias":"neeraka"},{"type":"user_nicename","display_name":"Satya Lokam","user_id":33532,"people_section":"Reseachers","alias":"satya"},{"type":"guest","display_name":"Pritam  Chandra","user_id":979698,"people_section":"Research Fellows","alias":""},{"type":"guest","display_name":"Satvik Golechha","user_id":979704,"people_section":"Research Fellows","alias":""},{"type":"guest","display_name":"Tanmay Sinha","user_id":979701,"people_section":"Research Fellows","alias":""}],"msr_research_lab":[199562],"msr_impact_theme":[],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/979650"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-project"}],"version-history":[{"count":4,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/979650\/revisions"}],"predecessor-version":[{"id":985002,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/979650\/revisions\/985002"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=979650"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=979650"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=979650"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=979650"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=979650"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}