{"id":841159,"date":"2022-04-30T09:14:44","date_gmt":"2022-04-30T16:14:44","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-research-item&p=841159"},"modified":"2022-05-05T07:06:03","modified_gmt":"2022-05-05T14:06:03","slug":"neurocompositional-computing-in-human-and-machine-intelligence-a-tutorial","status":"publish","type":"msr-research-item","link":"https:\/\/www.microsoft.com\/en-us\/research\/publication\/neurocompositional-computing-in-human-and-machine-intelligence-a-tutorial\/","title":{"rendered":"Neurocompositional computing in human and machine intelligence: A tutorial"},"content":{"rendered":"

The past decade has produced a revolution in Artificial Intelligence (AI), after a half-century
\nof AI repeatedly failing to meet expectations. What explains the dramatic
\nchange from 20th-century to 21st-century AI, and how can remaining limitations of
\ncurrent AI be overcome?<\/p>\n

Until now, the widely accepted narrative has attributed the recent progress in AI to
\ntechnical engineering advances that have yielded massive increases in the quantity of
\ncomputational resources and training data available to support statistical learning in
\ndeep artificial neural networks. Although these quantitative engineering innovations
\nare important, here we show that the latest advances in AI are not solely due to
\nquantitative<\/em>\u00a0increases in computing power but also\u00a0qualitative<\/em>\u00a0changes in how that
\ncomputing power is deployed. These qualitative changes have brought about a new
\ntype of computing that we call\u00a0neurocompositional computing<\/strong>.<\/p>\n

In neurocompositional computing, neural networks exploit two scientific principles
\nthat contemporary theory in cognitive science maintains are simultaneously
\nnecessary to enable human-level cognition. The Compositionality Principle asserts
\nthat encodings of complex information are structures that are systematically composed
\nfrom simpler structured encodings. The Continuity Principle states that the encoding
\nand processing of information is formalized with real numbers that vary continuously.
\nThese principles have seemed irreconcilable until the recent mathematical discovery
\nthat compositionality can be realized not only through the traditional discrete
\nmethods of symbolic computing, well developed in 20th-century AI, but also through
\nnovel forms of continuous neural computing\u2014neurocompositional computing.<\/p>\n

The unprecedented progress of 21st-century AI has resulted from the use of
\nlimited\u2014first-generation\u2014forms of neurocompositional computing. We show that
\nthe new techniques now being deployed in second-generation neurocompositional
\ncomputing create AI systems that are not only more robust and accurate than current
\nsystems, but also more comprehensible\u2014making it possible to diagnose errors in, and
\nexert human control over, artificial neural networks through interpretation of their
\ninternal states and direct intervention upon those states.<\/p>\n

Note:<\/em>\u00a0This tutorial is intended for those new to this topic, and does not assume
\nfamiliarity with cognitive science, AI, or deep learning. Appendices provide more
\nadvanced material. Each figure, and the associated box explaining it, provides an
\nexposition, illustration, or further details of a main point of the paper; in order to
\nmake these figures relatively self-contained, it has sometimes been necessary to repeat
\nsome material from the text. For a brief introduction and additional development of
\nsome of this material see \u201cNeurocompositional computing: From the central paradox of cognition
\nto a new generation of ai systems\u201d (arXiv: (opens in new tab)<\/span><\/a>2205.01128; to appear, AI Magazine<\/em>)<\/p>\n","protected":false},"excerpt":{"rendered":"

The past decade has produced a revolution in Artificial Intelligence (AI), after a half-century of AI repeatedly failing to meet expectations. What explains the dramatic change from 20th-century to 21st-century AI, and how can remaining limitations of current AI be overcome? Until now, the widely accepted narrative has attributed the recent progress in AI to […]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"footnotes":""},"msr-content-type":[3],"msr-research-highlight":[],"research-area":[13556,13545],"msr-publication-type":[193718],"msr-product-type":[],"msr-focus-area":[],"msr-platform":[],"msr-download-source":[],"msr-locale":[268875],"msr-field-of-study":[246694,246805],"msr-conference":[],"msr-journal":[],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-841159","msr-research-item","type-msr-research-item","status-publish","hentry","msr-research-area-artificial-intelligence","msr-research-area-human-language-technologies","msr-locale-en_us","msr-field-of-study-artificial-intelligence","msr-field-of-study-natural-language"],"msr_publishername":"","msr_edition":"","msr_affiliation":"","msr_published_date":"2022-5-5","msr_host":"","msr_duration":"","msr_version":"","msr_speaker":"","msr_other_contributors":"","msr_booktitle":"","msr_pages_string":"","msr_chapter":"","msr_isbn":"","msr_journal":"","msr_volume":"","msr_number":"MSR-TR-2022-5","msr_editors":"","msr_series":"","msr_issue":"","msr_organization":"Microsoft","msr_how_published":"","msr_notes":"52 pages main text, 78 pages total, 11 figures, 2 Appendices, 239 references. For a short presentation of some of this material, see https:\/\/arxiv.org\/abs\/2205.01128 (to appear in AI Magazine).","msr_highlight_text":"","msr_release_tracker_id":"","msr_original_fields_of_study":"","msr_download_urls":"","msr_external_url":"","msr_secondary_video_url":"","msr_longbiography":"","msr_microsoftintellectualproperty":1,"msr_main_download":"","msr_publicationurl":"","msr_doi":"","msr_publication_uploader":"","msr_related_uploader":[{"type":"file","viewUrl":"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/04\/Neurocompositional_computing__tutorial.pdf","id":"842353","title":"neurocompositional_computing__tutorial","label_id":"243118","label":0}],"msr_attachments":[{"id":842353,"url":"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2022\/05\/Neurocompositional_computing__tutorial.pdf"}],"msr-author-ordering":[{"type":"user_nicename","value":"Paul Smolensky","user_id":36353,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Paul Smolensky"},{"type":"text","value":"R. Thomas McCoy","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Roland Fernandez","user_id":33386,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Roland Fernandez"},{"type":"text","value":"Matthew Goldrick","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Jianfeng Gao","user_id":32246,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Jianfeng Gao"}],"msr_impact_theme":[],"msr_research_lab":[199565],"msr_event":[],"msr_group":[144931],"msr_project":[811039],"publication":[],"video":[],"download":[],"msr_publication_type":"techreport","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/841159"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-research-item"}],"version-history":[{"count":3,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/841159\/revisions"}],"predecessor-version":[{"id":842356,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/841159\/revisions\/842356"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=841159"}],"wp:term":[{"taxonomy":"msr-content-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-content-type?post=841159"},{"taxonomy":"msr-research-highlight","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-highlight?post=841159"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=841159"},{"taxonomy":"msr-publication-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-type?post=841159"},{"taxonomy":"msr-product-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-product-type?post=841159"},{"taxonomy":"msr-focus-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-focus-area?post=841159"},{"taxonomy":"msr-platform","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-platform?post=841159"},{"taxonomy":"msr-download-source","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-download-source?post=841159"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=841159"},{"taxonomy":"msr-field-of-study","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-field-of-study?post=841159"},{"taxonomy":"msr-conference","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-conference?post=841159"},{"taxonomy":"msr-journal","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-journal?post=841159"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=841159"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=841159"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}