Neurocompositional computing in human and machine intelligence: A tutorial

The past decade has produced a revolution in Artificial Intelligence (AI), after a half-century
of AI repeatedly failing to meet expectations. What explains the dramatic
change from 20th-century to 21st-century AI, and how can remaining limitations of
current AI be overcome?

Until now, the widely accepted narrative has attributed the recent progress in AI to
technical engineering advances that have yielded massive increases in the quantity of
computational resources and training data available to support statistical learning in
deep artificial neural networks. Although these quantitative engineering innovations
are important, here we show that the latest advances in AI are not solely due to
quantitative increases in computing power but also qualitative changes in how that
computing power is deployed. These qualitative changes have brought about a new
type of computing that we call neurocompositional computing.

In neurocompositional computing, neural networks exploit two scientific principles
that contemporary theory in cognitive science maintains are simultaneously
necessary to enable human-level cognition. The Compositionality Principle asserts
that encodings of complex information are structures that are systematically composed
from simpler structured encodings. The Continuity Principle states that the encoding
and processing of information is formalized with real numbers that vary continuously.
These principles have seemed irreconcilable until the recent mathematical discovery
that compositionality can be realized not only through the traditional discrete
methods of symbolic computing, well developed in 20th-century AI, but also through
novel forms of continuous neural computing—neurocompositional computing.

The unprecedented progress of 21st-century AI has resulted from the use of
limited—first-generation—forms of neurocompositional computing. We show that
the new techniques now being deployed in second-generation neurocompositional
computing create AI systems that are not only more robust and accurate than current
systems, but also more comprehensible—making it possible to diagnose errors in, and
exert human control over, artificial neural networks through interpretation of their
internal states and direct intervention upon those states.

Note: This tutorial is intended for those new to this topic, and does not assume
familiarity with cognitive science, AI, or deep learning. Appendices provide more
advanced material. Each figure, and the associated box explaining it, provides an
exposition, illustration, or further details of a main point of the paper; in order to
make these figures relatively self-contained, it has sometimes been necessary to repeat
some material from the text. For a brief introduction and additional development of
some of this material see “Neurocompositional computing: From the central paradox of cognition
to a new generation of ai systems” (arXiv: (opens in new tab)2205.01128; to appear, AI Magazine)