Neuro-Vector-Symbolic Architecture

Neither deep neural networks nor symbolic artificial intelligence (AI) alone has approached the kind of intelligence expressed in humans and animal behavior. Why? Symbolic AI focuses on solving cognitive problems, drawing upon the rich framework of symbolic computation to manipulate internal representations to perform reasoning and inference. But it lacks the ability to learn from example or by direct observation of the world. Deep neural nets, on the other hand, have the outstanding ability to learn from data. Without the richness of symbolic computation, neural nets lack the simple but powerful operations such as variable binding that allow for analogy making and reasoning. Based on our earlier experiences, we approach the problem from a very different perspective, inspired by the brain’s high-dimensional circuits and the unique mathematical properties of high-dimensional spaces. It leads us to neuro-vector-symbolic architecture (NeuroVSA), a cognitive architecture that combines the strengths of symbolic AI and neural nets, and yet has novel emergent properties of its own. By combining a small set of basic operations on high-dimensional vectors, we obtain a mathematical system that makes it possible to represent and manipulate data in ways that allow for reasoning and inference, and to learn internal representations from the statistics of data. NeuroVSA provides a novel look at data representation, associated operations, interactions, and more importantly in-memory computing substrates that naturally enable them.

In NeuroVSA, processing takes place with vectors where information is superposed rather than occupying non-overlapping fields. Learning is incremental and fast, and the results can be traced back to their cause; in other words, computing is transparent. A computing system based on NeuroVSA is remarkably robust and highly energy-efficient, resembling the efficiency of information processing in mammalian brains. NeuroVSA heavily leverages advances in post-CMOS electronic devices in particular the analog phase-change memory devices.