Designing explainable cognitive systems and explaining neural networks with plastic dynamical systems

The most powerful Artificial Intelligence (AI) to date is based on neural networks (NNs) and features a critical flaw, the lack of explainability, which undermines trust in its decision-making. Building explainable cognitive machines, as well as analysing the existing ones, is impeded by the lack of unambiguous technical definitions for cognition and its constituents, such as representations of memories and categories, behavioural code, learning and thinking, and for explainability itself. Employing dynamical systems (DS) approach to cognition, we introduce a cohesive, self-consistent, and self-contained mathematical framework for designing an explainable artificial cognitive system to specification, anchored in technical definitions we propose. Our definition of cognition is an interpretation in terms of DS theory of the existing definition from computational neuroscience, which builds upon an earlier idea of cognition as self-organisation of a vector field. This vector field, serving as both the behavioural code and the substrate for memory imprinting, evolves obeying some learning rules. We apply our framework to achieve two main goals. Firstly, we design a “perfect” explainable cognitive system that overcomes common NN issues, such as catastrophic forgetting, spurious memories, lack of lifelong learning, and limited explainability, while incorporating additional functions seen in humans and animals. Secondly, we reveal learning rules for the vector field of a simple NN, and benchmark them against those of the “perfect” system. We use these rules to partly explain how the NN learns, to interpret connection weights, and to highlight the inherent lack of explainability. Our framework fosters deeper comprehension of the existing AI, and paves the way for developing explainable AI with cognitive functions closer to those of biological systems.