Stable neural networks and connections to continuous dynamical systems
By: Matthias J. Ehrhardt, Davide Murari, Ferdia Sherry
Potential Business Impact:
Makes computer brains harder to trick.
The existence of instabilities, for example in the form of adversarial examples, has given rise to a highly active area of research concerning itself with understanding and enhancing the stability of neural networks. We focus on a popular branch within this area which draws on connections to continuous dynamical systems and optimal control, giving a bird's eye view of this area. We identify and describe the fundamental concepts that underlie much of the existing work in this area. Following this, we go into more detail on a specific approach to designing stable neural networks, developing the theoretical background and giving a description of how these networks can be implemented. We provide code that implements the approach that can be adapted and extended by the reader. The code further includes a notebook with a fleshed-out toy example on adversarial robustness of image classification that can be run without heavy requirements on the reader's computer. We finish by discussing this toy example so that the reader can interactively follow along on their computer. This work will be included as a chapter of a book on scientific machine learning, which is currently under revision and aimed at students.
Similar Papers
Control of dynamical systems with neural networks
Systems and Control
Teaches computers to control complex machines.
Machine Learning and Control: Foundations, Advances, and Perspectives
Optimization and Control
Explains how AI learns and creates new things.
On the Stability of Neural Networks in Deep Learning
Machine Learning (CS)
Makes computer brains more stable and trustworthy.