Topological Signatures of ReLU Neural Network Activation Patterns
By: Vicente Bosca , Tatum Rask , Sunia Tanweer and more
Potential Business Impact:
Finds patterns in how computer brains learn.
This paper explores the topological signatures of ReLU neural network activation patterns. We consider feedforward neural networks with ReLU activation functions and analyze the polytope decomposition of the feature space induced by the network. Mainly, we investigate how the Fiedler partition of the dual graph and show that it appears to correlate with the decision boundary -- in the case of binary classification. Additionally, we compute the homology of the cellular decomposition -- in a regression task -- to draw similar patterns in behavior between the training loss and polyhedral cell-count, as the model is trained.
Similar Papers
Discrete Functional Geometry of ReLU Networks via ReLU Transition Graphs
Machine Learning (CS)
Helps computers learn better by understanding their "thinking."
On the failure of ReLU activation for physics-informed machine learning
Machine Learning (CS)
Fixes computer learning for science problems.
The Geometry of ReLU Networks through the ReLU Transition Graph
Machine Learning (CS)
Maps how computer brains learn to work better.