A Spin Glass Characterization of Neural Networks
By: Jun Li
Potential Business Impact:
Understand how computer brains learn and work.
This work presents a statistical mechanics characterization of neural networks, motivated by the replica symmetry breaking (RSB) phenomenon in spin glasses. A Hopfield-type spin glass model is constructed from a given feedforward neural network (FNN). Overlaps between simulated replica samples serve as a characteristic descriptor of the FNN. The connection between the spin-glass description and commonly studied properties of the FNN -- such as data fitting, capacity, generalization, and robustness -- has been investigated and empirically demonstrated. Unlike prior analytical studies that focus on model ensembles, this method provides a computable descriptor for individual network instances, which reveals nontrivial structural properties that are not captured by conventional metrics such as loss or accuracy. Preliminary results suggests its potential for practical applications such as model inspection, safety verification, and detection of hidden vulnerabilities.
Similar Papers
Learning About Learning: A Physics Path from Spin Glasses to Artificial Intelligence
Disordered Systems and Neural Networks
Teaches physics students AI using old ideas.
Amorphous Solid Model of Vectorial Hopfield Neural Networks
Disordered Systems and Neural Networks
Stores more memories like a solid.
Probabilistic Computing Optimization of Complex Spin-Glass Topologies
Disordered Systems and Neural Networks
Solves hard puzzles faster using special computer bits.