WeightFlow: Learning Stochastic Dynamics via Evolving Weight of Neural Network
By: Ruikun Li , Jiazhen Liu , Huandong Wang and more
Potential Business Impact:
Helps computers understand how things change over time.
Modeling stochastic dynamics from discrete observations is a key interdisciplinary challenge. Existing methods often fail to estimate the continuous evolution of probability densities from trajectories or face the curse of dimensionality. To address these limitations, we presents a novel paradigm: modeling dynamics directly in the weight space of a neural network by projecting the evolving probability distribution. We first theoretically establish the connection between dynamic optimal transport in measure space and an equivalent energy functional in weight space. Subsequently, we design WeightFlow, which constructs the neural network weights into a graph and learns its evolution via a graph controlled differential equation. Experiments on interdisciplinary datasets demonstrate that WeightFlow improves performance by an average of 43.02\% over state-of-the-art methods, providing an effective and scalable solution for modeling high-dimensional stochastic dynamics.
Similar Papers
Flows and Diffusions on the Neural Manifold
Machine Learning (CS)
Makes AI learn better and find bad AI behavior.
Zero-Shot Forecasting of Network Dynamics through Weight Flow Matching
Computational Engineering, Finance, and Science
Predicts how things spread online, even new ones.
Geometric Flow Models over Neural Network Weights
Machine Learning (CS)
Makes AI learn new tasks with less data.