Regularity and Stability Properties of Selective SSMs with Discontinuous Gating
By: Nikola Zubić, Davide Scaramuzza
Potential Business Impact:
Makes AI remember things better and more reliably.
Deep Selective State-Space Models (SSMs), characterized by input-dependent, time-varying parameters, offer significant expressive power but pose challenges for stability analysis, especially with discontinuous gating signals. In this paper, we investigate the stability and regularity properties of continuous-time selective SSMs through the lens of passivity and Input-to-State Stability (ISS). We establish that intrinsic energy dissipation guarantees exponential forgetting of past states. Crucially, we prove that the unforced system dynamics possess an underlying minimal quadratic energy function whose defining matrix exhibits robust $\text{AUC}_{\text{loc}}$ regularity, accommodating discontinuous gating. Furthermore, assuming a universal quadratic storage function ensures passivity across all inputs, we derive parametric LMI conditions and kernel constraints that limit gating mechanisms, formalizing "irreversible forgetting" of recurrent models. Finally, we provide sufficient conditions for global ISS, linking uniform local dissipativity to overall system robustness. Our findings offer a rigorous framework for understanding and designing stable and reliable deep selective SSMs.
Similar Papers
Input-Power-to-State Stability of Time-Varying Systems
Systems and Control
Makes systems stable even with big, unpredictable inputs.
Learning to Dissipate Energy in Oscillatory State-Space Models
Machine Learning (CS)
Helps computers remember long stories better.
The Curious Case of In-Training Compression of State Space Models
Machine Learning (CS)
Shrinks computer models during learning for speed.