Matrix-Free Two-to-Infinity and One-to-Two Norms Estimation
By: Askar Tsyganov , Evgeny Frolov , Sergey Samsonov and more
Potential Business Impact:
Makes AI smarter and safer from attacks.
In this paper, we propose new randomized algorithms for estimating the two-to-infinity and one-to-two norms in a matrix-free setting, using only matrix-vector multiplications. Our methods are based on appropriate modifications of Hutchinson's diagonal estimator and its Hutch++ version. We provide oracle complexity bounds for both modifications. We further illustrate the practical utility of our algorithms for Jacobian-based regularization in deep neural network training on image classification tasks. We also demonstrate that our methodology can be applied to mitigate the effect of adversarial attacks in the domain of recommender systems.
Similar Papers
Near-optimal Rank Adaptive Inference of High Dimensional Matrices
Information Theory
Finds hidden patterns in messy data.
A general technique for approximating high-dimensional empirical kernel matrices
Machine Learning (Stat)
Makes computer predictions more accurate for complex data.
Statistical Limits for Finite-Rank Tensor Estimation
Information Theory
Find hidden patterns in complex data.