Iteratively reweighted kernel machines efficiently learn sparse functions
By: Libin Zhu , Damek Davis , Dmitriy Drusvyatskiy and more
Potential Business Impact:
Finds important patterns in data automatically.
The impressive practical performance of neural networks is often attributed to their ability to learn low-dimensional data representations and hierarchical structure directly from data. In this work, we argue that these two phenomena are not unique to neural networks, and can be elicited from classical kernel methods. Namely, we show that the derivative of the kernel predictor can detect the influential coordinates with low sample complexity. Moreover, by iteratively using the derivatives to reweight the data and retrain kernel machines, one is able to efficiently learn hierarchical polynomials with finite leap complexity. Numerical experiments illustrate the developed theory.
Similar Papers
Data-Efficient Kernel Methods for Learning Differential Equations and Their Solution Operators: Algorithms and Error Analysis
Machine Learning (Stat)
Teaches computers to solve math problems faster.
Learning functions, operators and dynamical systems with kernels
Machine Learning (CS)
Teaches computers to learn from data.
A Compositional Kernel Model for Feature Learning
Machine Learning (CS)
Finds important information, ignores junk.