Manifold Dimension Estimation: An Empirical Study
By: Zelong Bi, Pierre Lafaye de Micheaux
Potential Business Impact:
Finds hidden patterns in complex information.
The manifold hypothesis suggests that high-dimensional data often lie on or near a low-dimensional manifold. Estimating the dimension of this manifold is essential for leveraging its structure, yet existing work on dimension estimation is fragmented and lacks systematic evaluation. This article provides a comprehensive survey for both researchers and practitioners. We review often-overlooked theoretical foundations and present eight representative estimators. Through controlled experiments, we analyze how individual factors such as noise, curvature, and sample size affect performance. We also compare the estimators on diverse synthetic and real-world datasets, introducing a principled approach to dataset-specific hyperparameter tuning. Our results offer practical guidance and suggest that, for a problem of this generality, simpler methods often perform better.
Similar Papers
Estimation of Local Geometric Structure on Manifolds from Noisy Data
Statistics Theory
Finds the closest point on a hidden shape.
Curvature of high-dimensional data
Statistics Theory
Makes computers measure shapes better in any size.
A Novel Approach for Intrinsic Dimension Estimation
Machine Learning (CS)
Makes big data easier for computers to understand.