Empirical Orlicz norms

The empirical Orlicz norm based on a random sample is defined as a natural estimator of the Orlicz norm of a univariate probability distribution. A law of large numbers is derived under minimal assumptions. The latter extends readily to a linear and a nonparametric regression model. Secondly, sufficient conditions for a central limit theorem with a standard rate of convergence are supplied. The conditions for the CLT exclude certain canonical examples, such as the empirical sub-Gaussian norm of normally distributed random variables. For the latter, we discover a nonstandard rate of $n^{1/4} \log(n)^{-3/8}$, with a heavy-tailed, stable limit distribution. It is shown that in general, the empirical Orlicz norm does not admit any uniform rate of convergence for the class of distributions with bounded Orlicz norm.