Differentially private testing for relevant dependencies in high dimensions
By: Patrick Bastian, Holger Dette, Martin Dunsche
Potential Business Impact:
Finds hidden links in private data safely.
We investigate the problem of detecting dependencies between the components of a high-dimensional vector. Our approach advances the existing literature in two important respects. First, we consider the problem under privacy constraints. Second, instead of testing whether the coordinates are pairwise independent, we are interested in determining whether certain pairwise associations between the components (such as all pairwise Kendall's $τ$ coefficients) do not exceed a given threshold in absolute value. Considering hypotheses of this form is motivated by the observation that in the high-dimensional regime, it is rare and perhaps impossible to have a null hypothesis that can be modeled exactly by assuming that all pairwise associations are precisely equal to zero. The formulation of the null hypothesis as a composite hypothesis makes the problem of constructing tests already non-standard in the non-private setting. Additionally, under privacy constraints, state of the art procedures rely on permutation approaches that are rendered invalid under a composite null. We propose a novel bootstrap based methodology that is especially powerful in sparse settings, develop theoretical guarantees under mild assumptions and show that the proposed method enjoys good finite sample properties even in the high privacy regime. Additionally, we present applications in medical data that showcase the applicability of our methodology.
Similar Papers
Practically significant change points in high dimension -- measuring signal strength pro active component
Statistics Theory
Finds changes in data even when it's messy.
Testing independence and conditional independence in high dimensions via coordinatewise Gaussianization
Methodology
Finds hidden connections between data points.
Testing relevant difference in high-dimensional linear regression with applications to detect transferability
Methodology
Helps computers learn from old data for new tasks.