Efficient High-Dimensional Conditional Independence Testing

This article deals with the problem of testing conditional independence between two random vectors ${\bf X}$ and ${\bf Y}$ given a confounding random vector ${\bf Z}$. Several authors have considered this problem for multivariate data. However, most of the existing tests has poor performance against local contiguous alternatives beyond linear dependency. In this article, an Energy distance type measure of conditional dependence is developed, borrowing ideas from the model-X framework. A consistent estimator of the measure is proposed, and its theoretical properties are studied under general assumptions. Using the estimator as a test statistic a test of conditional independence is developed, and a suitable resampling algorithm is designed to calibrate the test. The test turns out to be not only large sample consistent, but also Pitman efficient against local contiguous alternatives, and is provably consistent when the dimension of the data diverges to infinity with the sample size. Several empirical studies are conducted to demonstrate the efficacy of the test against state-of-the-art methods.