Controlling the False Discovery Proportion in Matched Observational Studies

We provide an approach to exploratory data analysis in matched observational studies with a single intervention and multiple endpoints. In such settings, the researcher would like to explore evidence for actual treatment effects among these variables while accounting not only for the possibility of false discoveries, but also for the potential impact of unmeasured confounding. For any candidate subset of hypotheses about these outcomes, we provide sensitivity sets for the proportion of the hypotheses within the subset which are actually true. The resulting sensitivity statements are valid simultaneously over all possible choices for the rejected set, allowing the researcher to search for promising subsets of hypotheses that maintain a large estimated fraction of true discoveries even if hidden bias is present. The approach is well suited to sensitivity analysis, as conclusions that some fraction of outcomes are affected by the treatment exhibit larger robustness to unmeasured confounding than findings that any particular outcome is affected. We show how a sequence of integer programs, in tandem with screening steps, facilitate the efficient computation of the required sensitivity sets. We illustrate the practical utility of our method through both simulation studies and a data example on the long-term impacts of childhood abuse.