Conformal novelty detection with false discovery rate control at the boundary
By: Zijun Gao, Etienne Roquain, Daniel Xiang
Potential Business Impact:
Finds unusual things without being fooled.
Conformal novelty detection is a classical machine learning task for which uncertainty quantification is essential for providing reliable results. Recent work has shown that the BH procedure applied to conformal p-values controls the false discovery rate (FDR). Unfortunately, the BH procedure can lead to over-optimistic assessments near the rejection threshold, with an increase of false discoveries at the margin as pointed out by Soloff et al. (2024). This issue is solved therein by the support line (SL) correction, which is proven to control the boundary false discovery rate (bFDR) in the independent, non-conformal setting. The present work extends the SL method to the conformal setting: first, we show that the SL procedure can violate the bFDR control in this specific setting. Second, we propose several alternatives that provably control the bFDR in the conformal setting. Finally, numerical experiments with both synthetic and real data support our theoretical findings and show the relevance of the new proposed procedures.
Similar Papers
Full-conformal novelty detection: A powerful and non-random approach
Methodology
Finds unusual things in data, even if data changes.
Conformal novelty detection for replicate point patterns with FDR or FWER control
Methodology
Tests find more real results, fewer fake ones.
Selective Labeling with False Discovery Rate Control
Machine Learning (CS)
Makes AI labels trustworthy for important tasks.