Statistical Inference for Differentially Private Stochastic Gradient Descent
By: Xintao Xia, Linjun Zhang, Zhanrui Cai
Potential Business Impact:
Makes private data safe for computer learning.
Privacy preservation in machine learning, particularly through Differentially Private Stochastic Gradient Descent (DP-SGD), is critical for sensitive data analysis. However, existing statistical inference methods for SGD predominantly focus on cyclic subsampling, while DP-SGD requires randomized subsampling. This paper first bridges this gap by establishing the asymptotic properties of SGD under the randomized rule and extending these results to DP-SGD. For the output of DP-SGD, we show that the asymptotic variance decomposes into statistical, sampling, and privacy-induced components. Two methods are proposed for constructing valid confidence intervals: the plug-in method and the random scaling method. We also perform extensive numerical analysis, which shows that the proposed confidence intervals achieve nominal coverage rates while maintaining privacy.
Similar Papers
Rényi Differential Privacy for Heavy-Tailed SDEs via Fractional Poincaré Inequalities
Machine Learning (Stat)
Makes private computer learning work better.
Almost Sure Convergence Analysis of Differentially Private Stochastic Gradient Methods
Machine Learning (CS)
Makes private AI learn better and more reliably.
Online differentially private inference in stochastic gradient descent
Methodology
Keeps your personal data private while learning.