Understanding Overparametrization in Survival Models through Double-Descent
By: Yin Liu, Jianwen Cai, Didong Li
Classical statistical learning theory predicts a U-shaped relationship between test loss and model capacity, driven by the bias-variance trade-off. Recent advances in modern machine learning have revealed a more complex pattern, double-descent, in which test loss, after peaking near the interpolation threshold, decreases again as model capacity continues to grow. While this behavior has been extensively analyzed in regression and classification, its manifestation in survival analysis remains unexplored. This study investigates double-descent in four representative survival models: DeepSurv, PC-Hazard, Nnet-Survival, and N-MTLR. We rigorously define interpolation and finite-norm interpolation, two key characteristics of loss-based models to understand double-descent. We then show the existence (or absence) of (finite-norm) interpolation of all four models. Our findings clarify how likelihood-based losses and model implementation jointly determine the feasibility of interpolation and show that overfitting should not be regarded as benign for survival models. All theoretical results are supported by numerical experiments that highlight the distinct generalization behaviors of survival models.
Similar Papers
Double Descent and Overparameterization in Particle Physics Data
High Energy Physics - Experiment
Makes computer models better at guessing physics results.
The Double Descent Behavior in Two Layer Neural Network for Binary Classification
Machine Learning (Stat)
Finds a sweet spot for computer learning accuracy.
A dynamic view of some anomalous phenomena in SGD
Optimization and Control
Helps computers learn better by finding hidden patterns.