Negative binomial regression and inference using a pre-trained transformer
By: Valentine Svensson
Potential Business Impact:
Finds patterns in data much faster.
Negative binomial regression is essential for analyzing over-dispersed count data in in comparative studies, but parameter estimation becomes computationally challenging in large screens requiring millions of comparisons. We investigate using a pre-trained transformer to produce estimates of negative binomial regression parameters from observed count data, trained through synthetic data generation to learn to invert the process of generating counts from parameters. The transformer method achieved better parameter accuracy than maximum likelihood optimization while being 20 times faster. However, comparisons unexpectedly revealed that method of moment estimates performed as well as maximum likelihood optimization in accuracy, while being 1,000 times faster and producing better-calibrated and more powerful tests, making it the most efficient solution for this application.
Similar Papers
Negative binomial models for development triangles of counts
Methodology
Counts future insurance claims more accurately.
Flexible model for varying levels of zeros and outliers in count data
Methodology
Better counts for tricky data with many zeros.
Towards Theoretical Understanding of Transformer Test-Time Computing: Investigation on In-Context Linear Regression
Machine Learning (CS)
Makes computer writing smarter by trying many ideas.