Score: 1

On Stopping Times of Power-one Sequential Tests: Tight Lower and Upper Bounds

Published: April 28, 2025 | arXiv ID: 2504.19952v1

By: Shubhada Agrawal, Aaditya Ramdas

Potential Business Impact:

Finds best time to stop experiments faster.

Business Areas:
A/B Testing Data and Analytics

We prove two lower bounds for stopping times of sequential tests between general composite nulls and alternatives. The first lower bound is for the setting where the type-1 error level $\alpha$ approaches zero, and equals $\log(1/\alpha)$ divided by a certain infimum KL divergence, termed $\operatorname{KL_{inf}}$. The second lower bound applies to the setting where $\alpha$ is fixed and $\operatorname{KL_{inf}}$ approaches 0 (meaning that the null and alternative sets are not separated) and equals $c \operatorname{KL_{inf}}^{-1} \log \log \operatorname{KL_{inf}}^{-1}$ for a universal constant $c > 0$. We also provide a sufficient condition for matching the upper bounds and show that this condition is met in several special cases. Given past work, these upper and lower bounds are unsurprising in their form; our main contribution is the generality in which they hold, for example, not requiring reference measures or compactness of the classes.

Country of Origin
🇮🇳 🇺🇸 India, United States

Page Count
36 pages

Category
Mathematics:
Statistics Theory