Sequential Test for Practical Significance: Truncated Mixture Sequential Probability Ratio Test
By: Kyu Min Shim
Potential Business Impact:
Finds important results, not tiny ones.
We present a sequential testing method to identify a practically significant effect. We build on the existing mixture sequential probability ratio test (mSPRT) that can sequentially test for a non-zero treatment effect by using a truncated mixing distribution to differentiate between effects that are large enough to merit a real world action versus that are non-zero but too small to merit a real world action. We verify the Type-I error control of our method theoretically and empirically. We also extend this idea to sequentially test for one-sided practical significance such as non-inferiority testing, and show that we may still effectively control the Type-I error rate.
Similar Papers
A Simple Geometric Proof of the Optimality of the Sequential Probability Ratio Test for Symmetric Bernoulli Hypotheses
Statistics Theory
Find coin bias faster with fewer flips.
An Efficient Adaptive Sequential Procedure for Simple Hypotheses with Expression for Finite Number of Applications of Less Effective Treatment
Statistics Theory
Tests treatments, using less of the bad one.
Sequential One-Sided Hypothesis Testing of Markov Chains
Applications
Tests if data follows a pattern.