Monday 27.03.2017
Valid conditional inference has become a topic of increasing concern. Recently, significant research has been focused on how to compute appropriate p-values for inference post model selection. We address a slightly different problem: how can hypothesis testing be validly used to select a model? We want to use hypothesis testing to select one of the models identified by forward stepwise regression. This is a challenging task because the hypotheses being tested are suggested by the data and subsequent tests are only made if previous tests are rejected. Addressing the differences between these two challenges requires increased precision about the quantity of interest when using hypothesis testing for model selection. Our solution uses a sequential testing framework and demonstrates that multiple comparison methods can be adapted to this task. We also provide a flexible and practical algorithm, Revisiting Alpha-Investing (RAI), which yields a fast approximation to forward stepwise, performing model selection in O(nplog(n)) time while controlling the marginal false discovery rate.
Homepage of Kory D. Johnson