Information criteria for use under sparsity

This talk addresses the use of information criteria in variable selection for sparse, high-dimensional models. The talk consists of two parts. The first part concentrates on general information criteria in the context of variable selection without shrinkage. Shrinkage can be seen as a mechanism that reduces selection bias due to false positives. Without shrinkage, the observation dependent selection process has an additional biasing effect that is not taken into account by most selection criteria, as those criteria have been designed to assess the quality of a given, fixed model. We describe the selection bias as a so-called mirror effect, by formalizing the intuitive idea that false positives present themselves as better than average candidates for selection, whereas in reality, they are worse than average. Sparsity plays a double role in the analysis: on one hand, the mirror effect is much more important under sparsity. On the other hand, the description of the mirror effect and the resulting corrected information criterion requires a formal notion of sparsity.
Depending on time, the second part of the talk has special attention to the use of generalized cross validation in sparse variable selection. We present two new insights in the working of this specific criterion for use under sparsity. First, we show that the criterion is asymptotically efficient.  The convergence proceeds in probability, but in a somehow odd, rather heavy tailed fashion, which explains that the criterion fails from time to time.  Second, its practical application requires good behavior for nearly full models. The impact of this result on selection with and without shrinkage is explored.

Homepage