Gaussian process regression consists in predicting a continuous realization of a Gaussian process, given a finite number of observations of it.
When the covariance function of the Gaussian process is known, or when the statistician selects and fix a given covariance function, this prediction is made explicitly thanks to Gaussian conditioning. Thus, most classically, the covariance function is estimated in a first step, and kept fixed to its estimate in a second step, where prediction is carried out ("plug-in approach").
In this presentation, we address parametric estimation, and we consider the Maximum Likelihood and Cross Validation estimators of the covariance parameters. We analyze these two estimators in two cases.
1) Well-specified case where the true covariance function belongs to the parametric set of covariance functions used for estimation. We consider an increasing-domain asymptotic framework, based on a randomly-perturbed regular grid of observation points. We show that both estimators are consistent and asymptotically Gaussian with a square-root-of-n rate of convergence. It is observed that the Maximum Likelihood estimator has a smaller asymptotic variance.
2) Misspecified case where the true covariance function does not belong to the parametric set of covariance functions used for estimation. A finite-sample analysis of the case of the estimation of a single variance parameter is carried-out. It is concluded that, for design of observation points that are not too regular, Cross Validation is more robust than Maximum Likelihood. Empirical results confirm this findings for estimation of multivariate covariance parameters. Finally, based on ongoing work, an increasing-domain asymptotic result supports this conclusion. More precisely, for randomly located observation points, the Cross Validation estimator converges to the covariance parameter minimizing the integrated prediction error.
Homepage of François Bachoc