Tuning-Parameter Calibration

Regularized estimators consist of two terms, one for comparing model parameters to data and one for including prior information. The tuning parameters define the weighting: small tuning parameters emphasize the data, while large tuning parameters emphasize the prior information. An optimal tuning parameter balances the data and the prior information such that an estimator’s error for a given task is minimized. Data-driven calibration schemes try to find such optimal tuning parameters in practice.