WAIC and WBIC for mixture models

In Bayesian statistical inference, an unknown probability distribution is estimated from a sample using a statistical model and a prior. In general, a pair of a model and a prior may or may not be appropriate for unknown distribution; some evaluation procedures of the estimated result are necessary. If a statistical model is regular and the likelihood function can be approximated by some Gaussian function, then AIC and BIC can be applied to such evaluation processes. However, if a statistical model contains hierarchical structure or latent variables, then regularity condition is not satisfied. The information criteria WAIC and WBIC are devised so as to estimate the generalization loss and the free energy, respectively, even if the posterior distribution is far from any normal distribution and even if the unknown true distribution is not realizable by a statistical model. In this paper, we introduce mathematical foundation and computing methods of WAIC and WBIC in a normal mixture which is a typically singular statistical model, and discuss their properties in statistical inference. Also, we study the case that samples are not independently and identically distributed, for example, they are conditional independent or exchangeable.