Scalable Gaussian Process Classification With Additive Noise for Non-Gaussian Likelihoods.

Gaussian process classification (GPC) provides a flexible and powerful statistical framework describing joint distributions over function space. Conventional GPCs, however, suffer from: 1) poor scalability for big data due to the full kernel matrix and 2) intractable inference due to the non-Gaussian likelihoods. Hence, various scalable GPCs have been proposed through: 1) the sparse approximation built upon a small inducing set to reduce the time complexity and 2) the approximate inference to derive analytical evidence lower bound (ELBO). However, these scalable GPCs equipped with analytical ELBO are limited to specific likelihoods or additional assumptions. In this work, we present a unifying framework that accommodates scalable GPCs using various likelihoods. Analogous to GP regression (GPR), we introduce additive noises to augment the probability space for: 1) the GPCs with step, (multinomial) probit, and logit likelihoods via the internal variables and 2) particularly, the GPC using softmax likelihood via the noise variables themselves. This leads to unified scalable GPCs with analytical ELBO by using variational inference. Empirically, our GPCs showcase superiority on extensive binary/multiclass classification tasks with up to two million data points.