Multi-view RBM with Posterior Consistency and Domain Adaptation

Abstract The restricted Boltzmann machine (RBM) and extensions are rarely used in the field of multi-view learning. In this paper, we first present a multi-view RBM model for classification, which is named RBM with posterior consistency (PCRBM). PCRBM computes multiple representations by regularizing the marginal likelihood function with the consistency among representations from multiple views. However, PCRBM ignores the specific information of each view, where the learned representations just contain the consistency information among multiple views. To address this problem, we propose a novel multi-view RBM model and name it as RBM with posterior consistency and domain adaptation (PDRBM). PDRBM divides the hidden units of each view into two groups: the one is the view-consistency group that contains the consistency information among multiple views, and the other is the view-specific group that contains the information unique to this separate view. In addition, PDRBM balances the relationship among view-consistency hidden representations for multi-view classification. Contrasting with existing multi-view classification methods, PDRBM has achieved satisfactory results on two-class and multi-class classification multi-view datasets.