Image semantic segmentation with a novel stochastic model

In this paper, we propose a novel model relying on hierarchical Dirichlet processes (HDP) and hidden conditional random fields (HCRF) for image semantic segmentation. Our model contains middle-level latent variables between high-level labels and low-level visual features, which can be shared by different category instances so making the model flexible. Spatial structure priors are imposed on these latent variables, and the number of available pairwise potential functions is limited to reduce the complexity of the model. We also propose an efficient algorithm, inspired by stochastic gradient descent (SGD), to sample the assignments of discrete latent variables and to learn other model parameters, since no direct optimization algorithms are available for training such a model. Experimental results show the effectiveness of HDP-HCRF and our algorithm on MSRC-21 dataset.