Deep Top Similarity Hashing with Class-wise Loss for Multi-label Image Retrieval

Abstract One of the major challenges of learning to hash in large-scale image retrieval is the projective transformation from raw image to binary space with preserving semantic similarity. Recently, several deep hashing methods show many excellent properties compared with traditional hashing based on hand-designed representation. However, most of the existing hashing models only pay attention to the semantic similarity between image pairs, ignoring the ranking information of retrieval results, which limits its performance. In this paper, a novel deep hashing framework, named Deep Top Similarity Hashing with Class-wise loss (DTSH-CW), is proposed to preserve semantic similarity between top images of ranking list and query images. In this proposed framework, CNNs architecture with batch normalization module is adopted to extract deep semantic characteristics. With integrating the position information of images, a top similarity loss is carefully designed to ensure the similarities between top images of ranking list and query images. Unlike pair-wise or triplet-wise loss, by directly leveraging the class labels, a cubic constraint based on Gaussian distribution is introduced to optimize objective function so as to maintain semantic variations of different classes. Furthermore, in order to solve discrete optimization problem, Two-Stage strategy is developed to provide efficient model training. Quantities of comparison experiments on three multi-label benchmark datasets show that our proposed DTSH-CW achieves promising performance compared to several state-of-the-art hashing methods.