Deep Multi-Similarity Hashing with semantic-aware preservation for multi-label image retrieval

Recently, many deep supervised hashing has been developed for multi-label image retrieval applications and has already achieved good effects. However, current methods quantify the similarities between image pairs by counting the number of shared labels roughly, totally ignoring the semantic relationships of image pairs. Besides, the sample imbalance of different classes could bias the training process toward majority categories, deteriorating the model performance. To solve the above-mentioned problems, we propose a novel deep supervised hashing framework, named Deep Multi-Similarity Hashing with semantic-aware preserving (DMSH), to generate binary codes with high-level semantic similarity. Specifically, to obtain distinguishing features, Convolutional Neural Networks (CNN) with correlation operators are introduced to capture discriminative semantic features of the input images, including auto-correlation characteristics and cross-correlation characteristics. By jointly exploiting label-level and semantic-level similarity, the semantic-aware similarity module is developed to quantify the high-level semantic similarity of image pairs, distinguishing the differences beyond categories. Furthermore, the multi-similarity loss is incorporated into deep hashing to collect informative pairs efficiently, which could alleviate the impact of imbalanced sample pairs during model training. Extensive comparison experiments on three benchmark datasets demonstrate that our DMSH achieves promising performance with respect to different evaluation metrics.