Deep high-order supervised hashing

Abstract Recently, deep hashing has achieved excellent performance in large-scale image retrieval by simultaneously learning deep features and hash function. However, state-of-the-art methods for this task have so far failed to explore feature statistics higher than first-order. To address this problem, we propose two novel deep high-order supervised hashing (DHoSH) architectures based on point-wise labels (DHoSH-PO) and pair-wise labels (DHoSH-PA), respectively. The core of DHoSH is a trainable layer of bilinear features incorporated into existing deep convolutional neural network (CNN) for end-to-end learning. This layer captures the interaction of local features by bilinear pooling, using correlation to model dependencies of features within the same layer or cross-correlation for features across different layers of the CNN. Furthermore, DHoSH systematically employs the high-order statistics of features of multiple layers. Extensive experiments on commonly used image retrieval benchmarks show that our DHoSH-PO and DHoSH-PA models yield improved accuracy over its first-order counterparts and achieve effective performance for these benchmarks.