Fast Hard Negative Mining for Deep Metric Learning

Abstract Deep metric learning methods aim to measure similarity of data points (e.g. images) by calculating their distance in a high dimensional embedding space. These methods are usually trained by optimizing a ranking loss function, which is designed to bring together samples from the same class while separating them from samples from all other classes. The most challenging part of these methods is the selection of samples that contribute to effective network training. In this paper we present Bag of Negatives (BoN), a fast hard negative mining method, that provides a set, triplet or pair of potentially relevant training samples. BoN is an efficient method that selects a bag of hard negatives based on a novel online hashing strategy. We show the superiority of BoN against state-of-the-art hard negative mining methods in terms of accuracy and training time over three large datasets.