Balancing exploration and exploitation: An image-based approach to item retrieval with enhanced diversity

Abstract We propose a machine learning approach based on relevance feedback to visually retrieve e-commerce items, represented by images, from huge catalogs. A threefold strategy is presented to deal with the exploration–exploitation dilemma: retrieve user-relevant items; retrieve items that do not comply with user learned preferences, to avoid convergence to local minima; and retrieve items from uncertainty regions, to improve the learning curve, all in a stochastic manner. To mitigate the cold start problem, we present a novel sampling method to improve the diversity of retrieved items that employs a combination of a multidimensional projection method with an adaptive spatial data structure. We performed quantitative and qualitative user experiments using an annotated catalog of clothing items. Results revealed that our approach can rapidly improve the retrieval of appropriate items in a few user iterations while providing higher diversity than other relevant and recent approaches.