What's beyond query by example?

Over the last ten years, the crucial problem of information retrieval in multimedia documents has boosted research activities in the field of visual appearance indexing and retrieval by content. In the early research years, the concept of the «query by visual example» (QBVE) has been proposed and shown to be relevant for visual information retrieval. It is obvious that QBVE is not able to satisfy the multiple visual search usage requirements. In this paper, we focus on two major approaches that correspond to two different retrieval paradigms. First, we present the partial visual query that ignores the background of the images and allows a straight user expression on its visual interest without relevance feedback mechanism. The second retrieval paradigm consists in searching for the user mental image when no starting visual example is available. query by logical composition of region categories when a visual thesaurus is generated. This new approach relies on the unpervised generation of a visual thesaurus from which query by logical composition of region categories can be performed. This query paradigm is closely related to that of text retrieval. Mental image search is a challenging and promising issue for retrieval by visual content in the forthcoming years since it allows different rich user expression and interaction modes with the search engine.