MAPping the probability of start-up success

Twenty years ago, I fell into the trap of believing that good performance on TREC Ad Hoc could be turned into commercial success via an enterprise search (ES) start-up. Although we achieved a measure of success, the reality was very different from my expectations. It turned out that end-users cared about search relevance only to the extent of reaching for buckets of vitriol when search failed to find what they wanted, and people making purchasing decisions were more interested in other things: what repositories can be included in the search, how responsive search is to updates, what security models are provided, the appearance of the search pages, and achieving internal business goals -- even at the expense of end-user needs. Organizations often purchase search technology as part of another system, such as a content management system, or a records management system, leaving other repositories unsearchable or incompatibly searchable. Causes of enterprise search failure include dimensions studied in IR but an ES system often fails because of the way it is configured, or because it fails to cover information resources that end-users need. The dream of a comprehensive, highly relevant, fully secure single search interface to all of an organization's information is very rarely achieved. There is a huge market potential for a killer ES product but the biggest challenges in exploiting it lie outside the scope of IR research. I look forward to sharing lessons learned from attempting to commercialize IR technology, and new perspectives from my current employment at Bing.