Challenges for Recommender Systems Evaluation

Many businesses and web portals adopt Recommender Systems (RSs) to help their users to tame information overload and make better choices. Despite the fact that RSs should support user decision making, academic researchers, when evaluating the effectiveness of a RS, largely adopt offline rather than live user studies methods. We discuss the relationships between these evaluation methods by considering a tourism RS case study. We then suggest future directions to be taken by HCI and RS research to better assess the user’s value of RSs.