Designing for Serendipity in a University Course Recommendation System.

Collaborative filtering based algorithms, including Recurrent Neural Networks (RNN), tend towards predicting a perpetuation of past observed behavior. In a recommendation context, this can lead to an overly narrow set of suggestions lacking in serendipity and inadvertently placing the user in what is known as a "filter bubble." In this paper, we grapple with the issue of the filter bubble in the context of a course recommendation system in production at a public university. Our approach is to present course results that are novel or unexpected to the student but still relevant to their interests. We build one set of models based on course catalog descriptions (BOW) and another set informed by enrollment histories (course2vec). We compare the performance of these models on off-line validation sets and against the system's existing RNN-based recommendation engine in an online user study of undergraduates (N = 70) who rated their course recommendations along six characteristics related to serendipity. Results of the user study show a dramatic lack of novelty in RNN recommendations and depict the characteristic trade-offs that make serendipity difficult to achieve. While the machine learned course2vec models performed best on off-line validation tasks, it was the simple bag-of-words based recommendations that students rated as more serendipitous. We discuss the role of the kind of information presented by the system in a student's decision to accept a recommendation from either algorithm.