Pareto Solutions vs Dataset Optima: Concepts and Methods for Optimizing Competing Objectives with Constraints in Retrieval

When ranking search results for multiple objectives, such as maximizing the relevance and diversity of retrieved documents, competing objectives can induce a space of optimal solutions, each reflecting a different optimal trade-off over objectives. This raises several important questions. Firstly, what Pareto optimal solution set is induced by objective functions? Secondly, what are the best solutions achievable on a given dataset? Finally, how closely do the best dataset solutions approach the true Pareto optimal? We present a clear conceptual framing of these questions, with supporting terminology and visualizations, distinguishing Pareto vs. "dataset-best" solutions and providing strong intuition about how and why different optimization problems lead to different shapes and forms of solutions (regardless of optimization technique). We also provide benchmark problems for verifying the correctness of any Pareto machinery and show how existing multi-objective optimization (MOO) and filter methods can be used to provide accurate and interpretable answers to the above questions. Finally, we show how user-defined constraints imposed on the solution space can be effectively handled. In sum, we provide conceptual, translational, and practical contributions to solving MOO problems in retrieval.