Preference-Guided Planning: An Active Elicitation Approach

Planning under uncertainty has exploited human (domain) expertise in several different directions. One key research thrust in this direction is that of specifying preferences as advice to the planner in order to reduce the search over the space of plans. While successful, most of these approaches require that the advice be specified before planning. However, humans tend to give the most obvious advice and more importantly, this advice may not directly benefit the planner. We propose a framework in which the planner actively solicits preferences as needed. More specifically, our proposed planning approach computes the uncertainty in the plan explicitly and then queries the human expert for advice. This approach not only removes the burden on the human expert to provide all the advice upfront but also allows the learning algorithm to focus on the most uncertain regions of the plan space and query accordingly. Our results show that this collaborative approach allows for more efficient and effective problem solving compared to the standard planning as well as providing all the preferences in advance. Our framework treats the human input as soft preferences and allows to trade-off between potentially a sub-optimal expert and a complex plan space.