Information Theoretic Question Asking to Improve Spatial Semantic Representations

In this paper, we propose an algorithm that enables robots to improve their spatial-semantic representation of the environment by engaging users in dialog. The algorithm aims to reduce the entropy in maps formulated based upon user-provided natural language descriptions (e.g., "The kitchen is down the hallway"). The robot's available information-gathering actions take the form of targeted questions intended to reduce the entropy over the grounding of the user's descriptions. These questions include those that query the robot's local surround (e.g., "Are we in the kitchen?") as well as areas distant from the robot (e.g., "Is the lab near the kitchen?"). Our algorithm treats dialog as an optimization problem that seeks to balance the information-theoretic value of candidate questions with a measure of cost associated with dialog. In this manner, the method determines the best questions to ask based upon expected entropy reduction while accounting for the burden on the user. We evaluate the entropy reduction based upon a joint distribution over a hybrid metric, topological, and semantic representation of the environment learned from user-provided descriptions and the robot's sensor data. We demonstrate that, by asking deliberate questions of the user, the method results in significant improvements in the accuracy of the resulting map.