Supervised Autonomy Interface with Integrated Perception and Motion Planning for Telemanipulation Tasks

In scenarios where the environment and task are known and certain, autonomous robotic manipulation can be performed. However, when an unforeseen situation arises where prior information is unknown, timely human assistance becomes very useful in accomplishing the task. In this paper, we propose a supervisory autonomy system that allows a user to assist the robot in uncertain task scenarios by specifying, through an intuitive user interface, the discrete actions (e.g. grasping) to be performed on the selected objects. The robot has partial knowledge of the environment in that it is able to segment and localize objects. This information is shared with the user to allow easy object and action specification, after which the robot executes the action on the object autonomously while avoiding obstacles. We investigated the system using live perception of real objects with simulated robot and environment.