A Fog Robotic System for Dynamic Visual Servoing

Cloud Robotics is a paradigm where multiple robots are connected to cloud services via Internet to access “unlimited” computation power, at the cost of network communication. However, due to limitations such as network latency and variability, it is difficult to control dynamic, human compliant service robots directly from the cloud. In this work, we combine cloud robotics with an agile edge device to build a Fog Robotic system by leveraging an asynchronous protocol with a “heartbeat” signal. We use the system to enable robust teleoperation of a dynamic self-balancing robot from the cloud. We use the system to pick up boxes from static locations, a task commonly performed in warehouse logistics. To make cloud teleoperation more intuitive and efficient, we program a cloud-based image based visual servoing (IBVS) module to automatically assist the cloud teleoperator during the object pickups. Visual feedbacks, including apriltag recognition and tracking, are performed in the cloud to emulate a Fog Robotic object recognition system for IBVS. We demonstrate the feasibility of a dynamic real-time automation system using this cloud-edge hybrid design, which opens up possibilities of deploying dynamic robotic control with deep-learning recognition systems in Fog Robotics. Finally, we show that Fog Robotics enables the self-balancing service robot to pick up a box automatically from a person under unstructured environments.