Deep Reinforcement Learning for an Anthropomorphic Robotic Arm Under Sparse Reward Tasks.

Training operation skills of a robotic arm using Reinforcement Learning algorithms requires a process of reward shaping, which needs considerable time to adjust. Instead, the setting of sparse rewards makes the tasks clear and easy to modify when the task changes. However, it is a challenge for the agent to learn directly from sparse reward signals because of the lack of reward guidance. To solve this problem, we propose an algorithm based on the DDPG algorithm and add three techniques: Hindsight Experience Replay for improving sample efficiency, Expert Data Initialization for accelerating learning speed in the early stage, Action Clip Scaling for acting stable. For validating our algorithm, we built a simulation environment of an anthropomorphic robotic arm based on the Pybullet module and set up the training interface. There are two tasks trained under sparse reward signals: Push task, Pick and Place task. The experimental results show that the agent can quickly improve the operation skill level in the early stage. It also has a good convergence effect in the later stage, which effectively solving the sparse rewards problem.