Simultaneous Planning for Item Picking and Placing by Deep Reinforcement Learning

Container loading by a picking robot is an important challenge in the logistics industry. When designing such a robotic system, item picking and placing have been planned individually thus far. However, since the condition of picking an item affects the possible candidates for placing, it is preferable to plan picking and placing simultaneously. In this paper, we propose a deep reinforcement learning (DRL) method for simultaneously planning item picking and placing. A technical challenge in the simultaneous planning is its scalability: even for a practical container size, DRL can be computationally intractable due to large action spaces. To overcome the intractability, we adopt a fully convolutional network for policy approximation and determine the action based only on local information. This enables us to produce a shared policy which can be applied to larger action spaces than the one used for training. We experimentally demonstrate that our method can successfully solve the simultaneous planning problem and achieve a higher occupancy rate than conventional methods.