Optimized Formation Control Using Simplified Reinforcement Learning for a Class of Multi-Agent Systems with Unknown Dynamics

The paper proposes an optimized leader-follower formation control using a simplified reinforcement learning (RL) of identifier-critic-actor architecture for a class of nonlinear multi-agent systems. In general, optimal control is expected to be obtained by solving Hamilton-Jacobi-Bellman (HJB) equation, but the equation associated with a nonlinear system is solved difficultly by analytical method. Although the difficulty can be effectively overcame by the RL strategy, the existing RL algorithms are very complex because their updating laws are obtained by carrying out gradient descent algorithm to square of the approximated HJB equation (Bellman residual error). To multi-agent system, due to the state coupling problem, these methods will become difficult implementing. In the proposed optimized scheme, the RL updating laws are derived from negative gradient of the approximated HJB equation, therefore the control algorithm is significantly simple. Furthermore, in order to solve the problem of unknown system dynamics, an adaptive identifier is integrated into the control. Finally, the theory and simulation demonstrate that the optimized formation scheme can guarantee the desired control performance.