Learning linear parameter-varying control of small-scale helicopter using episodic natural actor-critic method

In this paper, we present the application of episodic natural actor-critic (eNAC) method to learn an LPV controller for the longitudinal motion of a small-scale helicopter. Rewards are online constructed using the quadratic error between the real output and desired output from reference model. The control law is given in a gain-scheduled PID form to accommodate nonlinear longitudinal dynamics. Natural gradients are computed while interacting with the environment, and scaling technique is applied to the gradient components associated with the scheduling parameters to improve the convergence rate of the learning process. To handle the partial observability of the small helicopter's dynamics, notch filter is connected in tandem to the longitudinal cyclic in advance. When performing cruise flight in nonlinear simulation, the learned LPV controller shows better performance in the respect of reference tracking, compared to the conventional PID controller, demonstrating the effectiveness of our method.