Data-driven tracking control approach for linear systems by on-policy Q-learning approach

This paper develops an improved tracking control approach for discrete-time (DT) linear system. It's combining the on-policy Q-learning and an extended state space formulation of systems. The main merit of the proposed approach lies in the optimal tracking controller can be found using the measured augmented variables including input, output and their past values together tracking errors rather than utilizing the system model parameters or state of systems, since they are hard to be available in practice. First, the optimal tracking problem of linear system is formulated as minimizing a specific cost function containing the objective of tracking a reference signal with respect to an extended state space realization. Therefore, we present on-policy Q-learning method, only utilizing the measured data without need of designing state observer, and the optimal tracking problem can be solved, which under consideration can track the reference signal via the optimal approach. Finally, the validity of the simulation results is proved.