Regret Bounds For Robust Adaptive Control Of The Linear Quadratic Regulator

Authors:
Sarah Dean
Horia Mania UC Berkeley
Nikolai Matni UC Berkeley
Benjamin Recht UC Berkeley
Stephen Tu UC Berkeley

Introduction:

The authors consider adaptive control of the Linear Quadratic Regulator (LQR), where anunknown linear system is controlled subject to quadratic costs.Leveraging recentdevelopments in the estimation of linear systems and in robust controller synthesis,the authors present the first provably polynomial time algorithm that achieves sub-linearregret on this problem.

Abstract:

We consider adaptive control of the Linear Quadratic Regulator (LQR), where anunknown linear system is controlled subject to quadratic costs. Leveraging recentdevelopments in the estimation of linear systems and in robust controller synthesis,we present the first provably polynomial time algorithm that achieves sub-linearregret on this problem. We further study the interplay between regret minimizationand parameter estimation by proving a lower bound on the expected regret interms of the exploration schedule used by any algorithm. Finally, we conduct anumerical study comparing our robust adaptive algorithm to other methods fromthe adaptive LQR literature, and demonstrate the flexibility of our proposed methodby extending it to a demand forecasting problem subject to state constraints.

You may want to know: