SIAM Journal on Control and Optimization, Vol.57, No.2, 1094-1126, 2019
STOCHASTIC epsilon-OPTIMAL LINEAR QUADRATIC ADAPTATION: AN ALTERNATING CONTROLS POLICY
This paper presents a continuous time stochastic linear quadratic (LQ) adaptive control algorithm for completely observed linear stochastic systems with unknown parameters. Based on a certainty equivalence approach, we propose to utilize an alternating controls policy, whereby the linear feedback matrix is switched between two -root epsilon-apart distinct matrices K-i, i = 1, 2. The associated adaptive estimation algorithm is designed so that it drives the maximum likelihood based estimate into the sets I(i )i = 1, 2, and consequently into I-1 boolean AND I-2, with I-i corresponding to the true closed loop dynamics under the ith control. A mild geometric assumption is shown to guarantee that I-1 boolean AND I-2 = 0*, the true parameter. This strongly consistent estimation, coupled with the alternating controls policy, then yields epsilon-optimal long-run LQ closed loop performance.
Keywords:adaptive stochastic control;linear quadratic control;persistent excitation;certainty equievalence;Bayesian embedding;alternating controls