Open Access Open Access  Restricted Access Subscription Access

The Research of Quadrotor Flight Control Based on Reinforcement Learning and ADP

Zhongjian Li, Xueyuan Li


This paper studies the application of Lookup-Table reinforcement learning method into the continuous state space control of quadrotor simulator and designs a attitude controller for the quadrotor simulator based on Q-learning; for the improvement of defects concerning difficulty in the learning algorithm’s convergence and low efficiency in learning when Qlearning is faced with large-scale and continuous-space optimized decision, the method of kernel approximate dynamic programming is introduced, Kernel-based Least-Squares Policy Iteration (KLSPI) is proposed, and a controller for the quadrotor simulator is designed based on this algorithm. The experiment shows that the reinforcement learning control method is of fast convergence speed, small steady-state error, strong adaptive ability and good control effect; when dealing with the problem of continuous state space, the Least-Squares Policy Iteration can converge better strategies with fewer training data compared with the traditional method of discretizing state space first.

Full Text:



Yidong Yang. Helicopter Flight Control. National Defence Industry Press


Sentang Wu,Yuhua Fei. Flight Control System. Beijing university of aeronautics and astronautics press.2005

Weilenmann MW, Geering HP. Test bench for rotorcraft hover control. J Guid Control Dyn.1994;17:729–36.

Shim DH, Kim HJ, Sastry S. Decentralized nonlinear model predictive control of multiple flying robots. In: Proc 42nd IEEE conf dec contr, Maui, HI; 2003. p. 3621–6.

AutoCopter express UAV. Cited Aug 2010.

Enns R, Si J. Helicopter flight control design using a learning control approach. In: Proc 39th IEEE conf dec contr, Sydney, Australia; 2000. p. 1754–9.

Wan EA, Bogdanov AA. Model predictive neural control with applications to a 6 DoF helicopter model. In: Proc 2001 American contr

conf, Arlington, VA; 2001. p. 488–93.

Ju Jiang,Mohamed S,Kamel,Pitch Control of an Aircraft with Aggregated Reinforcement Learning Algorithms[J].Proceedings of International Joint Conference on Neural Networks,Orlando,Florida ,USA,2007.

Xin Xu. reinforcement learning and approximate dynamic programming

[M]. Science Press.2010.5

Xin Xu, Dewen Hu, and X.C. Lu., Kernel-Based Least Squares Policy

Iteration for Reinforcement Learning[C]. IEEE Transactions On Neural Networks, Vol. 18, No.4, 2007.



  • There are currently no refbacks.