Open Access Open Access  Restricted Access Subscription Access

Local Minima Consistent Identification Method to Accelerate Multiagent Reinforcement Learning

Safreni Candra Sari, Widyawardana Adiprawita

Abstract


This paper presents a novel method in accelerating Multiagent Reinforcement Learning called Local Minima Consisten Identification or  LMCI. Incorporating LMCI in Multiagent Reinforcement Learning Algorithms will successfully accelerate the learning convergence. This method scales down the state space iteratively by distinguishing insignificant states from the significant one and then eliminating them while learning, which aggressively reduces the scale of the state space in the following learning episodes. This method is generally applicable for varying Multiagent Reinforcement Learning algorithms such as Multiagent Q() and Multiagent SARSA() in order to solve multiagent task challenges or general multiagent learning with large scale state space characteristic.

Full Text:

PDF

References


Sutton, R.S. and A.G. Barto, Reinforcement Learning: An Introduction 1998: MIT Press.

Bianchi, R.A.C., et al., Heuristically-Accelerated Multiagent Reinforcement Learning. Cybernetics, IEEE Transactions on, 2013. PP(99): p. 1-1.

Kaelbling, L.P., M.L. Littman, and A.W. Moore, Reinforcement learning: a survey. J. Artif. Int. Res., 1996. 4(1): p. 237-285.

Littman, M.L. Markov games as a framework for multi-agent reinforcement learning. in THE ELEVENTH INTERNATIONAL CONFERENCE ON MACHINE LEARNING. 1994. Morgan Kaufmann.

Busoniu, L., B.D. Schutter, and R. Babuska. Multiagent reinforcement learning with adaptive state focus. in 17th Belgian-Dutch Conference on Artificial Intelligence (BNAIC-05). 2005. Brussels, Belgium.

Claus, C. and C. Boutilier. The dynamics of reinforcement learning in cooperative multiagent systems. in National Conference on Artificial Intelligence (AAAI-98). 1998.

Watkins, C.J.C.H. and P. Dayan, Q-Learning. Machine Learning, 1992. 8(3-4): p. 279--292.

Schuitema, E., Reinforcement Learning on Autonomous Humanoid Robots, 2012.

Mataric, M.J. Reward Functions for Accelerated Learning. in ICML. 1994.

Konidaris, G. and A. Barto. Autonomous shaping: Knowledge transfer in

reinforcement learning. in Proceedings of the 23rd international conference on Machine learning. 2006. ACM.

Matignon, L., G.J. Laurent, and N. Le Fort-Piat, Reward function and initial values: better choices for accelerated goal-directed reinforcement learning, in Artificial Neural Networks–ICANN 20062006, Springer. p. 840-849.

Ma, X., et al., State-chain sequential feedback reinforcement learning for path planning of autonomous mobile robots. Journal of Zhejiang University Science C, 2013. 14(3): p. 167-178.

Drummond, C., Accelerating reinforcement learning by composing solutions of automatically identified subtasks. Journal of Artificial Intelligence Research (JAIR), 2002. 16: p. 59-104.

Taylor, M.E. and P. Stone. Speeding up reinforcement learning with behavior transfer. in AAAI 2004 Fall Symposium on Real-life Reinforcement Learning. 2004.

Celiberto, L.A., et al. Using transfer learning to speed-up reinforcement learning: a cased-based approach. in Robotics Symposium and Intelligent Robotic Meeting (LARS), 2010 Latin American. 2010. IEEE.

Norouzzadeh, S., L. Busoniu, and R. Babuska. Efficient Knowledge Transfer in Shaping Reinforcement Learning. in Proceedings of the 18th IFAC World Congress. 2011.

Takano, T., et al., TRANSFER LEARNING BASED ON FORBIDDEN RULE SET IN ACTOR-CRITIC METHOD. INTERNATIONAL JOURNAL OF

INNOVATIVE COMPUTING INFORMATION AND CONTROL, 2011. 7(5 B): p.

-2917.

Potapov, A. and M. Ali, Convergence of reinforcement learning algorithms and acceleration of learning. Physical Review E, 2003. 67(2): p. 026706.

Gao, Y. and F. Toni, Argumentation Accelerated Reinforcement Learning for RoboCup Keepaway-Takeaway, in Theory and Applications of Formal Argumentation, E. Black, S. Modgil, and N. Oren, Editors. 2014, Springer Berlin Heidelberg. p. 79-94.




DOI: http://dx.doi.org/10.21535%2FProICIUS.2015.v11.679

Refbacks

  • There are currently no refbacks.