臺灣博碩士論文加值系統

RoboCup(原名Robot World Cup Initiative)是一個國際性的教育研究組織，旨在透過一個標準的問題－機器人世界盃足球賽(RoboCup World Championship and Conference，簡稱RoboCup)來促進人工智慧與機器人科學領域的發展。在機器人科學－機器學習的議題中，有效率的行為學習方法是重要的研究課題。對於智慧型系統的設計而言，讓機器人在面對動態的環境中，能夠經過學習的過程而做出最佳的回應動作，將是極具挑戰性的問題。本文以一種混合式的技術架構，以規則庫的方式來呈現機器人的知識背景，利用模糊推論的方式來控制機器人高階的行為動作，並結合基因演算法將模糊規則逐步最佳化以適應動態的環境，達成機器人行為演化的目的。最後於RoboCup模擬平台上(RoboCup Soccer Simulator)實作驗證，使足球機器人針對高階的行為動作具備學習演進的能力，並在系統的執行速度與機器學習的效率上做分析與探討。

RoboCup is an international joint project to promote AI, robotics, and related fields. It is an attempt to foster artificial intelligence and robotics research by providing a standard problem where a wide range of technologies can be integrated and examined. The problem of an effective behavior learning of autonomous robots is one of the most important tasks of the modern robotics. In fact, it is well known that the learning to optimize actions of autonomous agents in a dynamic environment is one of the most complex challenges of the intelligent system design. In this paper, we propose a hybrid approach integrating fuzzy logic system with genetic algorithm for high-level skills learning of robots within the RoboCup simulation soccer domain. Through the experiments, we found that the proposed method has good property of computation efficiency and also has a good advantage applied to the environment of RoboCup.

摘 要..................................i
ABSTRACT..............................ii
誌 謝................................iii
目 錄.................................iv
表目錄................................vi
圖目錄...............................vii
第一章 緒論............................1
1.1 前言..............................1
1.2 研究動機與目的....................1
1.3 研究貢獻..........................2
1.4 章節編排..........................2
第二章 文獻探討........................4
2.1 代理人............................4
2.1.1 何謂代理人.....................4
2.1.2 代理人的特性...................5
2.1.3 代理人的優點...................6
2.1.4 代理人的應用...................6
2.2 模糊系統..........................7
2.2.1 模糊系統的架構.................7
2.2.2 模糊系統的特點.................8
2.2.3 模糊系統的應用.................9
2.3 基因演算法........................9
2.3.1 基因演算法之流程..............10
2.3.2 基因演算法之應用..............13
第三章 基因模糊系統...................14
3.1 混合型的系統架構.................14
3.2 模糊控制的方法...................15
3.3 基因編碼.........................16
3.3.1 歸屬函數的基因編碼形式........16
3.3.2 模糊規則的基因編碼形式........19
3.4 基因演算法的操作設定.............21
3.4.1 個體選擇......................21
3.4.2 交配方式......................21
3.4.3 突變方式......................22
3.4.4 基因修補策略..................22
3.4.5 精英策略......................23
3.5 基因模糊系統的演進步驟...........24
第四章 RoboCup案例研究................26
4.1 RoboCup模擬環境架構..............26
4.2 基因模糊系統設計與實作...........28
4.3 實驗設計與驗證...................32
4.3.1 實驗一........................32
4.3.2 實驗二........................38
4.3.3 實驗三........................45
4.4 相關研究比較.....................47
第五章 結論與未來展望.................48
參考文獻..............................49

[1] J. Peters and S. Schaal, "Reinforcement Learning for Operational Space Control," in Proceedings of IEEE International Conference on Robotics and Automation, pp. 2111-2116, Roma, 2007.
[2] H. Q. Min, J. A. Zeng, J. Chen and J. H. Zhu, "A Study of Reinforcement Learning in a New Multiagent Domain," in Proceedings of IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, Vol. 2, pp. 154-161, Sydney, NSW, 2008.
[3] J. Y. Kuo, M. L. Tsai, and N. L. Hsueh, “Goal Evolution based on Adaptive Q-learning for Intelligent Agent”, IEEE International Conference on Systems, Man and Cybernetics, Taiwan, 2006.
[4] A. Rao, and M. Georgeff, “BDI Agents: From Theory to Practice”, International Conference on Multi-Agent Systems, 1995.
[5] L. Vig, J. A. Adams, “Multi-robot coalition formation”, IEEE Transactions on Robotics, Vol. 22, Issue 4, pp. 637-649, 2006.
[6] L. A. Zadeh, “Outline of a new approach to the analysis of complex system and decision processed”, IEEE Transactions on Systems, Man and Cybernetics, vol. 3, no. 1, pp. 28-44, 1973.
[7] E. H. Mamdani, “Application of Fuzzy Logic to Approximate Reasoning Using Linguistic Synthesis”, IEEE Transactions on Computers, Vol. C-26, no. 12, pp. 1182-1191, 1977.
[8] T. Nakashima, M. Takatani, H. Ishibuchi and M. Nii, "The Effect of Using Match History on the Evolution of RoboCup Soccer Team Strategies," in Proceedings of IEEE Symposium on Computational Intelligence and Games, pp. 60-66, Reno, NV, 2006.
[9] H. Okada and T. Takagi, "Evaluation of Multi-objective Genetic Algorithm for RoboCupSoccer Team Evolution," in Proceedings of SICE Annual Conference, pp. 151-154, Tokyo, 2008.
[10] G. Narvydas, R. Simutis and V. Raudonis, "Autonomous Mobile Robot Control Using Fuzzy Logic and Genetic Algorithm," in Proceedings of 4th IEEE Workshop on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications, pp. 460-464, Dortmund, 2007.
[11] E. A. Fares, M. Elbardiny and M. M. Sharaf, "Adaptive Fuzzy Logic Controller of Visual Servoing Robot System by membership optimization using Genetic Algorithms," in Proceedings of IEEE International Conference on Computer Engineering and Systems, pp. 15-20, Cairo, 2007.
[12] S. Wu, Q. Li, E. Zhu, J. Xie and G. Zhichao, "Fuzzy Controller of Pipeline Robot Navigation Optimized by Genetic Algorithm," in Proceedings of IEEE Conference on Control and Decision, pp. 904-908, Yantai, Shandong, 2008.
[13] I. Kallel, N. Baklouti and A. M. Alimi, "Accuracy Preserving Interpretability with Hybrid Hierarchical Genetic Fuzzy Modeling: Case of Motion Planning Robot Controller," in Proceedings of IEEE International Symposium on Evolving Fuzzy Systems, pp. 312-317, Ambleside, 2006.
[14] A. A. Eftekharian and H. Sayyaadi, "Design of Mixed Fuzzy-GA Controller for SCARA Type Robot," in Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2173-2178, Beijing, 2006.
[15] M. Mozafari, A. M. Fard, V. Salmani and M. Naghibzadeh, "An Improved Fuzzy Mechanism for 3D Soccer Simulation Agent''s Shoot Skill," in Proceedings of Annual IEEE India Conference, pp. 1-6, New Delhi, 2006.
[16] N. G. Aghaei, H. Shahbazi, P. Farzaneh, A. Abdolmaleki and A. Khorsandian, "The structure of personality-based emotional decision making in robotic rescue agent," in Proceedings of IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pp. 1201-1206, Xian, 2008.
[17] V. Salmani, A. M. Fard and M. Naghibzadeh, "A Fuzzy Two-Phase Decision Making Approach for Simulated Soccer Agent," in Proceedings of IEEE International Conference on Engineering of Intelligent Systems, pp. 1-6, Islamabad, 2006.
[18] N. Iqbal and R. Kamran, "Learning Strategies Based on Fuzzy Set Rules for the Ideal Opponent Model," in Proceedings of ICET International Conference on Emerging Technologies, pp. 199-204, Islamabad, 2007.
[19] J. Peng, M. Liu, J. Liu, K. C. Lin and M. Wu, "A coordination model using fuzzy reinforcement learning for multi-agent system," in Proceedings of CTS International Symposium on Collaborative Technologies and Systems, pp. 129-136, Orlando, FL, 2007.
[20] K. Miao and M. Q. Li, "Role Assignment Strategy for Heterogeneous Robots Based on Fuzzy Evaluation And Fuzzy Inference," in Proceedings of ICARCV 9th International Conference on Control, Automation, Robotics and Vision, pp. 1-5, Singapore, 2006.
[21] C. Kavaklioglu and O. Kaynak, "A Type-2 Fuzzy Planner with Semi Qualitative World Model for Robocup Domain," in Proceedings of IEEE International Conference on Automation Science and Engineering, pp. 795-799, Scottsdale, AZ, 2007.
[22] Z. Liu, K. Chen and L. Liu, "A Uniform Gait Generation Method for RoboCup Humanoid Robot Based on Genetic Algorithms," in Proceedings of IEEE International Conference on Mechatronics and Automation, pp. 1620-1625, Luoyang, Henan, 2006.
[23] T. J. Juan, R. R. Carlos and R. Jorge, "Automatic Behavior Generation in a Multi-Agent System through Evolutionary Programming," in Proceedings of IEEE 3rd Latin American Robotics Symposium, pp. 2-9, Santiago, 2006.
[24] W. R. Plant, G. Schaefer and T. Nakashima, "An Overview of Genetic Algorithms in Simulation Soccer," in Proceedings of IEEE Congress on Evolutionary Computation, pp. 3897-3904, Hong Kong, 2008.
[25] R. Ozawa, Y. Mae and M. Minami, "Localization of Mobile Robots by Multiple Landmark Recognition," in Proceedings of SICE Annual Conference, pp. 1529-1535, Takamatsu, 2007.
[26] D. Bruijnen, W. Aangenent, J. van Helvoort and R. van de Molengraft, "From Vision to Realtime Motion Control for the RoboCup Domain," in Proceedings of IEEE International Conference on Control Applications, pp. 545-550, Singapore, 2007.
[27] H. Burchardt and R. Salomon, "Implementation of Path Planning using Genetic Algorithms on Mobile Robots," in Proceedings of IEEE Congress on Evolutionary Computation, pp. 1831-1836, Vancouver, BC, 2006.
[28] S. Piao and L. Sun, "Robot Action Acquisition by Self-Learning Fuzzy Controller," in Proceedings of Fifth International Conference on Fuzzy Systems and Knowledge Discovery, Vol.3, pp. 241-244, Shandong, 2008.
[29] J. H. Holland, Adaptation in Natural and Artificial System, University of Michigan Press, Ann Arbor, Mich, 1975.
[30] Z. Michalewicz, T. Logan, and S. Swaminath, “Evolutionary Operators for Continuous Convex Parameter Spaces”, In Proceedings of the 3rd annual Conference on Evolutionary Programming, World Scientific, pp. 84-97, 1994.
[31] Z. Michalewicz, “A survey of constraint handling techniques in evolutionary computation methods,” Proceedings of the 4th Annual Conference on Evolutionary Programming, MIT Press, pp. 135-155, Cambridge, MA, 1995.
[32] M. Gen and R. Cheng, “A survey of penalty techniques in genetic algorithms,” Proceedings of the IEEE Conference on Evolutionary Computation, pp. 804-809, 1996.
[33] X. Ma and X. Tian, "A Hybrid Evolution Genetic Algorithm for Constrained Optimization," in Proceedings of CIS ''08. International Conference on Computational Intelligence and Security, vol. 2, pp. 206-209, 2008.
[34] D. He, F. Wang and M. Jia, "An improved genetic algorithm for a type of nonlinear programming problems," in Proceedings of ICAL 2008. IEEE International Conference on Automation and Logistics, pp. 2582-2585, 2008.
[35] M. Markaki, A. Vasilakos and I. Kassotakis, "A hybrid genetic algorithm for the provision of isochronous service in high speed networks," in Proceedings of IEEE International Conference on Evolutionary Computation, pp. 133-137, 1997.
[36] T. Y. Chou, T. K. Liu, C. N. Lee and C. R. Jeng, "Method of Inequality-Based Multiobjective Genetic Algorithm for Domestic Daily Aircraft Routing," in Proceedings of IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans, vol. 38, no. 2, pp. 299-308, 2008.
[37] Y. Guo, X. Lin and W. Hu, "Study on Premature Convergence in Assembly Sequence Genetic Planning," in Proceedings of WCICA 2006. The Sixth World Congress on Intelligent Control and Automation, vol. 2, pp. 6470-6474, 2006.
[38] V. N. Dieu and W. Ongsakul, "Enhanced merit order and augmented Lagrange Hopfield network for ramp rate constrained unit commitment," in Proceedings of IEEE Power Engineering Society General Meeting, pp. 7, 2006.
[39] B. Chakraborty, T. Chen, T. Mitra and A. Roychoudhury, "Handling constraints in multi-objective GA for embedded system design," in Proceedings of the 19th International Conference on VLSI Design, pp. 6, 2006.
[40] M. Lukasiewycz, M. Glass, C. Haubelt and J. Teich, "SAT-decoding in evolutionary algorithms for discrete constrained optimization problems," in Proceedings of CEC 2007. IEEE Congress on Evolutionary Computation, pp. 935-942, 2007.
[41] G. Tan, B. Jiang and L. Yang, "A Novel Immune Genetic Algorithm-Based PID Controller Design and Its Application to CIP-I Intelligent Leg," in Proceedings of ICNC 2007. Third International Conference on Natural Computation, vol. 4, pp. 282-286, 2007.
[42] Y. P. Huang and Y. T. Chang and F.-E. Sandnes, "Using Fuzzy Adaptive Genetic Algorithm for Function Optimization," in Proceedings of NAFIPS 2006. Annual meeting of the North American on Fuzzy Information Processing Society, pp. 484-489, 2006.
[43] X. Ling, F. Zhang, Z. Zhu and L. Tang, "Estimating mechanics parameters of rock mass based on improved genetic algorithm," in Proceedings of 2008 Chinese Control and Decision Conference, pp. 4608-4612, 2008.
[44] Q. B. Do, H. Choi and G. H. Roh, "An Evolutionary Optimization of the Refueling Simulation for a CANDU Reactor," in Proceedings of IEEE Transactions on Nuclear Science, vol. 53, no. 5, pp. 2957-2961, 2006.
[45] Q. Wang, R. Wang and J. Chen "Design and application of PID controller based improved artificial immune algorithm," in Proceedings of ISSCAA 2008. 2nd International Symposium on Systems and Control in Aerospace and Astronautics, pp. 1-5, 2008.
[46] T. H. Chen. Integrating agent and web services for ATIS. Master thesis, National Central University, Taiwan, 2002.
[47] H. Z. Lin. Cooperative RoboCup agents using Genetic Case-based Reasoning. Master thesis, National Taipei University of Technology, Taipei, Taiwan, 2008.
[48] F. Y. Hsieh. Adaptive and Cooperative Learning for RoboCup agents. Master thesis, National Taipei University of Technology, Taiwan, 2008.
[49] Tomoharu Nakashima, Masahiro Takatani, Masayo Udo, and Hisao Ishibuchi, An Evolutionary Approach for Strategy Learning in RoboCup Soccer. in IEEE International Conference on Systems, Man and Cybernetics, pp. 2023-28, 2004.
[50] Enrico Pagello, Antonio D’Angelo, and Emanuele Menegatti. Cooperation Issues and Distributed Sensing for Multirobot Systems. Proceedings of the IEEE, Vol. 94, No. 7, pp. 1370-1383, 2006.
[51] Introduction to Expert System. http://lips.lis.ntu.edu.tw/YTCHIANG/STUDY/ others/expert/expert2.htm. (viewed on Jan 1, 2009).
[52] RoboCup Official Site. http://www.robocup.org/. (viewed on Jan 1, 2009).
[53] UvA: Intelligent Autonomous System. http://staff.science.uva.nl/~jellekok/ robocup/index_en.html. (viewed on Jan 1, 2009).
[54] a4ty: RoboCup Simulation League Adaptive Agents. http://www.cs.rtu.lv/dssg/en/ research/robocup/Default.htm. (viewed on Jan 1, 2009).