臺灣博碩士論文加值系統

本研究的目的是要發展一套自評自調模糊類神經網路，使機器能透過學習程序自動建立系統模型或控制器。本文以雙啟發規劃法為基礎配合採用高效率之珈克畢恩(Jacobian)解法推導出自評自調學習法則。在控制應用方面，此自評自調模糊類神經網路能夠從混沌開始終而建立合用的控制器，在建立系統模型應用方面，此網路能透過順序最佳化的學習程序逐漸逼近給予之任意時續函數。本設計採用菅野(Sugeno)一階模糊推論作為基本學習模組，再將其轉換並擴展為類神經網路的學習結構，並建立自評自調學習演算法以自動調整前件和後件的網路參數，終而達成自動學習的目標。本文詳述整個設計的細節，並搭配納倫珈(Narendra)基準系統來驗證此自評自調演算法的成效。最後將此自評自調模糊類神經網路應用於控制旋轉倒單擺的運動，電腦模擬結果顯示此旋轉倒單擺系統能夠從混沌開始學習，終而達成上甩、平衡和追隨行進的所有控制動作。

The goal of this research is to develop an adaptive critic neuro-fuzzy inference system (NFIS) for modeling and control. On the backbone of dual heuristic programming (DHP), a DHP adaptive critic learning scheme that utilizes an effective network Jacobian acquisition is proposed. In control applications, the adaptive critic NFIS can learn from scratch to achieve the control objective. In modeling applications, it can approximate arbitrary continuous function through sequential optimization. The learning structure is based on NFIS that contains fuzzy if-then rules of first-order Sugeno fuzzy model. The tuning rules of premise and consequent parameters are derived. Narendra’s benchmark system is used to verify the performance of the proposed adaptive critic learning algorithm. The ability of modeling is demonstrated by approximating a nonlinear continuous function. The proposed design is applied to obtain the control of a rotary inverted pendulum control. Simulation results show that the rotary pendulum system can learn from scratch to obtain swing-up, balancing and trajectory tracking control.

摘要 I
ABSTRACT III
LIST OF FIGURES VII
LIST OF TABLES IX
CHAPTER 1 1
INTRODUCTION 1
1.1 BACKGROUND OF THIS RESEARCH 1
1.2 MOTIVATION AND CONTRIBUTION 4
1.3 ORGANIZATION OF THIS THESIS 5
CHAPTER 2 7
BASIC OF THE ADAPTIVE CRITIC METHOD 7
2.1 MACHINE LEARNING 7
2.1.1 Supervised Learning 8
2.1.2 Unsupervised Learning 9
2.1.3 Reinforcement Learning 10
2.2 ADAPTIVE CRITIC DESIGN (ACD) 12
2.2.1 Relationship between Reinforcement Learning and Adaptive Critic Design 12
2.2.2 Optimization and Dynamic Programming 13
2.2.3 Approximating Dynamic Programming 14
2.2.4 Category of Adaptive Critic Design 15
2.3 DUAL HEURISTIC PROGRAMMING (DHP) 16
2.3.1 Basic Concept of DHP 16
2.3.2 Update Process 17
2.3.3 Equation for Update 19
2.3.4 Summary of DHP Procedure 20
2.4 NEURAL FUZZY INFERENCE SYSTEM 21
2.4.1 Preliminary 21
2.4.2 Network Architecture 22
CHAPTER 3 27
DESIGN OF ADAPTIVE CRITIC NEURO-FUZZY INFERENCE SYSTEM 27
3.1 MODIFYING NFIS FOR THE DHP ADAPTIVE CRITIC DESIGN 27
3.2 PARAMETER ADJUSTMENT 29
3.3 JACOBIAN ACQUISITION 32
3.4 DHP ADAPTIVE CRITIC DESIGN WITH ON-LINE MODELING 33
3.5 APPROXIMATING ARBITRARY CONTINUOUS FUNCTION 34
CHAPTER 4 37
PERFORMANCE EVALUATION OF ADAPTIVE CRITIC NEURO-FUZZY INFERENCE SYSTEM 37
4.1 NARENDRA’S BENCHMARK SYSTEM 37
4.1.1 Parameter Setup 38
4.1.2 Training Strategy 40
4.1.3 Simulation Result – Jacobian Term 40
4.1.4 Simulation Result – On-line Modeling 43
4.1.5 Comparing the Root-Mean-Square-Error (RMSE) 48
4.2 FUNCTION APPROXIMATION 49
CHAPTER 5 53
APPLICATION IN ROTARY INVERTED PENDULUM CONTROL 53
5.1 DYNAMIC MODEL OF THE ROTARY INVERTED PENDULUM (RIP) 54
5.2 CONTROL OBJECTIVE AND SYSTEM OVERVIEW 60
5.3 SPECIALIZED ADAPTIVE CRITIC NFIS FOR THE RIP CONTROL 61
5.3.1 Action and Critic Structures 61
5.3.2 Training Strategy 62
5.4 SIMULATION RESULTS OF THE RIP CONTROL 66
5.4.1 Simulation Results of Training for Balancing Control 66
5.4.2 Simulation Results of Training for Swing-up Control 68
5.4.3 Overall Test of the Training Result 69
CHAPTER 6 73
CONCLUSION 73
APPENDIX A 75
REFERENCES 77

[Barto, 1983]
Barto, A.G., R.S. Sutton and C.W. Anderson, “Neuronlike Elements that can Solve Difficult Learning Control Problems,” in proceeding of IEEE Transactions on Systems Man & Cybernetics, vol. 13, pp. 834-846, 1983
[Bousquet, 2003]
Olivier Bousquet, Ulrike von Luxburg and Gunnar Rätsch (editors.), Advanced Lectures on Machine Learning, ML Summer Schools 2003, Canberra, Australia, February 2-14, 2003, Tübingen, Germany, August 4-16, 2003, Revised Lectures, Springer, pp.72-112, 2004,.
[Jang, 1992]
J.S. Jang, “Self-learning fuzzy controllers based on temporal back propagation,” IEEE Trans. Neural Networks, vol. 3, no. 5, pp. 714-723, 1992.
[Jang, 1993]
J.S. Jang, “ANFIS: adaptive-network-based fuzzy inference system,” IEEE Trans. on System, Man and Cybernetics, vol. 23, no. 3, pp. 665-685, May/June, 1993.
[Jang, 1995]
J.S. Jang, Chuen-Tsai Sun, “Neuro-fuzzy modeling and control,” Proceedings of the IEEE, vol. 83, Issue 3, pp. 378-406, March 1995.
[Kaelbling, 1996]
Kaelbling, L.P., Littman, M.L. and Moore, A.W., “Reinforcement Learning: A Survey,” Journal of Artificial Intelligence Research 4, pp.237-285, May, 1996.
[Lendaris, 1997a]
Lendaris, G.G., and C. Paintz, “Training Strategies for Critic and Action Neural Networks in Dual Heuristic Programming Method,” Proceedings of International Conference on Neural Networks’97 (ICNN’97), Houston, IEEE Press, pp. 712-717, June, 1997.
[Lendaris, 1997b]
Lendaris, G.G., C. Paintz and T.T. Shannon, “More on Training Strategies for Critic and Action neural Networks in Dual Heuristic Programming Method” (Invited Paper), Proceedings of Systems Man & Cybernetics Society International Conference’97, Orlando, IEEE Press, October, 1997.
[Lendaris, 1998]
Lendaris, G.G., and Shannon, T.T., “Application Considerations for the DHP Methodology,” Proceedings of the International Joint Conference on Neural Networks’98 (IJCNN’98), Anchorage, IEEE Press, pp 1013-1018, March, 1998.
[Lendaris, 1999]
Lendaris, G.G., Shannon, T.T. and Rustan, A., “A Comparison of Training Algorithms for DHP Adaptive Critic Neuro-control,” Proceedings of International Conference on Neural Networks’99 (IJCNN''99), Washington,DC IEEE Press, July, 1999.
[Lendaris, 2001]
Lendaris,G.G., T.T. Shannon, L.J. Schultz, S. Hutsell and A. Rogers, “Dual Heuristic Programming for Fuzzy Control,” Proceedings of IFSA/NAFIPS Conference, Vancouver, B.C., July, 2001.
[Lin, 2006]
Wei-song Lin, Tzu-Wei Yang, Yu-Chun Shin, Tsi-Yu Chuang and Pei-te Liu, “Rotary Inverted Pendulum System: Design and Control,” CCMEE Lab., NTU, Jan., 2006.
[Narendra, 1994]
K. S. Narendra and S. Mukhopadhyay, “Adaptive control of nonlinear multivariable systems using neural networks,” Neural Networks, vol. 7, no. 5, pp. 737-752, 1994.
[Nilsson, 1996]
Nils J. Nilsson, Introduction to Machine Learning: an early draft of proposed textbook, Robotics Laboratory, Department of Computer Science, Stanford, Dec. 1996, chapter 1,9, and 11. http://ai.stanford.edu/~nilsson/mlbook.html
[Negnevitsky, 2002]
Michael Negnevitsky, Artificial Intelligence: A Guide to Intelligent Systems, Addison-Wesley, 2002, chapter 4, 6 and 8.
[Prokhorov, 1995]
D. Prokhorov, R. Santiago, and D Wunsch, “Adaptive Critic Designs: A Case Study for Neurocontrol,” Neural Networks, vol. 8, no. 9, pp. 1367-1372, 1995.
[Prokhorov, 1997]
D.V. Prokhorov, D.C. Wunsch, “Adaptive Critic Designs,” IEEE Trans. Neural Networks, vol. 8, no. 5, pp. 997-1007, September, 1997.
[Russell, 2003]
Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach, Prentice Hall, 2003, chapter 18-21.
[Sutton, 1984]
Sutton, R.S., “Temporal Aspects of Credit Assignment in Reinforcement Learning”, Doctoral Dissertation, Department of Computer and Information Science, University of Massachusetts, Amherst, MA, 1984.
[Shannon, 1999a]
Shannon, T.T., “Partial , Noisy and Qualitative Models for Adaptive Critic Based Neuro-control,” Proceedings of International Conference on Neural Networks’99 (IJCNN''99), Washington, D.C. IEEE Press, July, 1999.
[Shannon, 1999b]
Shannon, T.T., and Lendaris, G.G., “Qualitative Models for Adaptive Critic Neurocontrol,” Proceedings of IEEE SMC''99 Conference, Tokyo, IEEE Press, Oct, 1999.
[Shannon, 2000a]
Shannon, T.T. and Lendaris, G.G., “Adaptive Critic Based Approximate Dynamic Programming for Tuning Fuzzy Controllers,” Proceedings of IEEE-FUZZ 2000, IEEE Press, May, 2000.
[Shannon, 2000b]
T.T. Shannon and G.G. Lendaris, “A New Hybrid Critic-Training Method for Approximate Dynamic Programming,” Proceedings of International Society for the System Sciences, ISSS’2000, Toronto, August, 2000.
[Shannon, 2001]
Shannon, T.T. and G.G. Lendaris, “Adaptive Critic Based Design of a Fuzzy Motor Speed Controller,” Proceedings of ISIC2001, Mexico City, Mexico, September, 2001.
[Schultz, 2001]
Schultz, L.J., T.T. Shannon and G.G. Lendaris, “Using DHP Adaptive Critic Methods to Tune a Fuzzy Automobile Steering Controller,” Proceedings of IFSA/NAFIPS Conference, Vancouver, B.C., July, 2001.
[Shannon, 2004]
Shannon, T.T., “Linguistic Adaptive Critics for Tuning Fuzzy Controllers,” Proc. IEEE Conf. North American Fuzzy Information Processing, Alberta, Canada, 2004.
[Sutton, 1998]
Sutton, R.S. and Barto, A.G., Reinforcement Learning: An Introduction, MIT Press, January, 1998.
[Terasoft]
Terasoft Control Module for MATLAB/Simulink Embedded Target for TI C2000 DSP, User’s Manual, Terasoft Inc.