臺灣博碩士論文加值系統

本研究的目的是要發展一套順序學習式RBF類神經網絡，以配合雙啟發規劃法推導出自評學習控制系統，使機器能透過自評學習程序自動建立控制器。在順序學習式RBF類神經網絡方面，我們結合方向性RBF類神經網絡和自動增減神經元的訓練法則，得到自增減方向性RBF類神經網絡，簡稱GGP-DDRBFN。GGP-DDRBFN應用於學習非對稱的函數時，可達到比起一般的RBF類神經網路更精確及更平順的結果。在自評設計法方面，本文提出結合雙啟發規劃法及GGP-DDRBFN的設計，由於方向性類神經網路的特性可以有效的降低漣波效應，因此提高了雙啟發規劃法的效能。本文詳述整個設計的細節，並搭配實驗室所發展的自主輪型機器人為基準系統來驗證此自評演算法的成效。電腦模擬結果顯示，比起以多層感知類神經網絡為架構之雙啟發規劃法，本文所提出的自評演算法擁有更快的收斂速度，並在控制器所得的數學模型為一非完整系統模型且系統有未知干擾的情況下，能夠得到更精準的追跡特性。

The goal of this research is to develop a methodology for the design of sequential learning dual heuristic programming (DHP) control systems. A direction-dependent radial basis function network (DDRBFN) with the sequential learning algorithm of generalized growing-pruning (GGP) mechanism is developed to implement the DHP control system. DDRBFN outperforms traditional RBFN in approximating asymmetrical functions. Ripple phenomenon in the DDRBFN approximation becomes insignificant so that the partial derivative quantities can be calculated correctly in polarities. Based on GGP-DDRBFN, the DHP motion control system and the associated updating rules of the critic and the actor are developed. By implementing the DHP control system to conduct the motion of an autonomous wheeled mobile robot, the performance of the proposed design is verified. Simulation results show that, in the sequential learning, the GGP-DDRBFN-based DHP design converges significantly faster than the multi-layered perceptron network based design. In addition, under the circumstances of system model with unmodeled dynamics and unknown disturbance, GGP-DDRBFN-based DHP design obtains better tracking property.

摘要 i
ABSTRACT iii
Chapter 1 Introduction 1
1.1 Background of this research 1
1.2 Motivation and Contribution 5
1.3 Organization of this thesis 6
Chapter 2 Adaptive Critic Design and Dual Heuristic Programming 9
2.1 Adaptive Critic Design 9
2.1.1 Overview of Adaptive Critic Design 10
2.1.2 Preliminary of Mathematical Representation 12
2.1.3 Mathematical Formulation of ACD 13
2.1.4 Categories of ACD’s 15
2.2 Dual Heuristic Programming 17
2.2.1 DHP updating process 17
2.2.2 Training Strategies of DHP 23
2.2.3 Comparison among HDP, DHP, and GDHP 25
2.3 Summary 27
Chapter 3 Online Sequential Learning Radial Basis Function Networks 29
3.1 MLP and Batch Training 29
3.1.1 MLP neural network 30
3.1.2 Batch Training Algorithm 31
3.1.3 Limitations of Batch Training in Online Learning Applications 33
3.2 Online Sequential Learning RBFN 34
3.2.1 The Online Sequential Learning Requirement 35
3.2.2 RBFN 36
3.2.3 Online Sequential Learning Scheme 38
3.2.4 GGP-RBFN 39
3.2.5 Geometry meaning of significance of the RBF neuron 47
3.3 Online Sequential Learning Direction-Dependent RBF Neural Network 49
3.3.1 Advantages of DDRBFN 49
3.3.2 Definition of DDRBFN 50
3.3.3 GGP-DDRBFN Algorithm 53
Chapter 4 Sequential Learning Dual Heuristic Programming Control with GGP-DDRBFN 57
4.1 DHP Control by Sequential Learning 58
4.1.1 DHP design with GGP-DDRBFN 58
4.1.2 Critic Network Update 59
4.1.3 Actor Network Update and Jacobian Aquisition 60
4.1.4 DHP design with GGP-RBFN 62
4.1.5 Stability of DHP control by sequential learning 65
4.2 Performance of GGP-DDRBFN and GGP-RBFN in Sequential Learning 72
4.2.1 Mackey-Glass Chaotic Time Series Prediction 72
4.2.2 Ripple Effect of Function Approximation 77
4.3 DHP Control using MLP, GGP-RBFN and GGP-DDRBFN 83
4.3.1 Plant Model of WMR 83
4.3.2 Simulation results of MLP, GGP-RBFN and GGP-DDRBFN based DHP control with exact plant model 86
4.3.3 Simulation results of MLP, GGP-RBFN and GGP-DDRBFN based DHP control with unmodeled dynamics and unknown disturbance 96
Chapter 5 Conclusion 117
Reference 119

[Brown, 1994]
M. Brown and C. Harris, NeuroFuzzy Adaptive Modelling and Control, chapter 3 and 4, Prentice Hall, 1994.
[Burden, 2001]
R. L. Burden and J. D. Faires, Numerical analysis, 7th ed., chapter 6, Brooks/Cole, 2001.
[Friedman, 1982]
A. Friedman, Foundations of modern analysis, chapter 3, Dover, 1982.
[Huang, 2005]
G.-B. Hunag, P. Saratchandran, and N. Sundararajan, “A Generalized Growing and Pruning RBF (GGAP-RBF) Neural Network for Function Approximation,” IEEE Trans. on Neural Network, vol. 16, no. 1, pp. 57-67 January 2005.
[I. N. N. C. S. Committee, 2005]
I. N. N. C. S. Committee, Benchmark group on data modeling, January, 2005 [Online].
Available: http://neural.cs.nthu.edu.tw/jang/benchmark
[Junkins, 1972]
J. L. Junkins and J. R. Jancaitis, “Smooth irregular curves,” Photogram. Eng., vol. 38, no. 6, pp. 565–573, June. 1972.
[Kadirkamanathan, 1993]
V. Kadirkamanathan and M. Niranjan, “A Function Estimation Approach to Sequential Learning with Neural Networks,” Neural Computation, vol. 5 pp. 954-975, 1993.
[Kaelbling, 1996]
L.P. Kaelbling, M.L. Littman, and A.W. Moore, “Reinforcement Learning: A Survey,” Journal of Artificial Intelligence Research 4, pp.237-285, May, 1996.
[Lendaris, 1997a]
G. G. Lendaris and C. Paintz, “Training Strategies for Critic and Actor Neural Networks in Dual Heuristic Programming Method,” Proceedings of International Conference on Neural Networks’97 (ICNN’97), Houston, IEEE Press, pp. 712-717, June, 1997.
[Lendaris, 1997b]
G. G. Lendaris, C. Paintz, and T.T. Shannon, “More on Training Strategies for Critic and Actor neural Networks in Dual Heuristic Programming Method” (Invited Paper), Proceedings of Systems Man & Cybernetics Society International Conference’97, Orlando, IEEE Press, October, 1997.
[Lendaris, 1998]
G. G. Lendaris, and T. T. Shannon, “Application considerations for the DHP methodology,” in Proceedings of the International Joint Conference on Neural Networks’98 (IJCNN’98), Anchorage, IEEE Press, pp 1013-1018, March, 1998.
[Lendaris, 2000]
G. G. Lendaris, L. Schultz, and T. T. Shannon, “Adaptive critic design for intelligent steering and speed control of a 2-axle vehicle,” Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks, 2000. IJCNN 2000, vol. 3, pp. 73-78, July 2000.
[Lendaris, 2001]
G. G. Lendaris, T. T. Shannon, L. J. Schultz, S. Hutsell, and A. Rogers, “Dual Heuristic Programming for Fuzzy Control,” Proceeedings of IFSA / NAFIPS Conference, Vancouver, B.C., July, 2001.
[Liang, 2006]
N.-Y. Liang, G.-B. Hunag, P. Saratchandran, and N. Sundararajan, “A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks,” IEEE Trans. on Neural Network, vol. 17, no. 6, pp. 1411-1423, Nov. 2006.
[Lin, 2004]
W.-S. Lin, C.-L. Huang, M.-K. Chuang and G.-C. Liu, “Modeling a wheeled mobile robot for autonomous navigation design,” IASTED International Conference on Modeling, Identification and Control, pp. 275-280, Grindelwald, Switzerland, Feb. 2004
[Lin, 2007a]
W.-S. Lin, L.-H. Chang, and P.-C. Yang, “Adaptive critic anti-slip control of wheeled autonomous robot,” IEE/IET Control Theory and Applications, vol. 1, issue 1, pp. 51-57, Jan. 2007
[Lin, 2007b]
W.-S. Lin, P.-C. Yang, “DHP Adaptive Critic Motion Control of Autonomous Wheeled Mobile Robot,” IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning, pp. 311-317, Honolulu, HI, USA, April 2007
[Mackey, 1977]
M. C. Mackey and L. Glass, “Oscillation and chaos in physiological control systems,” Science, vol. 197, pp. 287–289, 1977.
[Marsden, 1993]
J. E. Marsden and M. J. Hoffman, Elementary Classical Analysis. 2nd edition, chapter 1, 4, 5, and 6, Freeman, 1993.
[Negnevitsky, 2004]
M. Negnevitsky, Artificial Intelligence: A Guide to Intelligent Systems, chapter 6, Addison-Wesley, 2002.
[Park, 2003]
J.-W. Park, R. G. Harley, and G. K. Venayagamoorthy, “Adaptive-critic-based optimal neurocontrol for synchronous generators in a power system using MLP/RBF neural networks,” IEEE trans. on Industry Applications, vol. 33, no 5, pp. 1529-1540, 2003.
[Platt, 1991]
J. C. Platt, “A Resource Allocating Network for Function Interpolation,” Neural Computation, vol. 3 pp. 213-225, 1991.
[Prokhorov, 1995]
D. Prokhorov and R. Santiago, and D. Wunsch, “Adaptive critrc designs:
a case study for Neurocontrol,” Neural Networks, vol. 8, pp. 1367-1372, 1995.
[Prokhorov, 1997]
D. Prokhorov and D. Wunsch, “Adaptive critrc designs,” IEEE Trans. on Neural Networks, vol. 8, pp. 997-1007, Sep. 1997.
[Schultz, 2001]
L. J. Schultz, T. T. Shannon, and G. G. Lendaris, “Using DHP Adaptive Critic Methods to Tune a Fuzzy Automobile Steering Controller,” Proceedings of IFSA/NAFIPS Conference, Vancouver, B.C., July, 2001.
[Shannon, 1999a]
T. T. Shannon, “Partial , Noisy and Qualitative Models for Adaptive Critic Based Neuro-control,” Proceedings of International Conference on Neural Networks''R99 (IJCNN''99), Washington, D.C., IEEE Press, July, 1999.
[Shannon, 1999b]
T. T. Shannon and G. G. Lendaris, “Qualitative Models for Adaptive Critic Neurocontrol,” Proceedings of IEEE SMC''99 Conference, Tokyo, IEEE Press, October, 1999.
[Shannon, 2000a]
T. T. Shannon and G. G. Lendaris, “A New Hybrid Critic-Training Method for Approximate Dynamic Programming,” Proceedings of International Society for the System Sciences, ISSS''R2000, Toronto, August, 2000.
[Shannon, 2000b]
T. T. Shannon and G. G. Lendaris, “Adaptive Critic Based Approximate Dynamic Programming for Tuning Fuzzy Controllers,” Proceedings of IEEE-FUZZ 2000, San Antonio, Texas, IEEE Press, May, 2000.
[Shannon, 2001]
T. T. Shannon and G. G. Lendaris, “Adaptive Critic Based Design of a Fuzzy Motor Speed Controller,” Proceedings of ISIC2001, Mexico City, Mexico, September, 2001.
[Shannon, 2003]
T. T. Shannon, R. A. Santiago, and G. G. Lendaris, “Accelerated Critic Learning In Approximate Dynamic Programming via Value Templates and Perceptual Learning,” Proceedings of International Joint Conference on Neural Networks''R032 (IJCNN'' 2003), paper #775, Portland, OR, IEEE Press, July, 2003.
[Si, 2001]
J. Si and Y.-T. Wang, “On-Line Learning Control by Association and Reinforcement,” IEEE Trans. on Neural Networks, vol. 12, no. 2, pp. 264-276, March, 2001.
[Singla, 2007]
P. Singla, K. Subbarao, and J. L. Junkins, “Direction-Dependent Learning Approach for Radial Basis Function Networks,” IEEE Trans. on Neural Network, vol. 18, no. 1, pp. 203 - 222, Jan. 2007.
[Venayagamoorthy, 2002]
G. K. Venayagamoorthy, R. G. Harley, and D. C. Wunsch, “Comparison of heuristic dynamic programming and dual heuristic programming adaptive critics for neurocontrol of a turbogenerator,” IEEE Trans. on Neural Networks, vol. 13, no. 3, pp. 764-773, 2002.
[Venayagamoorthy, 2003]
G. K. Venayagamoorthy, R. G. Harley, and D. C. Wunsch, “Dual heuristic programming excitation neurocontrol for generators in a multimachine power system,” IEEE Trans. on Industry Applications, vol. 39, no. 2, pp. 382-394, 2003.
[Werbos, 1977]
P. J. Werbos, “Approximate dynamic programming for real-time control and neural modeling,” in Handbook of Intelligent Contorl, White and Sofge, Eds. New York: Van Nostrand Reinhold, pp. 493-525, 1992.
[Werbos, 1990]
P. J. Werbos, “A menu of designs for reinforcement learning over time,” Neural Networks for Control, pp. 67-95, MIT Press, Cambridge, MA, 1990.
[Yan, 2000]
Yan Li, N. Sundararajan, and P. Saratchandran, “Analysis of Minimal Radial Basis Function Network Algorithm for Real-Time Identification of Nonlinear Dynamic Systems,” IEE Proceedings Part D - Control Theory and Applications, UK, vol. 147, no. 4, pp. 476-484, July 2000.
[Yingwei, 1997]
Lu Yingwei, N. Sundararajan, and P. Saratchandran, “A Sequential Learning Scheme for Function Approximation Using Minimal Radial Basis Function Neural Networks,” Neural Computation vol. 9, pp. 461-478, 1997.