臺灣博碩士論文加值系統

摘要

類神經網路由於具有學習訓練能力，常被使用於複雜的非線性系統識別與預測上，其中又以倒傳遞網路最為被廣泛的使用，但是卻有收斂速度緩慢以及容易收斂至局部極小值的問題。
本文針對倒傳遞網路學習率及性能函數的改變對於網路的影響作初步的探討，首先提出加入Delta-Bar-Delta Rule之倒傳遞網路（DBDNN）及加入Adaptive Gain之DBDNN，並以幾種不同的訓練對去和標準倒傳遞網路（BPNN）作收斂情況之比較；再提出兩種誤差函數：平均四次方誤差、、規則化均方誤差，連同均方誤差、平均絕對誤差及總合平方誤差等分別加入BPNN與DBDNN中去作訓練和比較。最後以Lyapunov Stability Theorem驗證其穩定性。

目錄
中文摘要 i
英文摘要 ii
目錄 iii
符號說明 v
第一章 緒論 1
1.1 前言 1
1.2 文獻回顧 2
1.3 研究動機 3
1.4 內容架構 3
第二章 類神經網路介紹 5
2.1 簡介 5
2.1.1 功能 5
2.1.2 分類 7
2.1.3 架構 8
2.2 倒傳遞網路介紹 8
2.2.2 最陡坡降法 9
2.2.2 具動量項之最陡坡降法 13
2.3 Delta-Bar-Delta Rule（DBD Rule） 14
2.4 誤差函數介紹 17
第三章 模擬分析 20
3.1 BPNN、DBDNN與加入adaptive gain之DBDNN之比較 20
3.2 BPNN使用不同誤差函數之比較 23
3.2.1 動態系統之模擬 24
3.3 DBDNN使用不同之誤差函數的比較 24
3.4 穩定度之驗證 25
3.5 結果與討論 27
第四章 結論 30
4.1 研究成果 30
4.2 未來展望 31
參考文獻 32

參考文獻

[1] W. T. Miller, R. S. Sutton, and P. J. Werbos, “Neural networks for control,” MIT Press, 1990.
[2] D. A. White and D. A. Sofge Edited, “Handbook of intelligent control,” Neural, Fuzzy, and Adaptive Approaches, Van Nostrand Reinhold, 1992.
[3] M. Gori and A. Tesi, “On the problem of local minima in backpropagation,” IEEE Trans.Pattern Anal. Mach. Intell,Vol. 14, No. 1, pp. 76-86, 1992.
[4] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning internal representation by error propagation,” Parralel Distributed Processing, Vol. 1, pp. 318-362, 1986.
[5] C. M. Kuan and K. Hornik, “Convergence of learning algorithms with constant learning rates,” IEEE Transations on Neural Networks, Vol. 2, No. 5, pp. 484-490, 1991.
[6] J. A. Hertz, A. Krogh, and R. G. Palmer, “Introduction to the theory of neural computation,” Redwood City, CA: Addison Wesley, 1991.
[7] J. K. Kruschke and J. R. Movellen, “Benefits of gain: Speeded learning and minimal hidden layers in back-propagation networks,” IEEE Transations on Neural Networks, Systems Man Cybernet. 21, pp. 273-280, 1991.
[8] Y. F. Yam and T. W. S. Chow, “Extended backpropagation algorithm,” Electronics Letters, Vol. 29, No. 19, pp. 1701-1702, 1993.
[9] R. L. Watrous, “Learning algorithms for connectionist networks: Applied gradient methods of nonlinear optimization,” IEEE First International Conference on Neural Networks, Vol. 2, pp. 619-627, 1987.
[10] J. C. Principe, N. R. Euliano, and W. C. Lefebvre, “Neural and adaptive systems: Fundamentals through simulations,” 1999.
[11] P. R. Nachtsheim, “A first order adaptive learning rate algorithm for back propagation networks,” IEEE international conference on Neural Networks, Vol. 1, No. 27, 1994.
[12] T. Yabuta and T. Yamada, “Neural networks controller characteristics with regard to adaptive control,” IEEE Transations on systems, Man and Cybernetics, Vol. 22, No. 1, pp. 170-177, 1992.
[13] V. J. Mathews and Z. Xie, “Stochastic gradient adaptive filter with gradient adaptive step size,” IEEE Transations on Signal Process, Vol. 41, No. 6, pp. 2075-2087, 1993.
[14] W. Ang and B. Farhang-Boroujeny, “A new class of gradient adaptive step-size LMS algorithms,” IEEE Transations on Signal Process, Vol. 49, No. 4, pp. 805-810, 2001.
[15] M. Moreira and E. Fiesler, “Neural networks with adaptive learning rate and momentum terms,” IDIAP Technical Report, Vol. 4, No. 95, 1995.
[16] C. C. Yu and B. D. Liu, “A Backpropagation algorithm with adaptive learning rate and momentum coefficient,” IEEE IJCNN, Vol. 2, No. 12-17, pp.1218-1223, 2002.
[17] V. V. Phansalkar and P. S. Sastry, “Analysis of the back-propagation algorithm with momentum”, IEEE Transations on Neural Networks, Vol. 5, No. 3, pp. 505-506, 1994.
[18] A. A. Minia and R. D. Williams, “Acceleration of back-propagation through learning and momentum adaptation,” IJCNN, Vol 1, pp. 676-679, 1990.
[19] R. A. Jacobs, “Increased rates of convergence through learning rate adaptation”, IEEE Transations on Neural Networks, Vol. 1, pp. 295-307, 1988.
[20] P. Gupta and N. K. Sinha,” An improved approach for nonlinear system identification using neural networks”, Journal of the Franklin Institute, Vol. 336, No. 4, May, pp. 721-734, 1999.
[21] A. I. Hanna and D. P. Mandic, “On “an improved approach for nonlinear system identification using neural networks””, Journal of the Franklin Institute, Vol. 340, No. 5, August, pp. 363-370, 2003.
[22] D. P. Mandic and I. R. Krcmar, “On training with slope adaptation for feedward neural networks,” Proceedings of the Fifth IEEE Seminar on Neural Network Applications in Electrical Engineering, pp. 42-45, 2000.
[23] S. H. Oh, “Improving the error backpropagation algorithm with a modified error function,” IEEE Transations on Neural Networks, Vol. 8, No. 3, pp. 799-803 , 1997.
[24] X. G. Wang, Z. Tang, H. Tamura, and M. Ishii, “A modified error function for the backpropagation algorithm,” Neurocomputing 57, pp. 477-484, 2004.
[25] E. B. Kosmatopoulos, M. A. Christdoulou and P. A. Ioannou, “Learning laws with exponential error convergence for recurrent neural networks,” Decision and Control, Vol. 3, No. 15-17, pp. 2810-2811, 1993.
[26] L. Behera, S. Kumar, and A. Patnaik, “A novel learning algorithm for feedforward networks using Lyapunov function approach,” Intelligent Sensing and Information Processing, pp. 277-282, 2004.
[27] Y. Fang and T. G. Kincaid, “Stability Analysis of Dynamical Neural Networks,” IEEE Transations on Neural Networks, Vol 7, No. 4, pp. 996-1006, 1996.
[28] I. Baruch and I. Stoyanov, “Global model of Neural Networks, Stability and learning,” Electrical and Electronics Engineers in Israel, pp. 4.3.5/1-4.3.5/5, 1995.
[29] A. M. Ostrowski, “Solutions of equations in Euclidean and Banach spaces,” New York: Academic, 1973.
[30] M. Lazar and O. Pastravanu, “A neural predictive controller for nonlinear systems,” Mathematics and Computers in Simulation, pp. 315-324, 2002.
[31] A. Upegui and E. Sanchez, “An FPGA platform for on-line topology exploration of spiking neural networks,” Microprocessors and Microsystems, Vol. 29, No. 5, pp. 211-223, 2005.
[32] 蘇柏仁，「PC-Based遠端即時監控於智慧型伺服馬達之研究」，海洋大學機械與機電工程研究所碩士學位論文，民國93年。
[33] A. Zaknich, “Neural Networks for Intelligent Signal Processing”, Vol. 4.
[34] M. T. Hagan, H. B. Demuth, and M. H. Beale, “Neural Network Design,” Thomson Learning, 1996.
[35] S. Haykin, “Neural networks : A comprehensive foundation,” Maxwell Macmillan, Canada, 1994.
[36] G. Cybenko, “Approximation by superpositions of a sigmoidal function,” Mathematics of controls, Signals, and Systems, Vol. 2, pp. 303-314, 1989.