臺灣博碩士論文加值系統

本篇論文旨在使用改良適應型粒子群體最佳演算法，尋找類神經網路之最佳權重值。使用倒傳遞法則是利用一個近似最陡坡降法的演算方式，但因為初始權重的關係，使其往往只能求得局部最佳解，無法保證求得全域最佳解。粒子群最佳演算法是一種可以使所有個體朝同一方向與目標前進的方法，因此可避免陷入局部最佳解。
但是在參數選擇上多依靠經驗選取，參數的選取會直接影響到穩定性和最佳化性能。因此我們提出改良適應型粒子群最佳演算法。其中主要調整粒子群最佳演算法中的慣性權重，使之以指數遞減方式逐漸減少。
最後的研究將驗證我們提出的方法可以獲得最佳的結果。

In this thesis, the weights of the artificial neural networks (ANN) are trained by modified adaptive particle swarm optimization (MAPSO). Back-propagation (BP) is an approximate steepest descent algorithm, but BP often finds the local optimal solution not the global optimal solution, because of the initial weights. The particle swarm optimization (PSO) method is one of the most powerful methods for letting the entire individuals move to the target, hence it can avoid to get the local optimal solution.
However, the parameters, which greatly influence the algorithm stability and performance, are selected by depending on the designers’ experience. MAPSO is hereby presented with exponential decrease weights.
Finally, the demonstrated examples are presented to illustrate the better performance of the proposed methodology (MAPSO-ANN).

Abstract in Chinese..................................i
Abstract.............................................ii
Acknowledgements.....................................iii
Contents.............................................iv
List of Figures......................................v
List of Tables.......................................vi
Chapter 1. Overview..................................1
1.1 Background and Motivation....................1
1.2 Organization of the Thesis...................3
Chapter 2. Artificial Neural Networks................4
2.1 Biological Inspiration.......................5
2.2 Neuron Model and Networks Topologies.........7
2.3 Learning Rules...............................11
Chapter 3. Particle Swarm Optimization...............13
3.1 Swarm Intelligence...........................13
3.2 Evolutionary Computation.....................15
3.3 Methodology of Particle Swarm Optimization...17
3.4 Adaptive PSO Algorithm.......................25
3.5 Modified Adaptive PSO Algorithm..............26
Chapter 4. Neural Networks Design Based on MAPSO.....28
4.1 Back-Propagation Algorithm...................28
4.2 Back-Propagation Design Based On MAPSO.......36
Chapter 5. Simulation Results........................39
5.1 Introduction.................................39
5.2 Results......................................40
Chapter 6. Conclusions...............................49
References...........................................50
List of Publications.................................53
Biography............................................54

[1].D. Taylor, B. Bojduj and F. Kurfess, “Towards Using Neural Networks to Perform Object-Oriented Function Approximation,” Proceedings of IEEE International Joint Conference on Neural Networks, pp. 3331-3337, 2008.
[2].G. Di Muro and S. Ferrari, “A Constrained-Optimization Approach to Training Neural Networks for Smooth Function Approximation and System Identification,” Proceedings of IEEE International Joint Conference on Neural Networks, pp. 2353-2359, 2008.
[3].J. Kenndy and R. C. Eberhart, “Particle Swarm Optimization,” Proceedings of IEEE International Conference on Neural Networks, pp. 1942-1948, 1995.
[4].R. C. Eberhart and J. Kennedy, “A New Optimizer Using Particle Swarm Theory,” Proceedings of the Sixth International Symposium on Micromachine and Human Science, Nagoya, Japan. pp. 39-43, 1995.
[5].L. Wang, Y. Zeng, C .Gui and H. Wang, “Application of Artificial Neural Network Supported by BP and Particle Swarm Optimization Algorithm for Evaluating the Criticality Class of Spare Parts,” Proceedings of IEEE Third International Conference on Natural Computation, pp. 528-532, 2007.
[6].C. G. Chang, D. W. Wang, Y. C. Liu and B. K. Qi, “Application of Particle Swarm Optimization Based BP Neural Network on Engineering Project Risk Evaluating,” Proceedings of IEEE Third International Conference on Natural Computation, pp. 750-754, 2007.
[7].H. N. Robert, Neurocomputing, Addison-Wesley, Reading, MA, 1990.
[8].W. McCulloch and W. Pitts, “A Logical Calculus of the Ideas Immanent in Nervous Activity,” Bulletin of Mathematical Biophysics, vol. 5, pp. 115-133, 1943.
[9].M. T. Hagan, H. B. Demuth, and M. Beale, Neural Network Design, Thomson Learning, 1996.
[10].S. Haykin, Neural Networks: a Comprehensive Foundation, 2nd edition, Prentice Hall, 1999.
[11].A. Zaknich, Neural Networks for Intelligent Signal Processing, World Scientific, 2003.
[12].M. Dorigo and L. M. Gambardella, “Ant Colonies for the Traveling Salsman Problem,” BioSystems, vol. 43, pp. 73-81, 1997.
[13].M. Dorigo and T. Stützle, Ant Colony Optimization, MIT Press, 2004.
[14].E. Bonabeau, M. Dorigo and G. Theraulaz, Swarm Intelligence: From Natural to Artificial Systems, Oxford University Press, 1999.
[15].F. Heppner and U. Grenander, “A Stochastic Nonlinear Model for Coordinated Bird Flocks,” The Ubiquity of Chaos, AAAS Publications, 1990.
[16].Y. Shi and R. Eberhart, “A Modified Particle Optimizer,” Proceedings of the 1998 IEEE World Congress on Computational Intelligence, pp. 69-73, 1998.
[17].Y. Shi and R. Eberhart, “Parameter Selection in Particle Swarm Optimization,” Proceedings of the 7th International Conference on Evolutionary Programming VII, vol. 1447, pp. 591-600, 1998.
[18].Z. Zhixia and L. Caiwu,“Application of the Improved Particle Swarm Optimizer to Vehicle Routing and Scheduling Problems,”Proceedings of 2007 IEEE International Conference on Grey Systems and Intelligent Services, pp. 18-20, 2007.
[19].Y. Chen, Q. Zhu and Z. Li,“A Novel Artificial Neural Network Based on Hybrid PSO-BP Algorithm in the Application of Adaptive PMD Compensation System,”ISNN 2007,Part III, LNCS 4493, pp. 311-319, 2007.
[20].F. Rosenblatt, “The Perceptron: a Probabilistic Model for Information Storage and Organization in the Brain,” Psychological Review, vol. 65, pp . 386-408, 1958.
[21].K. Hornik, M. Stinchcombe and H. White, “Multilayer feedforward networks are universal approximators,” Neural Networks, pp. 359-366, 1989.
[22].J. B. Li and Y. K. Chung, “A Novel Back-propagation Neural Network Training Algorithm Designed by an Ant Colony Optimization,” IEEE/PES Transmission and Distribution Conference Exhibition: Asia and Pacific, pp. 1-5, 2005.
[23].K. Adem and K. Dervis, “Training Recurrent Neural Networks for Dynamic System Identification using Parallel Tabu Search Algoruthm,” Proceedings of the IEEE International Symposium on Intelligent Control, pp. 113-118, 1997.
[24].C. Blum and K. Socha, “Training Feed-forward Neural Networks with Ant Colony Optimization: an Application to Pattern Classification,” Fifth International Conference on Hybrid Intelligent Systems, pp. 233-238, 2005.
[25].E. Alba and J. F. Chicano, “Training Neural Networks with GA Hybrid Algorithm,” Proceedings of GECCO, LNCS 3102, pp. 852-863, 2004.
[26].K. Hornik, M. Stinchcombe and H. White, “Univeral Approximation of an Unknown Mapping and its Derivatives using Multilayer Feedforward Networks,” Neural Networks, vol. 3, no. 5, pp. 551-560, 1990.
[27].D. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley, 1989.
[28].M. Gen and R. Cheng, Genetic Algorithms & Engineering Design, John Wiley & Sons, 1997.
[29].R. Jacob and R. S. Yahya, “Particle Swarm Optimization in electromagnetism,” IEEE transactions Antennas and Propagation, pp. 591-594, 2004.
[30].R. C. Eberhart and Y. Shi, “Particle Swarm Optimization: Developments, Applications and Resources,” Proceedings IEEE International Conference on Evolutionary Computation, pp. 81-86, 2001.