臺灣博碩士論文加值系統

本論文在研究一種新的演算法則，它利用基因演算法來訓練類神經網路的初始值，且其染色體為浮點數形式，這樣一來可節省計算時間。它不僅可改善倒傳遞演算法容易掉入區域最小解的缺點而且能夠克服基因法則無法有效收斂至鄰近區域最小解的困難。更進一步的研究單一基因交配比全部基因交配更快收斂至最佳解。最後，由模擬結果證實了此種演算法有較佳的收斂特性並且收斂時間也明顯的縮短。

This thesis investigates a novel neural network training technique, which employs the genetic algorithm to finding the initial values of the neural network. It is represented by a chromosome containing parameters in floating-point, so that the convergence rate to the minima becomes faster. This hybrid algorithm can overcome not only the drawback of easily slumping into local minima of back-propagation but also the genetic algorithm’s defect that can’t efficiently converge to the minima of the neighborhood. Further, the thesis shows that a gene changing one by one is better than that changing totally at once. Finally, the results of computer simulations reveal that this algorithm has a better convergence property, the time of global searching is obviously decreased.

Chapter 1 Introduction1
1.1 Motivation and Purpose…………………………………………..1
1.2 Organization of the Thesis………………………………………..3
Chapter 2 Introduction to genetic algorithm and neural network4
2.1 History of GA and NN…………………………………….4
2.2 Genetic Algorithm Theory Review………………………6
2.2.1 Basic Operations and Features………………………6
2.2.2 Simple Genetic Algorithm……………………………9
2.3 Neural Network Theory Review………………………… 13
2.3.1 Basic Operations and Features……………………..14
2.3.2 Back-propagation algorithm………………………….20
Chapter 3 Neural Network Training Technique by Using Genetic Algorithm29
3.1 Specialized Operators and Concepts………………… 29
3.1.1 Expression of Population……..……………………29
3.1.2 Rank-based Fitness…………………………………… 30
3.1.3 Rank-based Reproduction…………..………………32
3.1.4 Age and Lifetime………………………………………33
3.1.5 Pocket algorithm………………………………………34
3.1.6 Floating Crossover and Mutation……………………34
3.2 Neural Network Training Technique.………….………35
3.2.1 Weight……………………………………………………35
3.2.2 Genetic Algorithm………………………………………36
3.2.2.1 Parametric Crossover……………………………..36
3.2.2.2 Parametric Mutation………………………………38
3.2.3 Connection……………………………………………….38
3.2.4 Back-Propagation………………………………………39
3.2.5 Architecture…………………………………………….42
3.3 A Novel Genetic Algorithm………………………………43
3.3.1 Parametric Crossover………………………………….43
3.3.2 Parametric Mutation……………………………………44
Chapter 4 Simulation Results and Applications45
4.1 Simulation Results……………………………………….45
4.1.1 Problem Statement…………………….……………….45
4.1.2 Implementation…………………………….………….47
4.1.3 Simulation Results…………………………………….50
4.1.4 Analysis…………………………………………………52
4.2 Application………………………….……………………52
4.2.1 Problem Statement………………………………………52
4.2.2 Implementation………………………………………… 53
4.2.3 Simulation Result………………………………………55
4.2.4 Analysis………………………………………………… 57
Chapter 5 Conclusion 58
References 59

[1] D. E Rumelhart., G. E. Hinton, AND R. J. Williams, Learning representations by error propagation, In D. E. Rumelhart, J. L. Mc Clelland, etal., eds., Parallel Distributed Processing, VOL.1, chap.8. Cambridge, MA: MIT Press, 1986a.
[2] R. K. Belew, J. Mclnerney, and N. N. Schraudolph, Evolving Networks: Using the Genetic Algorithm with Connectionist Learning, Artificial Life II, CG. Laugton, C. Taylor, J. D. Farmer, and S. Rasmussen, Eds. Addison-Wesley, pp.511-547, 1991.
[3] Seong-Whan Lee, “Off-line Recognition of Totally Unconstrained Handwritten Numerals Using Multilayer Cluster Neural Network,” IEEE Transactions on Pattern Analysis and Machine Intelligence. Vol. 18. No.6, pp.648-652, 1996.
[4] L. C. Jain and R. K. Jain, “An Evolutionary Approach to Training Feed-Forward and Recurrent Neural Networks,“ IEEE, Second International Conference on Knowledge-Based Intelligent Electronic System, pp. 21-23 April, 1998, Adelaide, Australia.
[5] J. H., Holland, Adaptation in Natural and Artificial Systems. Ann Arbor: The University of Michigan Press, 1975.
[6] Melanie Mitchell, An Introduction to Genetic Algorithm. USA, MIT Press, 1997.
[7] Marilyn McCord Nelson W. T. Illingworth, A practical Guide to Neural Nets.
[8] Chin-Teng Lin and C. S. George Lee, Neural Fuzzy Systems: A Neuro-Fuzzy Synergism to Iintelligent Systems, 1999.
[9] D. E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley, Reading, Mass, 1989.
[10] T. S. Chung, Y. K. Wong and M. H. Wong, ”Application of Evolving Neural Network to unit commitment”, IEEE, Energy Management and Power Delivery, pp. 154-159, 1998.
[11] D. Goldberg, Genetic Algorithm in search, optimization and machine learning, Addison Wesley Publishing Company, 1989.
[12] M. A. El-Sharkawi and S. J. Huang, “Application of Genetic-Based Neural Network to Power System Static Security Assessment,” International Conference on Intelligent System Application to Power Systems, Montpellier, France, Sep pp.423-429, 1994.
[13] DAVIS. L., ED. Handbook of Genetic Algorithms, New York: Yan Nostrand Reinho., 1991,
[14] T. Munakata, “ Commercial and Industrial AI. ” Commun. ACM37 (3): 23-26, 1994.
[15] A. E. Bryson, AND Y. C. HO. Applied Optimal Control, New York: Blaisdell, 1969.
[16] P. J. Werbos, Beyond regression: New tools for prediction and analysis in the behavior sciences. Doctoral dissertation, Harvard University, 1974.
[17] Y. Lecuu, “A learning procedure for asymmetric network.” Cognitiva. 85:599-604, Paris, 1985.
[18] D. E. Rumelhart, G. E. Hinton, AND R. J. WILLIAMS., “Learning representations by back-propagation errors“. Nature 323:533-536, 1986b.
[19] K. T. Lee, Neural Network combined with Genetic Algorithm — Evolutionary Neural Network. , 1999.
[20] C. Darken and J. Moody, “Towards faster stochastic gradient search,” In Advances in Neural Information Processing Systems, pp.1009-1016, 1992, San Mateo, CA: Morgan Kaufmann.
[21] S. I. Gallant, Neural Network Learning and Expert Systems, The MIT Press, Massachusetts, 1993.
[22] D. Whitley, “The Genitor Algorithm and Selection Pressure: Why Rank-based Allocation Of Reproductive Trials is Best,“ The Third International Conference on Genetic Algorithms, Los Atlos, CA, pp.116-121, 1989.
[23] J. Abras, Z. Michalewicz and J. Mulawka, “GAVaPS- a Genetic Algorithm with Varying Population Size,” Proceedings of the Evolutionary Computation Conference, Orlando, pp. 73-78, 1994.
[24] E. K. Blum, “Approximation of Boolean functions by sigmoidal networks: Part : XOR and other two-variable functions,” Neural Computation, Vol.1, pp.532-540, 1989.
[25] Y. P. Chen and K.T. Lee, ”A new method combined Genetic Algorithm and Neural Network,” 自動控制研討會, pp. 162-164, 1999.
[26] Carl G. Looney, Pattern Recognition Using Neural Networks-Teory and Algorithms for Engineers and Scientists, 1997
[27] Gwo-En Wang and Jhing-Fa Wang, “A New Hierarchical Approach For Recognition Of Unconstrained Handwritten Numerals,” IEEE Transactions on Consumer Electronics, Vol. 40, No. 3, pp. 428-436, AUGUST 1994.