臺灣博碩士論文加值系統

在本研究中以一種新的學習演算法來訓練類神經網路，這方法是模擬物理中的電磁學理論，名為電磁演算法。假設所抽樣的樣本為帶電電荷，藉著吸引力與排斥力的作用使得該點往最佳解移動，直到整個空間趨於穩定，擺脫一般演算法如遺傳演算法、坡度陡降法及牛頓法會掉入區域最佳解的情況，進而改善網路之效能，本文以三個案例來進行網路訓練，分別為XOR、股價預測問題與紡織業的零售問題。而使用此演算法可以節省電腦訓練時間，訓練結果與遺傳演算法以及倒傳遞網路做比較，結果顯示該演算法在求最佳解上有較好的績效。
關鍵字：類神經網路、啟發式演算法、模擬電磁演算法、遺傳演算法

In this paper, a new heuristic algorithm of global optimization for training the neural network has been introduced. This method simulates the electromagnetism theory of physics by considering each point as an electrical charge. Through the attraction and repulsion of the charges, sample points move toward the optimality. It is not trapped into local optima like algorithms such as genetic algorithm, gradient descent method or Newton’s method. The convergence is carefully studied by using three illustrated examples that are XOR model, stock forecasting model and textile retail operation model, respectively. The performance measures are compared with genetic algorithm and back-propagation algorithm. Using this algorithm we can train the neural network with great saving on the computation time. The results indicate that our algorithm performed much better than genetic algorithm in finding the optimal solution globally.
Keywords: neural network, heuristic algorithm, electromagnetism, genetic algorithm

CHAPTER 1 INTRODUCTION………………..………………1
1.1 Neural Network………………………………………...…………1
1.2 Optimization Methods for Neural Network Learning…………….2
1.3 Electromagnetism Algorithm for Global Optimization Model..…4
1.4 Thesis Outline………………………………………..…………..4
CHAPTER 2 LITERATURE REVIEW………….…………..7
2.1 Neural Network…………………………………………………7
2.2 Optimization Methods………………………………………..…..8
CHAPTER 3 METHODOLOGY…………..………………..11
3.1 Artificial Neural Network……………………………..………11
3.1.1 Architecture………………..……………………………11
3.1.2 The Methods of Steepest Descent and Newton's..…..…16
3.1.3 Delta Learning Rule for Multiperceptron Layer……...…..19
3.2 Genetic Algorithms…………..…………………………………26
3.2.1 Biological Background………………………………….26
3.2.2 The Structure of Genetic Algorithm…………………..…32
3.2.3 Combining Neural Networks and Genetic Algorithm….…37
3.3 Electromagnetism Algorithm……………………………………38
3.3.1 General Scheme…………………………………………..40
3.3.2 Theoretical Study of the Algorithm…………………..….45
3.4 Examples………………………………………………………...46
3.4.1 Exclusive OR Problem……………………………………47
3.4.2 Hand-Calculated Example of the Genetic Algorithm…….50
3.4.3 Solving Problem with Electromagnetism Algorithm…..…53
CHAPTER 4 CASE STUDY……………………………….…58
4.1 The Continuous Exclusive-OR Problem………………………..58
4.1.1 The Model……..…………………..……………..……….58
4.1.2 Genetic Algorithm Training Results………………………59
4.1.3 Electromagnetism Algorithm Training Results………..…60
4.1.4 Summary……………………………………………...…..62
4.2 Stock Forecasting Model………………………………………..64
4.2.1 Parameter Design…………………………………………64
4.2.2 Back-Propagation Algorithm Training Results…………..64
4.2.3 Genetic Algorithm Training Results………………………66
4.2.4 Electromagnetism Algorithm Training Results…………..67
4.2.5 Summary………………………………………………….68
4.3 Textile Retail Operations Model…………………………....…70
4.3.1 Parameter Design…………………………………………70
4.3.2 Back-Propagation Algorithm Training Results…………...71
4.3.3 Genetic Algorithm Training Results………………………72
4.3.4 Electromagnetism Algorithm Training Results…………...73
4.3.5 Summary………………………………………………….74
CHAPTER 5 CONCLUSION AND FUTURE RESEARCH…..76
5.1 Conclusion……………………...…………….……...……..76
5.2 Future Research……………………..……………………....77
REFERENCES………………………..………………………..…..79
APPENDIX A Theoretical Study of the EM…………..………….82

1. Anderson, J. A., Silverstein, J. W., Ritz, S. A. and R. S. Jones. (1977), Distinctive Features, Categorical Perception, and Probability Learning: Some Applications of a Neural Model, Psychological Review, 84, 134 -451.
2. Armijo, L. (1996), Minimization of Functions Having Lipschitz Continuous First Partial Derivatives, Pacific Journal of Mathematics, 16,1-3.
3. Ben-Tal, A., Melmen A., and Zowe, J. (1990), Curved Search Methods for Unconstrained Optimization, Optimization, 5, 21, 669-695
4. Bribil, S.I., Fang, S.C. (2000), Electromagnetism for Global Optimization, submitted to Journal of Global Optimization.
5. Broomhead, D. S. and Lowe, D. (1988), Multi-variable Functional Interpolation and Adaptive Networks, Complex Systems, 2, 321-355.
6. Calvin, W. H. (1994), The Emergence of Intelligence, Scientific American, special issue on Life in the Universe, 79-85.
7. Dekkers, A. and Aarts, E. (1991), Global Optimization and Simulated Annealing. Mathematical Programming, 50,367-393.
8. Dorigo, V. M. M. and Colorni, A. (1996), The Ant System: Optimization by a Colony of Cooperating Agents, IEEE Transactions on System, Man and Cybernetics, 26,1-13.
9. Glover, J.K.F., Laguna, M. (1995), Genetic Algorithms and Tabu Search: Hybrids for Optimization, Computers and Operation Research, 22, 111-134.
10. Goldberg, D. (1989), Genetic Algorithms in search, New York：Wesley.
11. Holland J. H. (1975), Adaptation in Natural and Artificial Systems, University of Michigan Press, Ann Arbor, MI.
12. Holland J. H. (1992), Genetic Algorithms, Scientific America, 267, 66-72.
13. Hopfield, J. J.(1982), Neural Networks and Physical Systems With Emergent Collective Computational Abilities, National Academic Science of the USA, 79, 2554-2578.
14. Huyer, W.and Neumaier, A. (1999), Global Optimization by Multilevel Coordinate Search, Journal of Global Optimization, 14,331-355.
15. Ingber, L. and Rosen, B. E. (1992), Genetic algorithms and very fast simulated annealing, Mathematical and Computer Modeling, 16 ,87-100.
16. Jang, G. S. (1993), Intelligent Stock Trading System with Price Trend Prediction and Reversal Recognition Using Dual-Module Neural Networks, Journal of Applied Intelligence, 3, 225-248.
17. Jang, J. S., Sun, C. T. and Mizutani, E. (1997), Neuro-Fuzzy AND Soft Computing, New York：Practice Hall.
18. Lecun, Y. (1985), A Learning Scheme for Asymmetric Threshold Networks, Proceedings of Cognitive, 85, 599-604.
19. Michalewicz , Z. (1994),Genetic Algorithms + Data Structures = Evolution Programs, Springer Verlag, Berlin.
20. More, B.J. and Wu, Z. (1995), Global Smoothing and Continuation for Large-scale Molecular Optimization, Argonne National Laboratory, Illinois: Preprint MCS-P539-1095.
21. Muselli, M., and Ridella, S. (1990), Supervised Learning Using a Genetic Algorithm, Proceedings of INNC90, Paris, France.
22. Nuttle, H. L. W., King, R. E., and Hunter, N. A. (1991), A Stochastic Model of the Apparel- Retailing Process for Seasonal Apparel, Journal of Text. Inst., 82, 2, 247-259.
23. Kirkpatrick, S., Gelatt, C. D. and Vecchi, M. P. (1983), Optimization by Simulated Annealing, Science, 220, 671-680.
24. Kohonen, T. (1982), Self-Organized Formation of Topological Feature Maps, Biological Cybernetics, 43, 59-69.
25. Kramer, A. H. and Sangiovanni-Vincentelli (1989), A., Efficient Parallel Learning Algorithm for Neural Networks, Advances in Neural Information Processing Systems, 1, San Mateo, CA, 40-48.
26. Lefteri, H. T. and Robert, E. U. (1997), Fuzzy and Neural Approaches in Engineering, Canada.
27. Parker, D. B. (1985), Learning-Logic: Casting the Cortex of the Human Brain in Silicon, Technical Report TR-47, Center for Computational Research on Economics and Management Science, MIT.
28. Romeo, F. I. (1989), Simulated annealing: theory and applications to layout problems, PhD thesis, EECS Department, University of California at Berkeley, May.
29. Rooif, A., Jain, L. and Johnson, R. (1998), Neural Network Training Using Genetic Algorithms, Singapore.
30. Rumelhart, D. E., Hinton, G. R., and Williams, R. J. (1986), Learning Internal Representations by Error Propagation, Parallel Distributed Processing, 1, D. E..
31. Schaffer, J.D. (1989), A Study of Control Parameters Affecting Online Performance of Genetic Algorithms for Function Optimization, Proceedings of the 3rd International Conference on Genetic Algorithms, 51-60.
32. Törn, M.M.A, Viitanen,S.(1999), Stochastic Global Optimization: Problem Classes and Solution Techniques, Journal of Global Optimization,14,437-447.
33. Wodrich, M. and Bilchev, G. (1997), Cooperative Distributed Search: The Ant's Way, Journal of Control and Cybernetics, 26:3.
34. Wu, P. (1997), Neural Networks and Fuzzy Control with Application Manufacturing and Management, Ph.D dissertation, North Carolina State University, Raleigh, USA.
35. Wu, P., Fang, S. C., H. L. W., and King, R. E., (1995), Decision Surface Modeling of Apparel Retail Operations using Neural Network Technology, International Journal of Operations and Quantitative Management ,1,1, 33-47.
36. Zurada, J. M. (1995), Introduction to Artificial Neural Systems, Boston.