臺灣博碩士論文加值系統

過去的研究證明，變動架構的類神經網路具有求解任何可學習性問題的能力。傳統的作法上，變動類神經網路架構大多是為了降低輸出結果誤差。然而，只根據輸出結果誤差來決定類神經網路架構，會造成為了降低輸出結果誤差，而增加網路架構複雜度的問題。過去關於輸出誤差及複雜度的問題，缺乏一套整合的理論基礎。
針對上述問題，本文引入多目標及進化演算法的概念，在輸出結果誤差與網路架構複雜度兩者間，求得一平衡點。使得發展類神經網路架構時，能在可接受的誤差範圍內，得到架構較簡單的網路。
本篇論文選擇了Xor 和 8-bit parity check兩個問題來作為效能測試。結果顯示，MeNet所得到的類神經網路架構在隱藏神經元個數、連線權重值個數、突變運算次數、網路架構複雜度及執行時間這幾方面的效果都明顯優於傳統的方法。而在輸出結果誤差這方面，雖然數據顯示MeNet的誤差稍微提高，不過仍可接受的範圍之內。尤其在愈複雜的問題上，提升的效果差距更是明顯。

Interest in algorithms which dynamically construct artificial neural networks has been growing in recent years. The traditional methods that have been used to construct near optimal artificial neural networks are to minimize the sequence of error functions associated with the growing network. But, constructing a near optimal artificial neural networks by considering the sequence of error unilaterally, will lead to more complicated network architecture.
This paper proposes an evolutionary system for constructing neural networks named “MeNet”. Combining the concepts of “Evolutionary Programming” and ”Multi-Objective Programming”, MeNet minimizes the sequence of error and the complexity of network architecture simultaneously.
MeNet has been tested on two benchmark problems, including Xor problem and 8-bit parity checking problem. The results show that MeNet can produce neural network architecture with lower complexity while keeping the acceptable accuracy. Such effects are clear, especially in testing more complicated problem.

第1章 緒論 1
1.1 研究動機與背景 1
1.2 研究目的 3
1.3 研究流程 3
1.4 本文架構 4
第2章 相關文獻回顧 6
2.1 前言 6
2.2 類神經網路簡介 6
2.3 類神經網路之設計與改進 10
2.4 全連結型前饋式類神經網路 11
2.5 進化演算法簡介 13
2.6 EANN＆EPNET簡介 15
2.7 多目標規劃簡介 16
第3章 研究方法 20
3.1 前言 20
3.2 EANN裡改進網路架構的步驟 20
3.3 EPNET與MENET的改進網路架構的步驟 21
3.3.1 Random initiation of ANNs 22
3.3.2 Initial partial training 23
3.3.3 Rank-base Selection 23
3.3.3.1 STEP 1 of EPNet 23
3.3.3.2 STEP 1 of MeNet’s SSE 24
3.3.3.3 STEP 1 of MeNet’s t_net 24
3.3.3.4 Multi-Objective Programming of MeNet 26
3.3.3.5 Rank-base Selection STEP 2 28
3.3.4 Mutations 28
3.3.4.1 Hybrid Training 29
3.3.4.2 Hidden node deletion 30
3.3.4.3 Connection deletion 30
3.3.4.4 Connection/Node addition 31
3.3.4.5 Cell Division 31
3.3.5 Obtain the new generation 32
3.3.6 Further training 32
第4章 研究結果分析 33
4.1 測試條件 33
4.2 時間複雜度的定義 36
4.3 XOR PROBLEM測試 38
4.3.1 測試問題說明 38
4.3.2 測試結果 38
4.3.3 綜合數據分析 40
4.3.3.1 隱藏神經元個數 40
4.3.3.2 連線權重數 41
4.3.3.3 突變運算次數 42
4.3.3.4 輸出結果誤差 43
4.3.3.5 網路架構複雜度 43
4.3.3.6 執行時間 44
4.4 8-BIT PARITY CHECKING PROBLEM 45
4.4.1 測試問題說明 45
4.4.2 測試結果 46
4.4.3 綜合數據分析 48
4.4.3.1 隱藏神經元個數 48
4.4.3.2 連線權重數 49
4.4.3.3 突變運算次數 50
4.4.3.4 輸出結果誤差 51
4.4.3.5 網路架構複雜度 52
4.4.3.6 執行時間 53
4.5 小結 54
第5章 結論 57
參考文獻： 59

Diane E. Kirrane , ”Machine learning” , Training & Development Journal , Vol. 44 , ISS. 12 , 24-29 , December , 1990
Eun-Jung Song , Min-Jea Tahk , “ Three-dimensional midcourse guidance using neural networks for interception of ballistic targets ” , IEEE Trans. Aerospace and Electronic Systems , Vol. 38 , No. 2 , April , 2002
Henson T. , W. Huxhold , and D. Bowman , “An enhanced neural network learning algorithm with simulated annealing” , Third workshop on Neural Network , 87-94 , February , 1992
Mahmood R. Azimi-Sadjadi , ”Recursive Dynamic Node Creation in Multi-layer Neural Networks” , IEEE Trans. Neural Networks , Vol 4. , No 2 , March , 1993
Martin T. Hagan , Howard B. Demuth and Mark Beale , “ Neural Network Design ,” International Thomson Publishing Company , 1995
Meng Joo Er , Shiqian Wu , Juwei Lu , and Hock Lye Toh , “ Face recognition with radial basis function (rbf) neural networks ,” IEEE Trans. Neural Networks , Vol. 13 , No. 3 , May , 2002
Paul Sajda , Clay Spence , and John Pearson , “ Learning contextual relationships in mammograms using a hierarchical pyramid neural network ,” IEEE Trans. Medical Imaging , Vol. 21 , No. 3 , March , 2002
P. Arena , M. Bucolo , L. Fortuna , L. Occhipinti , “ Cellular neural networks for real-time DNA microarray analysis ,” IEEE Engineering in Medicine and Biology , March/April , 2002
Rudy Setiono and Lucas Chi Kwong Hui , “ Use of a quasi-Newton method in a feedforward neural network construction algorithm ,” IEEE Trans. Neural Networks , Vol. 6 , No. 1 , January , 1995
Scott E. Fahlman and Christian Lebiere , “The Cascade-Correlation Learning Architecture ,” School of Computer Science Carnegie-Mellon University Pittsburgh , PA 15213 , August , 1991
Xicai Yue , Datian Ye , Chongxun Zheng , and Xiaoyu Wu , “ Neural networks for improved text-independent speaker identification ,” IEEE Engineering in Medicine and Biology , March/April , 2002
X. Yao and Y. Liu , “ A New Evolutionary System for Evolving Artificial Neural Networks ,” IEEE Trans. Neural Networks , Vol 8. , pp. 694-713 , May , 1997
X. Yao , “ Evolving Artificial Neural Networks ,” Proceedings of the IEEE , Vol. 87 , No. 9 , September , 1999
H. J. Zimmermann , “ Fuzzy Set Theory and Its Applications ,” Kluwer Academic Publishers , 1985
許志義，「多目標決策」，五南圖書有限公司，二月，1994
葉怡成， 「類神經網路模式應用與實作」 ，第六版，儒林圖書有限公司，1999