跳到主要內容

臺灣博碩士論文加值系統

(54.224.133.198) 您好!臺灣時間:2022/01/29 21:43
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果 :::

詳目顯示

我願授權國圖
: 
twitterline
研究生:李衍博
研究生(外文):Yen-Po Lee
論文名稱:分割且克服之學習與模組化感知器神經網路之研究
論文名稱(外文):Divide-and-Conquer Learning and Modular Perceptron Network
指導教授:傅心家傅心家引用關係
指導教授(外文):Prof. Hsin-Chia Fu
學位類別:博士
校院名稱:國立交通大學
系所名稱:資訊工程系
學門:工程學門
學類:電資工程學類
論文種類:學術論文
論文出版年:2002
畢業學年度:90
語文別:中文
論文頁數:108
中文關鍵詞:多層感知器分割與克服學習模組化感知器網路權值估測過度學習維度縮減
外文關鍵詞:Multilayer perceptronDivide-and-Conquer LearningModular Perceptron NetworkWeight EstimationCurse of DimensionalityDimensionality Reduction
相關次數:
  • 被引用被引用:1
  • 點閱點閱:187
  • 評分評分:
  • 下載下載:0
  • 收藏至我的研究室書目清單書目收藏:1
本篇論文提出了一個以模組化感知器網路(Modular Perceptron Network (MPN))為基礎,整合分割與克服學習(Divide-and-Conquer Learning (DCL))的方法與權值估測(Weight Estimation)技巧的模組化類神經網路架構之創新設計。最後再將感知器網路進行維度縮減(Dimensionality Reduction),在解決維度障礙(Curse of Dimensionality)有顯著效益。
當一個 multilayer perceptron 在訓練過程中,如果陷入到局部極小值區域或是平坦區域,則無法獲得一良好的學習性能,而本論文所建議的Divide-and-Conquer Learning (DCL)方法可被應用於劃分目前較難學習的訓練樣本區成為兩個較易學習的樣本區,可化解局部極小值區域或是平坦區域的困境。 當產生其中一個新的學習樣本區時 self-growing perceptron network 及其初始估測權值將被建構。 另外一個的樣本區將在原有的神經網路上重新學習。整個學習樣本區重複的進行學習與權值估測,直到MPN可以完全學習全部的訓練樣本為止。 本篇論文所建議的MPN已經利用大家所熟悉的Two-spirals problem及另選三組廣為人知實際的紀錄資料進行評估與比較。 在Two-spirals problem的訓練階段,MPN配以權值估測可利用非常少的data presentations (相對其他方法減少99.01%~ 87.86%) 達成較佳的學習性能、較佳的普遍性(相對其他方法高4.0%)及在retrieving階段花費較少的處理時間(相對其他方法減少2.0% ~81.3% )。在三組實際的紀錄資料訓練過程顯示,這個 MPNs架構再配合權值估算(Weight Estimation)與單一的MLP比較,除上述的優點外其過度學習(overfitting)現象可以減少或避免。 另外, 在解決高維度輸入資料在學習過程中所產生的維度障礙(Curse of Dimensionality),本文亦建立一評估演算法,以克服此一問題,由試驗結果顯示輸入資料維度可大幅降低(減少75%~88%),data presentations亦有明顯下降(約減少70%~95%),如此可獲得一小型化的學習網路,但是學習及測試性能仍能維持原有水準。由於此架構具有自我增生及快速局部學習之特質,所以此種類神經網路模組(MPN)對於一個快速改變的環境下很容易適用於 on-line and/or incremental learning的需求。
A novel Modular Perceptron Network (MPN) and Divide-and-Conquer Learning (DCL) schemes with Weight Estimation for the design of modular neural networks are proposed. Finally the perceptron networks is performed on Dimensionality reduction with obvious effectiveness for solution in Curse of Dimensionality.
When a training process in a multilayer perceptron falls into a local minimum or stalls in a flat region, the proposed DCL scheme is applied to divide the current training data region (e.g., a hard to be learned training set) into two easier (hopely) to be learned regions. The learning process continues when a self-growing perceptron network and its initial weight estimation are constructed for one of the newly partitioned regions. Another partitioned region will resume the training process on the original perceptron network. Data region partitioning, weight estimating and learning are iteratively repeated until all the training data are completely learned by the MPN. We have evaluated and compared the proposed MPN with several representative neural networks on the Two-spirals problem and real-world database.
On learning the the Two-spirals problem, the MPN achieves better weight learning performance by requiring much less data presentations ( less 99.01%~ 87.86%) during the network training phases, and better generalization performance (4.0% better), and less processing time (less 2.0% ~ 81.3%) during the retrieving phase. On learning the real-world data, the MPNs show the same performance as above problem learning results and less overfitting compared to single MLP. Otherwise, for solution in curse of dimensionality, induced from high dimensional input data during learning process. An evaluation algorithm is established in this paper to overcome this problem. It shows that from the experimental results the dimension of the input data can be largely decreased (less 75% ~ 88%) and the data presentations are also reduced (less 70% ~ 90%). So a small size MPNs can be procured with learning and testing performance maintained as the good level as before. In addition, due to its self-growing and fast local learning characteristics, the modular network MPN can easily adapt to on-line and/or incremental learning requirements for a rapid changing environment.
1 Introduction
2 A Survey of Modular Neural Networks
2.1 Definition of the MNN
2.2 Categories of the MNN
2.3 Advantages of the MNN
3 Modular Perceptron Networks
3.1 Introduction
3.2 Networks Architecture
3.3 Divide-and-Conquer Learning and the Error Correlation Partition
3.4 Learning scheme in the Integration Engine
4 Experiments and evaluation of MPN
4.1 Experiment on the Two-Spiral Problem
4.2 Experiment on the Pima Indians Diabetes Dataset
4.3 Experiment on the Credit Approval Dataset
4.4 Experiment on the Voting Records Dataset
5 Weight Estimation for MPN
5.1 Introduction
5.2 Weight Vector Orientation Estimation
5.3 Weight Vector Quantity Estimation
6 Experiments and evaluation of MPN with Weight Estimation
6.1 Experiment on the Two-Spiral Problem
6.2 Experiment on the Pima Indians Diabetes Dataset
6.3 Experiment on the Credit Approval Dataset
6.4 Experiment on the Voting Records Dataset
7 Dimension Reduction for MPN
7.1 Introduction
7.2 The Evaluation Function for the Reduction Criterion
7.3 The Algorithm for the Dimension Reduction
8 Experiments of Dimensionality Reduction for MPN
8.1 Experiment on the Pima Indians Diabetes Dataset
8.2 Experiment on the Credit Approval Dataset
8.3 Experiment on the Voting Records Dataset
9 Conclusions and Future Work
9.1 Summary of Results
9.2 Future Researches
@BOOK{Haykin99,
author={Simon Haykin},
title={Neural Networks: A comprehensive
Foundation},
publisher={Prentice Hall, Inc.},
year={1999},
}
@BOOK{MHP90,
editor={Alianna J. Maren, Craig T. Harston, and Robert M. Pap},
title={Handbook of
neural computing applications},
publisher={CA: Academic Press},
address={San Diego},
year={1990},
}
@BOOK{Nilsson65,
author={ N. J. Nilsson},
title={Learning machines: Foundations of Trainable
Pattern-Classifying Systems},
publisher={McGraw-Hill},
address={New York},
year={1965},
}
% ------------------------------------------------------------------------
@ARTICLE{FredSchap96a,
author={Y. Freund and Robert E. Schapire},
title={Experiments with a New Boosting Algorithm},
journal={Machine Learning: Proceedings of the Thirteenth International
Conference},
year={1996},
}
@ARTICLE{Friedman97,
author={ N. Friedman, D. Geiger, and M. Goldsmidt},
title={Bayesian Network Classifiers},
journal={ Machine Learning},
volume={29},
year={1997},
pages={131-163},
}
@ARTICLE{hanson,
author={Stephen Jose Hanson},
title={Meiosis Networks},
journal={Advances in Neural Information
Processing Systems},
volume={2},
year={1990},
pages={533-541},
publisher={ CA: Morgan Kaufmann},
address={San Mateo},
}
@ARTICLE{Hashem97,
author={S. Hashem},
title={Algorithms for
optimal linear combinations of neural networks},
journal={IEEE Int.
Conf. Neural Networks(ICNN97)},
volume={1},
year={1997},
pages={242-247},
}
@ARTICLE{Jacobs91a,
author={R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton},
title={Adaptive mixtures of local experts},
journal={Neural Computation},
volume={3},
year={1991},
pages={79-87},
}
@ARTICLE{Jordan94,
author={M. I. Jordan, R. A. Jacobs},
title={Hierarchies mixtures of experts and the EM algorithm
},
journal={Neural Computation},
volume={6},
year={1994},
pages={181-214},
}
% --------------------------------------------------------------------------
@ARTICLE{karay,
author={N. B. Karayiannis},
title={Hybrid learning schemes for
fast training of feed-forward neural networks},
journal={ Mathematics
and Computers in Simulation},
volume={41},
year={1996},
pages={13-28},
}
@ARTICLE{LW88,
author={ K. K. Lang and M. K. Witbrock},
title={Learning to tell two spirals
apart},
journal={Proceedings of the 1988 Connectionist Models Summer
School},
publisher={ CA: Morgan Kaufmann},
address={San Mateo},
year={1991},
pages={52-61},
}
@ARTICLE{lee,
author={Y. P. Lee and H. C. Fu},
title={Weight Estimation for
the Learning of Modular Perceptron Networks},
journal={Proceedings
of the 1999 IEEE Workshop on Neural Networks for Signal Processing
IX},
pages={103-111},
year=1999,
}
@ARTICLE{uci,
author={P. M. Murphy and D. W. Aha},
title={UCI repository of machine learning databases},
journal={ http://www.ics.uci.edu/$\sim$mlearn/MLRepository.html.},
}
@ARTICLE{Naftaly97,
author={U. Naftaly, N. Intrator and D. Horn},
title={Optimal ensemble averaging of neural network},
journal={Neural Networks},
volume=8,
year=1997,
pages=藫-296",
}
@ARTICLE{RonGaw95,
author={Eric Ronco and Peter Gawthrop},
title={Modular Neural Networks: a state of the art},
journal={Technical
report: CSC-95026 Center for System and control University of
Glasgow},
year=1995,
}
@ARTICLE{Rudy,
author={Rudy Setiono and Huan Liu},
title={Neural-Network Feature Selector},
journal={IEEE Trans. on Neural Networks},
volume={8},
number={3},
pages={654-662},
month=May,
year={1997},
}
@ARTICLE{FredSchap95,
author={Y. Freund and R. E. Schapire},
title={A Decision-Theoretic
Generalisation of On-line Learning and an Application to Boosting},
journal={Proceedings of the Second European Conference on Computational
Learning Theory},
pages={23-37},
month=March,
year={1995},
}
@ARTICLE{fahlman,
author={S. E. Fahlman, C. Lebiere},
title={The
cascade-correlation learning architecture},
journal={Advances in Neural Information Processing Systems},
volume={2},
pages={ 524-532},
publisher={ CA: Morgan Kaufmann},
address={San Mateo},
year={1990},
}
% ------------------------------------------------------------------------
@ARTICLE{Schap90,
author={Robert Schapire},
title={The Strength of Weak Learnability},
journal={ Machine Learning},
volume={5},
number={2},
pages={197-227},
year={1990},
}
@ARTICLE{Valiant84,
author={L. G. Valiant},
title={ A theory of the learnable},
journal={Communications of the ACM},
volume={27},
number={11},
pages={1134-1142},
year={1984},
}
@ARTICLE{wynne,
author={ Mike Wynne-Jones},
title={Node Splitting: A Constructive Algorithm for Feed-Forward Neural Networks},
journal={Neural Computing and Applications},
volume={1},
number={1},
pages={17-22},
year={1993},
}
%----
@ARTICLE{Drucker95,
author={ Harris Drucker, and Corinna Cortes},
title={Boosting decision trees},
journal={In Advances in Neural
Information Processing Systems 8},
pages={xxxxx},
year={1995},
}
@ARTICLE{Drucker93,
author={Harris Drucker, Robert E. Schapire, and Patrice Simard},
title={Improving Performance in Neural Networks Using Boosting},
journal={In Advances in Neural Information Processing Systems 5},
volume={1},
number={1},
pages={42-49},
year={1993},
}
@ARTICLE{Drucker92,
author={ H. Drucker, and R. Schapire and P. Simard},
title={Boosting Performance in Neural Networks},
journal={International
Journal of Pattern Recognition and Artifical Intelligence},
pages={705-719},
year={1993},
}
@ARTICLE{dk91,
author={K. I. Diamantaras, S. Y. Kung},
title={An Unsupervised Neural Model for
Oriented Principal Component Extraction},
journal={Proc. Int.
Conf. Acoustics, Speech, Signal Processing},
pages={1049-1052},
year={1991},
}
@ARTICLE{Chin94,
author={C. C. Chiang, H. C. Fu},
title={Using Multithreshold
Quadratic Sigmoidal Neurons to improve Classification
Capability of Multilayer Perceptrons},
journal={IEEE Trans. on
Neural Networks},
volume={5},
number={3},
pages={516-519},
year={1994},
}
@ARTICLE{chiang,
author={C. C. Chiang, and H. C. Fu},
title={A divide-and-conquer
methodology for modular supervised neural network design},
journal={International Conference on Neural Networks},
pages={119-124},
year={1994},
}
@ARTICLE{Auda99,
author={Gasser Auda, Mohamed Kamel},
title={Modular Neural Networks: A survey},
journal={International Journal of Neural Systems},
volume={9},
number={2},
pages={129-151},
year={1999},
}
@ARTICLE{AtiAiy98,
author={A. Atiya and R. Aiyad, and S. Shaheen},
title={A practical gated expert network},
journal={Proc. 1998 IEEE
International Joint Conference on Neural Network (IJCNN)},
volume={1},
pages={419-424},
address={Alaska},
month=May,
year={1998},
}
@ARTICLE{ash,
author={ T. Ash,},
title={Dynamic node creation in backpropagation networks},
journal={in the proceedings of Connection Science},
volume={1},
pages={365-376},
year={ 1989},
}
@ARTICLE{tenorio,
author={M. F. Tenorio, W. T. Lee,},
title={Selforganizing neural networks for the idenfication problem},
journal={In David S. Touretzky, editor, Advances in Neural Information
Processing Systems}, address={Morgan Kaufmann, San Mateo, CA,}
volume={II},
pages={56-61},
year={ 1989},
}
@ARTICLE{cios,
author={K. J. Cios, N. Liu,},
title={Amachine learning method for generation of
a neural network architecture: a continuous ID3 algorithm},
journal={IEEE Transaction on Neural Networks},
volume={3},
number={2},
pages={280-291},
month={March},
year={ 1992},
}
@ARTICLE{ooyen,
author={A. V. Ooyen, B. Nienhuis,},
title={Improving the convergence of the
back-propagation algorithm },
journal={ Neural Networks},
volume={5},
pages={465-471},
year={ 1992},
}
@ARTICLE{lee2,
author={H. C. Fu, Y. P. Lee, C. C. Chiang, and H. T. Pao,},
title={Divide-and-Conquer Learning and Modular Perceptron Networks},
journal={IEEE Transactions on Neural Networks},
volume={12},
number={2},
pages={250-263},
month={March},
year={ 2001},
}
@ARTICLE{robert,
author={R. A. Jacobs,},
title={Increased rates of convergence through learning
rate adaption},
journal={ Neural Networks},
volume={1},
pages={295-307},
year={ 1988},
}
@ARTICLE{langley,
author={P. Langley},
title={Selection of relevant features in machine learning},
journal={ Proceedings of the AAAI Fall Symposium on Relevance},
pages={1-5},
year={ 1994},
}
@ARTICLE{john,
author={ G. H. John, R. Kohavi, and K. Pfleger,},
title={Irrelevant features and the subset selection problem},
journal={Proceedings of Ninth National Conference on Artifical Intelligence},
pages={129-134},
year={ 1992},
}
@ARTICLE{almu,
author={ H. Almuallim, and T. G. Dietterich,},
title={Learning boolean concepts in the presence of many irrelevant features,},
journal={ Artifical Intelligence journal,},
volume={69},
pages={279-306},
year={ 1994},
} @ARTICLE{mich,
author={ Michael J. Pazzani,},
title={Searching for dependencies in Bayesian classifiers},
journal={ proceeding of the 5th Int. workshop on Artifical Intelligence and Statistics},
volume={V},
year={ 1995},
}
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top