(3.238.7.202) 您好!臺灣時間:2021/02/25 09:34
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果

詳目顯示:::

我願授權國圖
: 
twitterline
研究生:李得盛
研究生(外文):Li, Te-Sheng
論文名稱:應用統計與類神經網路模式於監督式分類問題
論文名稱(外文):Statistical and Neural Models for Supervised Classification
指導教授:蘇朝墩蘇朝墩引用關係
指導教授(外文):Su, Chao-Ton
學位類別:博士
校院名稱:國立交通大學
系所名稱:工業工程與管理系
學門:工程學門
學類:工業工程學類
論文種類:學術論文
論文出版年:2002
畢業學年度:90
語文別:英文
論文頁數:86
中文關鍵詞:分類k個最鄰近法線性辨別分析法馬氏距離基因演算法倒傳綱路放射基準機能綱路學習向量化綱路
外文關鍵詞:classificationk nearest neighborlinear discriminant analysisMahalanobis distancegenetic algorithmbackpropagation networkradial basis function networklearning vector quantization
相關次數:
  • 被引用被引用:7
  • 點閱點閱:515
  • 評分評分:系統版面圖檔系統版面圖檔系統版面圖檔系統版面圖檔系統版面圖檔
  • 下載下載:0
  • 收藏至我的研究室書目清單書目收藏:4
應用統計和類神經網路模式於監督式分類問題
研究生:李得盛 指導教授:蘇朝墩博士
國立交通大學
工業工程與管理系
摘要
多維度資料間充滿著模糊不清與許多變異,資料間的關係不易釐清。傳統上建立資料分類系統需藉助輸入變數所形成的規則,而在大量資料的情況下,這些規則的形成特別因難。在過去的數十年間,有許多監督型分類演算法已經成功地實行在多維度的資訊系統上,這些演算中可分為統計分類器和類神經網路分類器兩大類。本論文首先比較不同的統計分類器與類神網路分類器運用在原始的輸入變數上,統計分類器為k個最鄰近法 (KNN, k-nearest neighbor)、線性辨別分析法 (LDA, linear discriminant analysis) 和馬氏距離法(MD, Mahalanobis distance);類神經分類器依序為倒傳遞類神經網路 (BP, backpropagation)、放射基準機能網路 (RBF, radial basis function) 和學習向量量化網路 (LVQ, learning vector quantization)。這些方法的比較主要是以分類的正確率作為評估的準則。接下來,為了能從多維度的資訊系統中去除多餘的變數以增進分類的效率與正確率,本論文提出幾項有關變數縮減的方法論,能夠篩選出重要因子,利用較少的變數,即可完成分類的任務,並且不影響原先的分類正確率。他們是馬氏田口法(MTS, Mahalanobis-Taguchi method)、輸入節點選擇法 (INS, input node selection)、結合倒傳遞類神經網路與基因演算法的BP-GA (backpropagation-genetic algorithm)、結合放射基準機能網路與基因演算法的RBF-GA (radial basis function-genetic algorithm) 結合學習向量量化網路與基因演算法的LVQ-GA (learning vector quantization-genetic algorithm)。以上所提的縮減變數的方法將與統計上逐步辨別分析(stepwise discriminant analysis) 所得的結果作一比較。
本論文提供原始變數分類器的基礎,也提出縮減變數的方法論以及應用這些方法在實際個案上的分類結果。在分類結果上,醫學檢驗資料上,利用原始變數進行分析具有較佳結果的是馬氏距離(MD);利用縮減變數的模式具有較佳結果的是MTS, INS和LVQ-GA。例二利用原始變數所得較佳結果為LDA和RBF;利用縮減變數的模式具有較佳結果的是INS其次為MTS和LVQ-GA。因此,本論文所提的方法的確可刪除多餘變數,即可篩選重要變數以利未來的應用。最後,本論文將各方法的優缺點與實際應用上的注意事項作一比較與說明。
The relationships among multi-dimensional data with ambiguity and variation are difficult to explore. The traditional approach to building a data classification system requires the formulation of rules by which the input data can be analyzed. The formulation of such rules is very difficult with large sets of input data. Various algorithms for supervised classification of multi-dimensional data have been implemented in the past decades. Among these algorithms, statistical and neural classifiers are two major methodologies used in literature. In this dissertation, a comparison of different statistical and neural network algorithms using all the original input variables for classification is first presented. Three statistical classifier: k-nearest neighbor (KNN), linear discriminant analysis (LDA) and Mahalanobis distance (MD) are considered. Meanwhile, three types of neural classifiers: back-propagation (BP) neural network, radial basis function network (RBF), and learning vector quantization (LVQ) are also discussed in order to compare the accuracy of classification with those of using statistical classifiers. Next, in order to eliminate the redundant variables in multi-dimensional data set and increase classification efficiency and accuracy, we also herein proposed the variable reduction techniques. They are Mahalanobis-Taguchi system (MTS), input nodes selection (INS), BP combined with GA procedure (named BP-GA), RBF combined with GA (named RBF-GA) and LVQ combined with GA (LVQ-GA). A benchmark method, stepwise discriminant analysis, is employed to compare the accuracy with those of reduced models.
This dissertation includes an introduction of the theoretical background of the classifiers, their implementation procedures, and two case studies to evaluate their performance. Whether the full model or reduced model, both neural networks and statistical models are demonstrated to be efficient and effective methods for multi-dimensional data classification. In example one, MD outperforms the neural classifiers and statistical models for full models. Compared to the full models, MTS, INS and LVQ-GA reduced models result in the higher classification accuracy. In example two, LDA and RBF outperform the other models for full models. Compared to the full models, INS, MTS, and LVQ-GA reduced models result in the higher classification accuracy. It is also shown that the proposed variable reduction techniques indeed eliminate the redundant variables in the multi-dimensional system. Once a subset of variables is selected, the more important variables can be used for rule extraction or future application. In conclusion, the comparison and discussion of these approaches are presented in view of practical and theoretical consideration.
CONTENTS
Chinese Abstract ………………………………………………………………………… i
Abstract …………………………………………………………………………………. ii
Acknowledgements ……………………………………………………………………… iii
Contents ………………………………………………………………………………… iv
List of Figures …………………………………………………………………………… vii
List of Tables …………………………………………………………………………… viii
CHAPTER 1 INTRODUCTION ………………………..…………………………….. 1
1.1 Overview ……………..………………………………………………………… 1
1.2 Research Motivations ….………………………………………………………. 2
1.3 Research Objectives …………………………………………………………… 4
1.4 Organization ……………………………………………………………………. 7
CHAPTER 2 LITERATURE REVIEW……………………………………………... 8
2.1 Overview of classification approaches ………………..……………………… 8
2.2 Statistical and neural networks classifiers for data classification ……….….… 9
2.3 Reduced models for multi-dimensional data classification ………………..….. 11
2.3.1 Mahalanobis-Taguchi system …………………………………………… 11
2.3.2 Input node selection ……………………………………………………… 12
2.3.2 Other reduced models ……………………………………………………. 12
CHAPTER 3 METHODOLOGIES FOR FULL MODELS ………………………… 14
3.1 Introduction ……..………………………………………………………………. 14
3.2 Statistical classification approaches .…………………………………………… 14
3.2.1 k-nearest neighbor……………………………………………………….. 14
3.2.2 Linear discriminant analysis ……………………………………………. 16
3.2.3 Mahalanobis distance …………………………………………………… 17
3.3 Neural networks classification approaches …………………………………….. 20
3.3.1 Back-propagation neural network .………….……………………………. 21
3.3.2 Radial basis function network ………………………………………….... 23
3.3.3 Learning vector quantization ……………………………………………... 25
CHAPTER 4 METHODOLOGIES FOR REDUCED MODELS …….…………….. 27
4.1 Introduction …………………………………………………………………….. 27
4.2 Mahalanobis-Taguchi system ………………………………………..………… 27
4.3 Input nodes selection …………….……………………………………………… 29
4.4 GA-based reduced models ……………………………………………………… 31
CHAPTER 5 NUMERICAL ILLUSTRATIONS ……………………………………. 39
5.1 Introduction …………………………………………………………………….. 39
5.2 Example 1: Medical Data ………………………………………………………. 39
5.2.1 Problem description ..……………………….…………………………….. 39
5.2.2 Full modeling by statistical classifiers ……………………….…………. 42
5.2.3 Full modeling by neural network classifiers …………………….………. 47
5.2.4 Reduced modeling by Mahalanobis-Taguchi system ……………………. 47
5.2.5 Reduced modeling by input nodes selection ……………….……………. 50
5.2.6 Reduced modeling by GA-based reduced classifiers ……….…………… 52
5.3 Example 2: Glass Identification ………………………………………………. 55
5.3.1 Problem description ..……………………….……..…………………….. 55
5.3.2 Full modeling by statistical classifiers …………………………………… 55
5.3.3 Full modeling by neural network classifiers …………………………….. 57
5.3.4 Reduced modeling by Mahalanobis-Taguchi system …………………….. 57
5.3.5 Reduced modeling by input nodes selection ……………………………… 60
5.3.6 Reduced modeling by GA-based reduced classifiers …………………….. 61
CHAPTER 6 COMPARISON AND DISCUSSION…………………………………… 64
6.1 Introduction …………………………………………………………………… 64
6.2 Example 1: Medical Data …………………………………………………….. 64
6.2.1 Comparison among statistical and neural network classifiers in full modeling………………………………………………………………………. 64
6.2.2 Comparison among statistical and neural network classifiers in reduced modeling……………………………………………………………………….. 68
6.3 Example 2: Glass Identification ……………………………………………… 71
6.3.1 Comparison among statistical and neural network classifiers in full modeling………………………………………………………………………. 71
6.3.2 Comparison among statistical and neural network classifiers in reduced modeling……………………………………………………………………….. 73
CHAPTER 7 CONCLUSIONS …………………………………………………………76
REFERENCES …………………………………………………………………………. 80
REFERENCES
1. Andrews, R. and Diederich, J., 1996, Rules and networks. Proceedings of Rule Extraction Trained Artificial Neural Networks Workshop, AISB.
2. Andrews, R., Diederich, J. and Tickle, A. B., 1995, Survey and critique of techniques for extracting rules from trained artificial neural networks. Knowledge-Based System, 8, 373-389.
3. Antony, J., 2000, Multi-response optimization in industrial experiments using Taguchi’s quality loss function and principal component analysis. Quality Reliability Engineering International, 16, 3-8.
4. Backer, S. D., Naud, A. and Scheunders P., 1998, Non-linear dimensionality reduction techniques for unsupervised feature extraction. Pattern Recognition Letters, 19, 711-720.
5. Bertels, K., Jacques, J. M., Neuberg, L. and Gatot, L., 1999, Qualitative company performance evaluation: linear discriminant analysis and neural network models. European Journal of Operational Research, 115, 608-615.
6. Brown, C. W. and Lo, S. C. 1998, Chemical information based on neural network processing of Near-IR Spectra, Analytic Chemistry, 70, 2983-2990.
7. Brunzell, H. and Eriksson, J., 2000, Feature reduction for classification of multidimensional data. Pattern Recognition, 33, 1741-1748.
8. Bruzzone, L. and Prieto, D. F., 1999, A technique for the selection of kernel-function parameters in RBF neural networks for classification of remote-sensing images. IEEE Transactions on Geosciences and Remote Sensing, 37, 1179-1184.
9. Bruzzone, L. and Serpico, S. B., 1997, Classification of imbalanced remote-sensing data by neural networks. Pattern Recognition Letters, 18, 1323-1328.
10. Chen, G., McAvoy, T.J., Piovoso, L. and Michael, J., 1998, Multivariate statistical controller for on-line quality improvement, Journal of process control, 8, 139-149.
11. Chung, Y. Y. and Wong, M. T., 1998, High accuracy handwritten character recognition system using contour sequence moments. International Conference on Signal Processing Proceedings, 2, 1249-1252.
12. Collantes, E. R., Duta, R., Welsh, W. J. Zielinske, W. L. and Brower, J., 1997, Preprocessing of HPLC trace impurity patterns by wavelet packets for pharmaceutical fingerprinting using artificial neural networks. Analytical Chemistry, 69, 1392-1397.
13. Efthimiadu, I. and Tham, M. T., 1991, On-line statistical process control of chemical processes. IEE Conference Publication, 2, 700-705.
14. Feng, W. and Yang, X., 1998, Statistical correlation analysis and modeling in quality control, Press of Tsinghua University, 38, 63-66.
15. Fogel, D. B., Wasson, E. C., Boughton, E. M., Porto, V. W. and Angeline, P. J., 1998, Linear and neural models for classifying breast masses. IEEE Transactions on Medical Imaging, 17, 485-488.
16. Frame, A. J., Undrill, P. E., Cree, M. J., Olson, J. A., McHardy, K. C., Sharp, P. F. and Forrester, J. V., 1998, Comparison of computer-based classification methods applied to the detection of microaneurysms in ophthalmic fluoresce in angiograms. Computers in Biology and Medicine, 28, 225-238.
17. Ghosh, J. and Chakravarthy, S. V., 1994, Rapid kernel classifier. A link between the self-organizing feature map and the rapid basis function network. Journal of Intelligent Material Systems and Structures, 5, 211-219.
18. Giacinto, G., Roli, F. and Bruzzone, L., 2000, Combination of neural and statistical algorithms for supervised classification of remote-sensing images. Pattern recognition letters, 21, 385-397.
19. Guo, R. and Pandit, S. M., 1998, Automatic threshold selection based on histogram lodes and a discriminant criterion. Machine vision and applications, 10, 331-338.
20. Haering, N. and Da Vitoria Lobo, N., 1999, Features and classification methods to locate deciduous trees in images. Computer Vision and Image Understanding, 75, 133-149.
21. Hagberg, G., 1998, From magnetic resonance spectroscopy to classification of tumors. A review of pattern recognition methods. NMR in Biomedicine, 11, 148-156.
22. Hoya, T., 1998, Graph theoretic techniques for pruning data and their applications. IEEE Transactions on Signal processing, 46, 2574-2579.
23. Hush, D. R. and Home, B. G., 1993, Progress in supervised neural networks. IEEE Signal Processing Magazine, 10, 8-39.
24. Hwang, Y. S. and Bang, S. Y., 1997, Efficient method to construct a radial basis function neural network classifier. Neural Networks, 10, 1495-1503.
25. Jimenez, L. O. and Landgrebe, D. A., 1999, Hyperspectral data analysis and supervised feature reduction via projection pursuit. IEEE Transactions on Geoscience and Remote Sensing, 37, 2653-2661.
26. Jiyong, M. and Wen, G., 1998, Supervised learning Gaussian mixture model. Journal of Computer Science and Technology, 13, 471-474.
27. Jouny, I. and Shaw, S., 1999, Target identification using discriminative learning and feature extraction. Proceedings of SPIE - The International Society of Optical Engineering, 3718, 406-413.
28. Karayiannis, N. B., 1999, Reformulating learning vector quantization and radial basis neural networks. Fundamental Information, 37, 137-175.
29. Kaseko, M. S., Lo, Z. P. and Ritchie, S. G., 1994, Comparison of traditional and neural classifiers for pavement-crack detection. Journal of Transportation Engineering, 120, 552-569.
30. Kato, N., Suzuki, M., Omachi, S., Aso, H. and Nemoto, Y., 1999, A handwritten character recognition system using directional element feature and asymmetric Mahalanobis distance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21, 258-263.
31. Khan, S. I. and Ritchie, S. G., 1998, Statistical and neural classifiers to detect traffic operational problems on urban arterials. Transportation Research part C: Emerging Technologies, 6, 291-314.
32. Kwak, N. K. and Lee, C., 1997, A neural network application to classification of health status of HIV/AIDS patients. Journal of Medical System, 21, 87-97.
33. Loftsgaarden, D. O. and Queensberry, C. P., 1965, A nonparametric estimate of a multivariate density function. Annals of Mathematical Statistics, 36, 1049-1051.
34. Malhotra, M. K., Sharma, S. and Nair, S. S., 1999, Decision making using multiple models. European Journal of Operational Research, 114, 1-14.
35. McNitt-Gray, M. F., Huang, H. K. and Sayre, J. W., 1995, Feature selection in the pattern classification problem of digital chest radiograph segmentation. IEEE Transactions on Medical Imaging, 14, 537-547.
36. Meyer-Base, A. and Watzel, R., 1998, Transformation radial basis neural network for relevant feature selection. Pattern Recognition Letters, 19, 1301-1306.
37. Oja, E. and Valkealahti, K., 1996, Co-occurrence maps: quantizing multidimensional texture histograms. Pattern Recognition Letters, 17, 723-730.
38. Otsu, N., 1979, A threshold selection method from gray-level histograms. IEEE Transaction on Systems Man and Cybernetics. SMC 9, 62-66.
39. Pfurtscheller, G., Kalcher, J., and Nerper, C., 1996, On-line EEG classification during externally paced hand movements using a neural network-based classifier, Electroncephalogr. Clinical Neruophsiology, 99, 416-425.
40. Pham, D. T. and Bayro-Corrochano, E. J., 1995, Neural networks for classifying surface defects on automotive valve stem seals. International Journal of Machine Tools and Manufacture, 35, 1115-1124.
41. Raudys, S., 1998, Evolution and generalization of a single neurone: I. single-layer perceptron as seven statistical classifiers. Neural Networks, 11, 283-296.
42. Raudys, S., 1998, Evolution and generalization of a single neurone: II. complexity of statistical classifiers and sample size considerations. Neural Networks, 11, 297-313.
43. Raymer, M. L., Punch, W. F., Goodman, E. D., Kuhn, L. A. and Jain, A. K., 2000, Dimensionality reduction using genetic algorithms. IEEE Transactions on Evolutionary Computation, 4, 164-171.
44. Romero, R. D., Touretzky, D. S. and Thibadeau, R. H., 1997, Optical Chinese character recognition using probabilistic neural networks. Pattern Recognition, 30, 1279-1292.
45. Rumelhaart, D. E., McClelland, J. L. and PDP Research Group, 1986, Parallel distributed processing. MIT Press.
46. Santa Cruz, C. and Dorronsoro, J. R., 1998, Nonlinear discriminant algorithm for feature extraction and data classification. IEEE Transactions on Neural Networks, 9, 1370-1376.
47. Santoso, S., Powers, E. J., Grady, W. M. and Parsons, A. C., 2000, Power quality disturbance waveform recognition using wavelet-based neural classifier-Part I: Theoretical Foundation. IEEE Transactions on Power Delivery, 15, 222-232.
48. Sarle, W. S., 2000, How to measure importance of inputs? SAS Institute Inc., Cary, NC, USA. ftp://ftp.sas.com./put/neural/importance.html.
49. Selvi, S. T. Arumugam, S. and Ganesan, L., 2000, BIONET: an artificial neural network model for diagnosis of diseases. Pattern Recognition Letters, 21, 721-740.
50. Serpico, S. B, Bruzzone, L. and Roli, F., 1996, An experimental comparison of neural and statistical non-Parametric algorithms for supervised classification. Pattern Recognition Letters, 17, 1331-1341.
51. Sette, S., Boullart, L., Langenhove, L. V. and Kiekens, P., 1997, Optimizing the fiber-to-yarn production process with a combined neural network/genetic algorithm approach. Textile Research Journal, 67, 84-92.
52. Shah, N. K. and Gemperline, P. J., 1990, Combination of the Mahalanobis distance and residual variance pattern recognition techniques for classification of Near-Infrared reflectance spectra. Analytic Chemistry, 62, 465-470.
53. Siedlecki, W. and Sklansky, J., 1989, A note on genetic algorithms for large-scale feature selection. Pattern Recognition Letters, 10, 335-347.
54. Sujana, H., Swarnamani, S., and Suresh, S., 1996, Application of artificial neural networks for the classification of liver lesions by image texture parameters, Ultrasound in Medicine & Biology, 22, 1177-1181.
55. Sutter, J. M. and Jurs, P. C., 1997, Neural network classification and quantification of organic vapors based on fluorescence data from a fiber-optic sensor array. Analytic Chemistry, 69, 856-862.
56. Taguchi, Genichi, 1998, Mathematics for Quality Engineering. Journal of Quality Engineering Forum, 6, 5-10.
57. Tahani, H., Plummer, B. and Hemamalini, N. S., 1996, a new data reduction algorithm for pattern classification. IEEE conference, 3446-3449.
58. Torrieri, D., 1999, The eigenspace separation transform for neural-network classifiers. Neural Networks, 12, 419-427.
59. Tsai, C. Y., 2000, An iterative feature reduction algorithm for probabilistic neural networks. Omega, 28, 513-524.
60. Tsukimoto, H., 2000, Extraction rules from trained neural networks. IEEE Transactions on Neural Networks, 11, 377-389.
61. Wang, X. and Paliwal, K. K., 1999, Using minimum classification error training in dimensionality reduction. IEEE conference, 338-345.
62. Younis, K. S., Rogers, T. K. and DeSimio, M. P., 1996, Vector quantization based on dynamic adjustment of Mahalanobis distance, Aerospace and Electronics Conference, Proceedings of IEEE of 1996, 2, 858-862.
63. Zhang, S., Franke, J. E., Niemczyk, T. M., Haaland, D. M., Cox, J. N. and Banerjee, I. 1997, Testing of a rapid fault detection model for quality control: Borophoshposilicate glass thin films monitored by infrared absorption spectroscopy, Journal of Vacuum Science Technology, Part B 15, 955-960.
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top
系統版面圖檔 系統版面圖檔