臺灣博碩士論文加值系統

資訊科技的蓬勃發展導致資料快速地增加，而這些資料中可能隱含有價值的資訊，機器學習與資料探勘是能從資料中萃取知識與法則的工具。而分類是其中很重要的議題，透過建構好的分類器將資料正確地歸類是很重要的。然而過去許多著名的分類器如支援向量機、類神經網路與K個最臨近點分法有很好的分類結果，但缺乏以人能理解的方式呈現。因此本研究提出融入後設認知策略的複數模糊認知圖 (Metacognitive complex fuzzy cognitive map, McCFCM)，其結合後設認知 (Metacognitive)、模糊認知圖 (Fuzzy cognitive map, FCM)與複數型模糊集合 (Complex fuzzy sets)，並應用於分類問題中。在MCCFCM建模上，分成參數學習與結構學習。參數學習中，使用標準粒子群最佳化演算法 (Standard particle swarm optimization, SPSO)來調整複數型模糊集合位置與FCM的連結權重；結構學習中，使用二元粒子群演算法 (Binary particle swarm optimization, BPSO)調整FCM連結架構。為了建出更強健的器分類器，本研究使用一對多 (One-against-all, OAA)的訓練方法，同時使用費雪分值 (Fisher score, F-score)挑選出重要的屬性。本研究使用加州大學爾灣分校 (University of California-Irvine)的機器學習資料庫中10個資料集來驗證本研究提出之方法，並與其他著名的研究方法比較分類結果。

The rapid development of information system has led to increase a large number of data, and these data usually imply valuable information, for which machine learning and data mining are useful tools to extract knowledge and rules from data. Classification is one of important issues, and it is important to construct a good classifier that can classify the data correctly. Although some famous classifiers had been presented, such as support vector machine (SVM), artificial neuro network (ANN), and K-nearest neighbors (KNN), they lack of being understood by people. Therefore, in this study, to classification problems, we propose a metacognitive complex fuzzy cognitive map (McCFCM) that combines metacognitive, complex fuzzy sets and fuzzy cognitive map. The modeling of McCFCM classifier comprises the phases of parameter learning and structure learning. In the parameter learning phase, the method of standard particle swarm optimization (SPSO) is use to adjust the location of the complex fuzzy sets and the weights of all the connections in FCM; In the structure learning phase, the algorithm of binary particle swarm optimization (BPSO) is used to establish or erase some connections in FCM. In order to build a more robust classifier, we also use one-against-all (OAA) to decompose the dataset whose data are with multiple classes into several binary-class subsets, and the Fisher score (F-score) is used to pick important features for the problem of classification. In this study, ten datasets from the University of California-Irvine (UCI) machine learning repository have been used to evaluate the performance by the proposed McCFCM classifier, whose results are compared with those by other noted classification algorithms.

中文摘要 i
英文摘要 ii
誌謝 iii
目錄 iv
圖目錄 vi
表目錄 vii
符號說明 ix
第一章 緒論 1
1.1 研究背景 1
1.2 研究動機與目的 1
1.3 研究方法 2
第二章 文獻探討 4
2.1 特徵選取 4
2.1.1 Filter類型之特徵選取 4
2.1.2 Wrapper類型之特徵選取 5
2.1.3 費雪分值 5
2.2 模糊集合 7
2.2.1 模糊集合的緣起 7
2.2.2 模糊集合之型態 8
2.2.3 複數型模糊集合 9
2.3 後設認知 11
2.3.1 後設認知的緣起 11
2.3.2 後設認知相關研究 11
2.3.3 後設認知於機器學習之應用 13
2.4 模糊認知圖 14
2.4.1 模糊認知圖定義 14
2.4.2 模糊認知圖建構 15
2.4.3 模糊認知圖計算 17
2.5 最佳化演算法 18
2.5.1 粒子群最佳化演算法 19
2.5.2 標準粒子群最佳化演算法 20
2.5.3 二元粒子群最佳化演算法 23
第三章 系統設計與架構 26
3.1 融入後設認知的模糊認知圖之架構與設計 26
3.2 融入後設認知的模糊認知圖之學習策略 28
3.2.1 McCFCM之客體層面 31
3.2.2 McCFCM之後設層面 32
第四章 實驗 36
4.1 實驗一：威斯康辛州乳癌(原始)資料集 36
4.2 實驗二：印地安人糖尿病 41
4.3 實驗三：鳶尾花資料集 45
4.4 實驗四：脊柱資料集(3類) 48
4.5 實驗五：脊柱資料集(2類) 52
4.6 實驗六：國會投票紀錄資料集 55
4.7 實驗七：心臟病資料集 60
4.8 實驗八：乳房組織資料集 64
4.9 實驗九：生育力資料集 67
4.10 實驗十：紅酒資料集 70
4.11 實驗結果小結 74
第五章 結論與討論 75
5.1. 後設認知之探討 75
5.2. 模糊認知圖之探討 75
5.3. SPSO-BPSO多重式演算法探討 76
5.4. 特徵選取之探討 76
5.5. 限制 76
5.6. 結論 76
第六章 未來研究方向 78
6.1. 特徵選取演算法隻改良 78
6.2. 最佳化演算法之改良 78
6.3. 模糊認知圖之改良 78
參考文獻 79

[1] R. Karchin, K. Karplus, and D. Haussler, “Classifying G-protein coupled receptors with support vector machines,” Bioinformatics, vol. 18, no. 1, pp. 147-159, Jan. 2002.
[2] U. M. Braga-Neto and E. R. Dougherty, "Is cross-validation valid for small-sample microarray classification?," Bioinformatics, vol. 20, pp. 374-380, 2004.
[3] H. Mamitsuka, "Selecting features in microarray classification using ROC curves," Pattern Recognition, vol. 39, pp. 2393-2404, 2006.
[4] A. Lorenz, M. Blum, H. Ermert, and T. Senge, “Comparison of different neuro-fuzzy classification systems for the detection of prostate cancer in ultrasonic images,” IEEE Proceedings in Ultrasonics Symposium, vol. 2, pp. 1201-1204, 1997.
[5] S. M. Odeh, "Using an Adaptive Neuro-Fuzzy Inference System(AnFis) Algorithm for Automatic Diagnosis of Skin Cancer," Journal of Communication and Computer, vol. 8, pp. 751-755, 2011.
[6] A. Das and M. Bhattacharya, “A study on prognosis of brain tumors using fuzzy logic and genetic algorithm based techniques,” International Joint Conference on Bioinformatics, Systems Biology and Intelligent Computing, pp. 348-351, 2009.
[7] A. Das and M. Bhattacharya, “GA based neuro fuzzy techniques for breast cancer identification,” International Machine Vision and Image Processing Conference, pp. 136-141, 2008.
[8] E. Papageorgiou, P. Spyridonos, D. T. Glotsos, C. D. Stylios, P. Ravazoula, G. Nikiforidis, et al., "Brain tumor characterization using the soft computing technique of fuzzy cognitive maps," Applied Soft Computing, vol. 8, pp. 820-828, 2008.
[9] J. Huang, X. Shao, and H. Wechsler, "Face pose discrimination using support vector machines (SVM)," in Pattern Recognition, 1998. Proceedings. Fourteenth International Conference on, 1998, pp. 154-156.
[10] M. S. Bartlett, G. Littlewort, C. Lainscsek, I. Fasel, and J. Movellan, "Machine learning methods for fully automatic recognition of facial expressions and facial actions," in Systems, Man and Cybernetics, 2004 IEEE International Conference on, 2004, pp. 592-597.
[11] A. U. Khan, T. Bandopadhyaya, and S. Sharma, "Classification of stocks using self organizing map," International Journal of Soft Computing Applications, vol. 4, pp. 19-24, 2009.
[12] K. Schierholt and C. H. Dagli, “Stock market prediction using different neural network classification architectures,” in Proc. IEEE/IAFE 1996 Conf. Comput. Intell. Financial Eng., New York, 1996, pp. 72–78.
[13] C. J. Huang, D. X. Yang, and Y. T. Chuang, “Application of wrapper approach and composite classifier to the stock trend prediction,” Expert Systems with Applications, vol. 34, no. 4, pp. 2870-2878, 2008.
[14] C. T. Lin, and C. S. George Lee, "Neural-Network-Based Fuzzy Logic Control and Decision System," IEEE Transaction on Computer, Vol. 40, No. 12, December 1991.
[15] J.-S. Jang, "ANFIS: Adaptive-network-based fuzzy inference system," IEEE Transactions on Systems, Man and Cybernetics, vol. 23, pp. 665-685, 1993.
[16] M. L. Huang, H. Y. Chen, and J. J. Huang, “Glaucoma detection using adaptive neuro-fuzzy inference system,” Expert Systems with Applications, vol. 32, no. 2, pp. 458-468, 2007.
[17] V. Vapnik, The Nature of Statistical Learning Theory, Springer, New York, 1995.
[18] P. N. Tan, M. Steinbach, and V. Kumar, Introduction to data mining, Addison-Wesley, 2006.
[19] Jiang L X, Cai Z H, and Wang D H, et al.. “Survey of improving k-nearest-neighbor for classification [C].” Fourth International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), Haikou, China, Aug 24-27, 2007: 679-683.
[20] S. Kirkpatrick, D. G. Jr., and M. P. Vecchi, "Optimization by simulated annealing," science, vol. 220, pp. 671-680, 1983.
[21] R. Eberhart and J. Kennedy, "A new optimizer using particle swarm theory," Proceedings of the Sixth International Symposium on Micro Machines and Human Science, pp. 39-43, 1995.
[22] J. Kennedy and R. Eberhart, "Particle swarm optimization," in Neural Networks, 1995. Proceedings., IEEE International Conference on, 1995, pp. 1942-1948.
[23] D. Karaboga, "An idea based on honey bee swarm for numerical optimization," Techn. Rep. TR06, Erciyes Univ. Press, Erciyes, 2005.
[24] D. E. Goldberg, "Genetic algorithms in search, optimization, and machine learning," 1989.
[25] D. Bratton and J. Kennedy, "Defining a standard for particle swarm optimization," in Swarm Intelligence Symposium, 2007. SIS 2007. IEEE, 2007, pp. 120-127.
[26] M. Clerc: Standard Particle Swarm Optimization from 2006 to 2011. http://www.particleswarm.info
[27] M. Clerc: Particle Swarm Optimization. International Scientific and Technical Encyclopedia, 2006.
[28] J. Kennedy and R. C. Eberhart, "A discrete binary version of the particle swarm algorithm," in Systems, Man, and Cybernetics, 1997. Computational Cybernetics and Simulation., 1997 IEEE International Conference on, 1997, pp. 4104-4108.
[29] M. A. Khanesar, M. Teshnehlab, and M. A. Shoorehdeli, "A novel binary particle swarm optimization," in Control & Automation, 2007. MED'07. Mediterranean Conference on, 2007, pp. 1-6.
[30] B. Kosko, "Fuzzy cognitive maps," International journal of man-machine studies, vol. 24, pp. 65-75, 1986.
[31] E. I. Papageorgiou and J. L. Salmeron, "A Review of Fuzzy Cognitive Maps research during the last decade," 2012.
[32] D. Ramot, R. Milo, M. Friedman, and A. Kandel, "Complex fuzzy sets," Fuzzy Systems, IEEE Transactions on, vol. 10, pp. 171-186, 2002.
[33] D. Ramot, M. Friedman, G. Langholz, and A. Kandel, "Complex fuzzy logic," Fuzzy Systems, IEEE Transactions on, vol. 11, pp. 450-461, 2003.
[34] S. Dick, "Toward complex fuzzy logic," Fuzzy Systems, IEEE Transactions on, vol. 13, pp. 405-414, 2005.
[35] Chunshien Li and Tai-Wei Chiang, “Complex neuro-fuzzy ARIMA forecasting — a new approach using complex fuzzy sets,” IEEE Transactions on Fuzzy Systems, vol. 21, no. 3, pp. 567-584, June 2013.
[36] J. H. Flavell, "Metacognition and cognitive monitoring," American psychologist, vol. 34, pp. 906-911, 1979.
[37] J. Doak, An Evaluation of Feature Selection Methods and Their Application to Computer Security: University of California, Computer Science, 1992.
[38] J. Liang, S. Yang, and A. Winstanley, "Invariant optimal feature selection: A distance discriminant and feature ranking based solution," Pattern Recognition, vol. 41, pp. 1429-1439, 2008.
[39] C. M. Bishop, Neural networks for pattern recognition: Oxford university press, 1995.
[40] X. He, D. Cai, and P. Niyogi, "Laplacian score for feature selection," Advances in Neural Information Processing Systems, vol. 18, p. 507, 2006.
[41] A. R. Webb, Statistical pattern recognition: Wiley, 2003.
[42] R. Kohavi and G.H. John, “Wrappers for Feature Subset Selection,” Artificial Intelligence, vol. 97, nos. 1-2, pp. 273-324, 1997.
[43] D. W. Aha and R. L. Bankert, "A comparative evaluation of sequential feature selection algorithms," in Learning from Data, ed: Springer, 1996, pp. 199-206.
[44] H. Chai and C. Domeniconi, "An evaluation of gene selection methods for multi-class microarray data classification," in Proceedings of the Second European Workshop on Data Mining and Text Mining in Bioinformatics, 2004, pp. 3-10.
[45] L. A. Zadeh, "Fuzzy sets," Information and control, vol. 8, pp. 338-353, 1965.
[46] D. Ramot, R. Milo, M. Friedman, and A. Kandel, "Complex fuzzy sets," Fuzzy Systems, IEEE Transactions on, vol. 10, pp. 171-186, 2002.
[47] D. Ramot, M. Friedman, G. Langholz, and A. Kandel, "Complex fuzzy logic," Fuzzy Systems, IEEE Transactions on, vol. 11, pp. 450-461, 2003.
[48] S. Dick, "Toward complex fuzzy logic," Fuzzy Systems, IEEE Transactions on, vol. 13, pp. 405-414, 2005.
[49] Chunshien Li and Tai-Wei Chiang, “Complex neuro-fuzzy ARIMA forecasting — a new approach using complex fuzzy sets,” IEEE Transactions on Fuzzy Systems, vol. 21, no. 3, pp. 567-584, June 2013.
[50] T. O. Nelson and L. Narens, "Metamemory: A theoretical framework and new findings," The psychology of learning and motivation, vol. 26, pp. 125-141, 1990.
[51] M. T. Cox, "Metacognition in computation: A selected research review," Artificial Intelligence, vol. 169, pp. 104-141, 2005.
[52] G. S. Babu and S. Suresh, "Parkinson’s Disease Prediction Using Gene Expression-A Projection Based Learning Meta-cognitive Neural Classifier Approach," Expert Systems with Applications, 2012.
[53] R. Savitha, S. Suresh, and N. Sundararajan, "A meta-cognitive learning algorithm for a fully complex-valued relaxation network," Neural Networks, vol. 32, pp. 209-218, 2012.
[54] G. Sateesh Babu and S. Suresh, "Meta-cognitive Neural Network for classification problems in a sequential learning framework," Neurocomputing, vol. 81, pp. 86-96, 2012.
[55] G. S. Babu and S. Suresh, “Meta-cognitive rbf network and its projection based learning algorithm for classification problems,” Applied Softcomputing, vol. 13, no. 1, pp. 654 – 666, 2013.
[56] R. Savitha, S. Suresh, and N. Sundararajan, "A meta-cognitive learning algorithm for a fully complex-valued relaxation network," Neural Networks, vol. 32, pp. 209-218, 2012.
[57] G. Sateesh Babu and S. Suresh, "Meta-cognitive Neural Network for classification problems in a sequential learning framework," Neurocomputing, vol. 81, pp. 86-96, 2012.
[58] R. Savitha, S. Suresh, and N. Sundararajan, "A meta-cognitive learning algorithm for a fully complex-valued relaxation network," Neural Networks, vol. 32, pp. 209-218, 2012.
[59] G. Sateesh Babu and S. Suresh, "Meta-cognitive Neural Network for classification problems in a sequential learning framework," Neurocomputing, vol. 81, pp. 86-96, 2012
[60] E. I. Papageorgiou, K. E. Parsopoulos, C. S. Stylios, P. P. Groumpos, and M. N. Vrahatis, "Fuzzy cognitive maps learning using particle swarm optimization," Journal of Intelligent Information Systems, vol. 25, pp. 95-121, 2005.
[61] K. E. Parsopoulos, E. I. Papageorgiou, P. Groumpos, and M. N. Vrahatis, "A first study of fuzzy cognitive maps learning using particle swarm optimization," in Evolutionary Computation, 2003. CEC'03. The 2003 Congress on, 2003, pp. 1440-1447.
[62] E. I. Papageorgiou and P. P. Groumpos, "A new hybrid method using evolutionary algorithms to train fuzzy cognitive maps," Applied Soft Computing, vol. 5, pp. 409-431, 2005.
[63] 人工智慧—最佳化方法-陳鍾誠的網站
(http://ccckmit.wikidot.com/so:introduction) Accessed June 8, 2013.
[64] P. M. Murphy and D. W. Aha, UCI Repository of Machine Learning Databases, University of California, Department of Information and Computer Science, 1994.(http://archive.ics.uci.edu/ml/datasets.html) Accessed June 8, 2013.
[65] G. I. Salama, M. B. Abdelhalim, and M. A. Zeid, “Breast cancer diagnosis on three different datasets using multi-classifiers,”International Journal of Computer and Information Technology, vol. 1, no. 1, pp. 36-43, Sep. 2012.
[66] C. M. Bishop and M. E. Tipping, "Variational relevance vector machines," in Proceedings of the Sixteenth conference on Uncertainty in artificial intelligence, 2000, pp. 46-53.
[67] A. Marcano-Cedeño, J. Quintanilla-Domínguez, and D. Andina, “WBCD breast cancer database classification applying artificial metaplasticity neural network,” Expert Systems with Applications, vol. 38, no. 8, pp. 9573-9579, Aug. 2011.
[68] J. J. Rodriguez, L. I. Kuncheva, and C. J. Alonso, "Rotation forest: A new classifier ensemble method," Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 28, pp. 1619-1630, 2006.
[69] M. Karabatak, and M. Cevdet Ince, “An expert system for detection of breast cancer based on association rules and neural network,” Expert Systems with Applications, vol. 36, no. 2, pp. 3465-3469, Mar. 2009.
[70] S.-j. Wang, A. Mathew, Y. Chen, L.-f. Xi, L. Ma, and J. Lee, "Empirical analysis of support vector machine ensemble classifiers," Expert Systems with Applications, vol. 36, pp. 6466-6476, 2009.
[71] D. Opitz and R. Maclin, “Popular ensemble methods: an empirical study,” Journal of Artificial Intelligence Research, vol. 11, pp. 169-198, 1999.
[72] T. Kiyan and T. Yildirim, “Breast cancer diagnosis using statistical neural networks,” IU-Journal of Electrical & Electronics Engineering, vol. 4, no. 2, pp. 1149-1153, Jun. 2004.
[73] M. Ashraf, K. Le, and X. Juang, “Iterative weighted k-NN for constructing missing feature values in Wisconsin breast cancer dataset,” 2011 3rd International Conference on Data Mining and Intelligent Information Technology Applications (ICMiA), pp. 23-27, Oct. 2011.
[74] O. Pujol and D. Masip, "Geometry-based ensembles: toward a structural characterization of the classification boundary," Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 31, pp. 1140-1146, 2009.
[75] M. Fallahnezhad, M. H. Moradi, and S. Zaferanlouei, “A hybrid higher order neural classifier for handling classification problems,” Expert Systems with Applications, vol. 38, no. 1, pp. 386-393, Jan. 2011.
[76] G. H. John and P. Langley, "Estimating continuous distributions in Bayesian classifiers," in Proceedings of the Eleventh conference on Uncertainty in artificial intelligence, 1995, pp. 338-345.
[77] K. M. Salama and A. A. Freitas, “ABC-Miner: An ant-based bayesian classification algorithm,” Swarm Intelligence, Lecture Notes in Computer Science, vol. 7461, pp. 13-24, Sep. 2012.
[78] L. Li, “Perceptron learning with random coordinate descent,” Computer Science Technical Report CaltechCSTR:2005.006, California Institute of Technology, Aug. 2005.
[79] M. Grochowski and W. Duch, “Fast projection pursuit based on quality of projected clusters,” Adaptive and Natural Computing Algorithms, Lecture Notes in Computer Science, vol. 6594, pp. 89-97, Apr. 2011.
[80] S. Şahan, K. Polat, H. Kodaz, and S. Güneş, "The medical applications of attribute weighted artificial immune system (AWAIS): diagnosis of heart and diabetes diseases," in Artificial Immune Systems, ed: Springer, 2005, pp. 456-468.
[81] Y. Wu and S. Ng, "Combining Neural Learners with the Naive Bayes Fusion Rule for Breast Tissue Classification," in Industrial Electronics and Applications, 2007. ICIEA 2007. 2nd IEEE Conference on, 2007, pp. 709-713.
[82] J. Estrela da Silva, Dr J. P. Marques de Sá, and J. Jossinet, “Classification of breast tissue by electrical impedance spectroscopy,” Medical and Biological Engineering and Computing, vol. 38, no. 1, pp. 26-30, 2000.
[83] D. Gil, J. L. Girela, J. D. Juan, M. J. Gomez-Torres, and M. Johnsson, “Predicting seminal quality with artificial intelligence methods,” Expert Systems with Applications, vol. 39, no. 16, pp. 12564-12573, Nov. 2012
[84] J.R. Quinlan, C4.5: Programs for machine learning, Morgan Kaufman Publishers Inc., 1993.
[85] J. Bacardit and J.M. Garrell, “Bloat control and generalization pressure using the minimum description length principle for a pittsburgh approach learning classifier system,” Proceedings of the 2003-2005 International conference on Learning Classifier Systems, pp. 59-79, 2007.
[86] H. Mohamadi, J. Habibi, M. S. Abadeh, and H. Saadi, “Data mining with a simulated annealing based fuzzy classification system,” Pattern Recognition, vol. 41, no.5, pp. 1824-1833, May 2008.
[87] K. S. Park and S. H. Kim, "Fuzzy cognitive maps considering time relationships," International Journal of Human-Computer Studies, vol. 42, pp. 157-168, 1995.