臺灣博碩士論文加值系統

資料探勘的技術幫助我們從資料中找出內隱特徵與外顯的知識價值。隨著時代的進步，更多的資料探勘技術被成功的提出並廣泛的討論。而支援向量機則是目前類神經網路中首屈一指的分類器，並被廣泛的應用在信用卡評估、財務合作、信用借貸、文字分類、手寫判別、語音辨識與生物資訊。在應用在許多分類問題上，資料探勘的技術已趨於成熟，而演算法也不斷地再創新，但大多數的方法都屬於資料導向，這造成了學術與業界巨大的鴻溝。為了縮小這些差距，我們採取先備專業知識。我們應用專業知識於資料集中，發現某些屬性與類別中存在著單調性。當考慮資料的先備知識，我們需要加入一些單調性限制式於模型當中，如支援向量機。而合併單調性限制式與分類技術這可讓外顯知識更加合理化。在文獻探討中，過去的研究假設輸入資料是具有順序性且誤差是被忽略的，但這些假設無法在現實中被避免。因此，為了克服這些缺點，在本研究我們根據資料單調性，提出一個新的具有單調性限制模型。我們期待本研究所提出的方法實驗結果會比原始支援向量機還要好。

Data mining techniques support us to find out the hidden patterns and to extract valuable knowledge from databases. With the process of the time, more and more data mining methods have been successively proposed and widely discussed. Support vector machine (SVM) is a state-of-the-art artificial neural network (ANN) based on statistical learning. SVM has been widely applied in many fields, such as credit rating, forecasting corporate financial distress, consumer loan evaluation, text categorization, handwriting recognition, speaker verification, and bioinformatics. In many applications of classification problems, the data mining techniques are relatively mature as it comes to algorithm innovation. However, most of them are data-driven, and it causes a big gap between academic and business goal. To alleviate the predicament, we take into account a priori domain knowledge. In many real-world problems, we can see that there are some monotonicity relationships between the class and some of the attributes. When considering such prior knowledge about the data, one needs to add some monotonicity constraints into the classification model, as SVM. It has been shown that a classification technique incorporated with monotonicity constraints can obtain explicit knowledge that is more comprehensible. Some research assumed the input data has an ordinal and the bias term is ignored. However, this assumption may not be valid in practice. Therefore, to overcome these shortcomings, in this study we propose a new SVM model with monotonicity constraints that are inequalities and are based on the partial order in the input data. The results of the experiments show that the proposed method, which considers the prior domain knowledge of monotonicity, performs better than the original SVM.

摘要 I
ABSTRACT II
誌謝 III
CONTENTS V
List of Table VII
List of Figure VIII
Chapter 1 Introduction 1
1.1 Background and motivation 1
1.2 Objectives 3
1.3 Thesis Organization 3
Chapter 2 Literature Review 5
2.1 Support Vector Machines 5
2.1.1 Linear SVM 7
2.1.2 Non-Linear SVM 9
2.1.3 Mercer’s condition 10
2.2 Multi-Classes SVM 11
2.2.1 One-Against-All 11
2.2.2 One-Against-One 12
2.3 Classification with Monotonicity Constraints 13
Chapter 3 Research Methodology 16
3.1 Data preprocessing 17
3.2 Concept of monotonicity 18
3.3 Derivation of the Monotonicity Constrained SVM (MC-SVM) Model 20
3.3.1 Monotonic Constrained SVM for liner model 20
3.3.2 Monotonicity Constrained SVM for non-liner model 24
3.4 Computing b 27
3.5 MC-SVM Algorithm 28
Chapter 4 Experiment and result analysis 32
4.1 Environment of Experiments and Data Collection 32
4.2 Performance measures 33
4.3 Experiment step 35
4.4 Experiment result 36
Chapter 5 Conclusion and feature work 43
5.1 Conclusion 43
5.2 Feature work 43
Reference 45

Archer, N. P., ＆ Wang, S. H. (1993). Learning Bias in Neural Networks and an Approach to Controlling Its Effects in Monotonic Classification. Ieee Transactions on Pattern Analysis and Machine Intelligence, 15(9), 962-966.
Arun Kumar, M., ＆ Gopal, M. (2010). A Comparison Study on Multiple Binary-Class SVM Methods for Unilabel Text Categorization. Pattern Recognition Letters, 31(11), 1437-1444.
Boser, B. E., Guyon, I. M., ＆ Vapnik, V. N. (1992). A training algorithm for optimal margin classifiers. Paper presented at the Proceedings of the fifth annual workshop on Computational learning theory, Pittsburgh, Pennsylvania, United States.
Burges, C. J. C. (1998). A tutorial on Support Vector Machines for pattern recognition. Data Mining and Knowledge Discovery, 2(2), 121-167.
Cao, L., Zhang, C., Yu, P. S., ＆ Zhao, Y. (2010). D 3 M Methodology. 27-47. doi: 10.1007/978-1-4419-5737-5_2
Courant, R., ＆ Hilbert, D. (1953). Methods of mathematical physics (1st English ed.). New York,: Interscience Publishers.
Courant, R., ＆ Hilbert, D. (1970). Methods of Mathematical Physics (Vol. I, II). New York: Wiley Interscience.
Cristianini, N., ＆ Shawe-Taylor, J. (2000). Support Vector Machines and other kernel-based learning methods Cambridge University Press, 2000 - Ordering Information.
Davis, S. M., ＆ Botkin, J. W. (1994). The monster under the bed : how business is mastering the opportunity of knowledge for profit. New York: Simon ＆ Schuster.
Decherchi, S., Ridella, S., Zunino, R., Gastaldo, P., ＆ Anguita, D. (2010). Using Unsupervised Analysis to Constrain Generalization Bounds for Support Vector Classifiers. Ieee Transactions on Neural Networks, 21(3), 424-438. doi: Doi 10.1109/Tnn.2009.2038695
Dembczyński, K., Kotłowski, W., ＆ Słowiński, R. (2008). Ensemble of Decision Rules for Ordinal Classification with Monotonicity Constraints
Rough Sets and Knowledge Technology. In G. Wang, T. Li, J. Grzymala-Busse, D. Miao, A. Skowron ＆ Y. Yao (Eds.), (Vol. 5009, pp. 260-267): Springer Berlin / Heidelberg.
Doumpos, M., ＆ Pasiouras, F. (2005). Developing and testing models for replicating credit ratings: A multicriteria approach. Computational Economics, 25, 327–341.
Doumpos, M., ＆ Zopounidis, C. (2009). Monotonic Support Vector Machines for Credit Risk Rating. new Mathematics and Natural Computation 5(3), 557 - 570.
Doumpos, M., Zopounidis, C., ＆ Golfinopoulou, V. (2007). Additive Support Vector Machines for Pattern Classification. IEEE Transactions on Systems, Man and Cybernetics - Part B: Cybernetics, 37(3), 540-550.
Duivesteijn, W., ＆ Feelders, A. (2008). Nearest Neighbour Classification with Monotonicity Constraints , . Paper presented at the 2008 European Conference on Machine Learning and Knowledge Discovery in Databases, Antwerp, Belgium Springer-Verlag.
Evgeniou, T., Boussios, C., ＆ Zacharia, G. (2005). Generalized Robust Conjoint Estimation. Marketing Science, 24(3), 415-429. doi: 10.1287/mksc.1040.0100
Evgeniou, T., C., ＆ Boussios, e. a. (2005). Generalized Robust Conjoin Estimation. Marketing Science, 24(3), 415 - 429.
Falck, T., Suykens, J., ＆ De Moor, B. (2009). Robustness Analysis for Least Squares Kernel Based Regression: an Optimization Approach. Paper presented at the The 48th IEEE Conference on Decision and Control (CDC 2009) Shanghai, China.
Gamarnik, D. (1998). Efficient learning of monotone concepts via quadratic optimization. In: Proceedings of the eleventh annual conference on computational learning theory, ACM Press, New York., 134–143.
Greco, S., Matarazzo, B., ＆ Słowiński, R. (1998). A new rough set approach to evaluation of bankruptcy risk. In: Zopounidis, C. (ed.) Operational Tools in the Management of Financial Risks, Kluwer Academic Publishers, Dordrech., 121–136.
Gruber, C., Gruber, T., Krinninger, S., ＆ Sick, B. (2010). Online Signature Verification With Support Vector Machines Based on LCSS Kernel Functions. IEEE Transactions on Systems, Man and Cybernetics - Part B: Cybernetics, 40(4), 1088-1101.
Hu, Q., Che, X., Zhang, L., Zhang, D., Guo, M., ＆ Yu, D. (2011). Rank Entropy Based Decision Trees for Monotonic Classification. Knowledge and Data Engineering, IEEE Transactions on, PP(99), 1-1.
Hua, Z. S., Wang, Y., Xu, X. Y., Zhang, B., ＆ Liang, L. (2007). Predicting Corporate Financial Distress Based on Integration of Support Vector Machine and Logistic Regression. Expert Systems with Applications, 33(2), 434-440.
Huang, W., Nakamoria, Y., ＆ Wang, S. Y. (2005). Forecasting stock market movement direction with support vector machine. Computers ＆ Operations Research, 32, 2513–2522.
Huang, Z., Chen, H., Hsu, C.-J., Chen, W.-H., ＆ Wu, S. (2004). Credit rating analysis with support vector machines and neural networks: a market comparative study. Decision Support Systems, 37(4), 543-558. doi: 10.1016/s0167-9236(03)00086-1
Khemchandani, R., Jayadeva, ＆ Chandra, S. (2009). Knowledge based proximal support vector machines. European Journal of Operational Research, 195(3), 914-923. doi: DOI 10.1016/j.ejor.2007.11.023
Kim, H. S., ＆ Sohn, S. Y. (2010). Support vector machines for default prediction of SMEs based on technology credit. European Journal of Operational Research, 201(3), 838-846. doi: DOI 10.1016/j.ejor.2009.03.036
Kramer, K. A., Hall, L. O., Goldgof, D. B., Remsen, A., ＆ Luo, T. (2009). Fast Support Vector Machines for Continuous Data. Ieee Transactions on Systems Man and Cybernetics Part B-Cybernetics, 39(4), 989-1001. doi: Doi 10.1109/Tsmcb.2008.2011645
Lauer, F., Suen, C. Y., ＆ Bloch, G. (2007). A Trainable Feature Extractor for Handwritten Digit Recognition Pattern Recognition, 40(6), 1816-1824.
Li, S. T., Shiue, W., ＆ Huang, M. H. (2006). The Evaluation of Consumer Loans Using Support Vector Machines. Expert Systems with Applications, 30(4), 772-782.
Müller, K.-R., Smola, A., Rätsch, G., Schölkopf, B., Kohlmorgen, J., ＆ Vapnik, V. (1997). Predicting time series with support vector machines. Paper presented at the International Conference on Artificial Neural Networks

Mariéthoz, J., ＆ Bengio, S. (2007). A Kernel Trick for Sequences Applied to Text-Independent Speaker Verification Systems. Pattern Recognition, 40(8), 2315-2324.
Mercer, J. (1909). Functions of Positive and Negative Type, and their Connection with the Theory of Integral Equations. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, 209(ArticleType: research-article / Full publication date: 1909 / Copyright © 1909 The Royal Society), 415-446.
Michael Doumpos, ＆ Zopounidis, C. (2009). Monotonic Support Vector Machines for Credit Risk Rating. New Mathematics and Natural Computation, 5(3), 557-570. doi: 10.1142/S1793005709001520
Mukherjee, S., Osuna, E., ＆ Girosi, F. (1997). Nonlinear prediction of chaotic time series using a support vector machine. Paper presented at the IEEE Workshop on Neural networks for Signal Processing 7, Amelia Island, FL.
Na, M. G., Park, W. S., ＆ Lim, D. H. (2008). Detection and Diagnostics of Loss of Coolant Accidents Using Support Vector Machines. IEEE Transactions Nuclear Science, 55(1), 628-636.
Pazzani, M. J., Mani, S., ＆ Shankle, W. R. (2001). Acceptance of rules generated by machine learning among medical experts. Methods of Information in Medicine, 40, 380-385.
Pelckmans, K., Espinoza, M., Brabanter, J., Suykens, J. A. K., ＆ Moor, B. (2005). Primal-Dual Monotone Kernel Regression. Neural Processing Letters, 22(2), 171-182. doi: 10.1007/s11063-005-5264-1
Pelckmans, K., Espinoza, M., De Brabanter, J., Suykens, J. A. K., ＆ De Moor, B. (2005). Prime-Dual Monotone Kernel Regression. Neural Processing Letters, 22(2), 171-182.
Pendharkar, P. C. (2005). A data envelopment analysis-based approach for data preprocessing. IEEE Transactions on Knowledge and Data Engineering, 17(10), 1379-1388.
Pendharkar, P. C., ＆ Rodger, J. A. (2003). Technical efficiency-based selection of learning cases to improve forecasting accuracy of neural networks under monotonicity assumption. Decision Support Systems, 36(1), 117-136. doi: Doi 10.1016/S0167-9236(02)00138-0
Potharst, R., ＆ Feelders, A. J. (2002). Classification trees for problems with monotonicity constraints. SIGKDD Explor. Newsl., 4(1), 1-10. doi: 10.1145/568574.568577
Ravikumar, B., Thukaram, D., ＆ Khincha, H. P. (2009). An Approach Using Support Vector Machines for Distance Relay Coordination in Transmission System. IEEE Transactions on Power Delivery, 24(1), 79-88.
Ryu, Y. U., Chandrasekaran, R., ＆ Jacob, V. (2007). Data classification using the isotonic separation technique: Application to breast cancer prediction. European Journal of Operational Research, 181, 842–854.
Schölkopf, B., ＆ Smola, A. J. (2002). Learning with kernels : support vector machines, regularization, optimization, and beyond. Cambridge, Mass.: MIT Press.
Shilton, A., Palaniswami, M., Ralph, D., ＆ Tsoi, A. C. (2005). Incremental training of support vector machines. Ieee Transactions on Neural Networks, 16(1), 114-131. doi: Doi 10.1109/Tnn.2004.836201
Shin, K. S., Lee, T. S., ＆ Kim, H. J. (2005). An application of support vector machines in bankruptcy prediction model. Expert Systems with Applications, 28(1), 127-135. doi: DOI 10.1016/j.eswa.2004.08.009
Smola, A. J., Schölkopf, B., ＆ Muller, K. R. (1998). The connection between regularization operators and support vector kernels. Neural Networks, 11(4), 637-649.
Trafalis, T. B. (1999). Primal-dual Optimization methods in Neural Networks and Support Vector Machines training. University of Oklahoma.
Van Gestel, T., Baesens, B., Suykens, J. A. K., Van den Poel, D., Baestaens, D. E., ＆ Willekens, M. (2006). Bayesian Kernel Based Classification for Financial Distress Detection. European Journal of Operational Research,, 172(3), 979-1003.
Vapnik, V. N. (1995). The nature of statistical learning theory. New York: Springer.
Vapnik, V. N. (1998). Statistical learning theory. New York: Wiley.
Vazirigiannis, M., Halkidi, M., ＆ Gunopulos, D. (2003). Uncertainty handling and quality assessment in data mining. London ; New York: Springer.
Wang, S. H. (1995). The Unpredictability of Standard Back-Propagation Neural Networks in Classification Applications. Management Science, 41(3), 555-559.
Wang, S. H. (2003). Adaptive non-parametric efficiency frontier analysis: a neural-network-based model. Computers ＆ Operations Research, 30(2), 279-295.
Wouter Duivesteijn, ＆ Feelders, A. (2008). Nearest Neighbour Classification with Monotonicity Constraints. Lecture Notes in Computer Science, 5211/2008, 301-306. doi: 10.1007/978-3-540-87479-9_38
Xu, Y., Wang, X.-B., Ding, J., Wu, L.-Y., ＆ Deng, N.-Y. (2010). Lysine Acetylation Sites Prediction Using an Ensemble of Support Vector Machine Classifiers Journal of Theoretical Biology, 264(1), 130-135.
Yu, H.-F., Hsieh, C.-J., Chang, K.-W., ＆ Lin, C.-J. (2010). Large linear classification when data cannot fit in memory. Paper presented at the Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, Washington, DC, USA.