臺灣博碩士論文加值系統

本文將幾種 Boosting 類機器學習方法應用於信用違約預測。信用卡違約預測是信用風險管理中十分重要的一環，也是信用經濟發展過程中的必經之路。本文以邏輯回歸作為基準，主要觀察 Boosting 類方法在信用違約預測中的作用。實驗結果表明，Booting 類方法訓練後的增強模型比起邏輯回歸有更好的預測表現。相信 Booting 類方法可以幫助金融機構加快信貸模式的發展，從而指導他們的決策並提高盈利能力。

We use several machine learning techniques applied in credit default prediction project. Forecasting of credit card default is one of the necessary elements in developing of the credit economy. In my thesis, I focus on the boosting methods to identify the credit card default because the individual classifiers after boosting can predict better results in many data com- petitions. The results of experiment demonstrate that trained boosting models achieve better prediction than logistic regression. We hope boosting models may help financial institutions to speed up their credit model development thus guide their decision-making and improve prof- itability.

摘要.................................i
Abstract...............................ii
Acknowledgement .........................iii
Table of Contents ..................................... iv
List of Figures ....................................... vi
List of Tables........................................ vii
1 Introduction....................................... 1
1.1 Literature Review................................. 1
1.2 Problem Statement ................................ 5
1.3 Organization of Thesis .............................. 6
2 Methodology ...................................... 7
2.1 Logistic Regression................................ 7
2.2 Decision Tree................................... 9
2.3 Boosting...................................... 11
2.3.1 AdaBoost................................. 11
2.3.2 GBDT................................... 13
2.3.3 XGBoost ................................. 15
2.3.4 LightGBM ................................ 16
2.4 Experimental procedure.............................. 17
3 Performance Measures................................. 18
3.1 Confusion Matrix................................. 18
3.2 ROCCurve.................................... 19
3.3 Value Added Measure............................... 21
4 Illustration1 ...................................... 23
4.1 DataDescription ................................. 23
4.2 Exploratory Data Analysis ............................ 23
4.3 Data Preprocessing ................................ 30
4.4 Results....................................... 33
5 Illustration2 ...................................... 35
5.1 Data Description ................................. 35
5.2 Exploratory Data Analysis ............................ 35
5.3 Data Preprocessing ................................ 38
5.4 Results....................................... 41
6 Conclusion ....................................... 43
References ......................................... 46

[1] T. Fawcett and F. Provost, “Adaptive fraud detection,” Data Mining and Knowledge Discovery, vol. 1, no. 3, pp. 291–316, Sep 1997. [Online]. Available: https: //doi.org/10.1023/A:1009700419189
[2] D. Lo, H. Cheng, J. Han, S.-C. Khoo, and C. Sun, “Classification of software behaviors for failure detection: A discriminative pattern mining approach,” in Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ser. KDD ’09. New York, NY, USA: ACM, 2009, pp. 557–566. [Online]. Available: http://doi.acm.org/10.1145/1557019.1557083
[3] M. Roughan, S. Sen, O. Spatscheck, and N. Duffield, “Class-of-service mapping for qos: A statistical signature-based approach to ip traffic classification,” in Proceedings of the 4th ACM SIGCOMM Conference on Internet Measurement, ser. IMC ’04. New York, NY, USA: ACM, 2004, pp. 135–148. [Online]. Available: http://doi.acm.org/10.1145/1028788.1028805
[4] M. P. S. Brown, W. N. Grundy, D. Lin, N. Cristianini, C. W. Sugnet, T. S. Furey, M. Ares, and D. Haussler, “Knowledge-based analysis of microarray gene expression data by using support vector machines,” Proceedings of the National Academy of Sciences, vol. 97, no. 1, pp. 262–267, 2000. [Online]. Available: https://www.pnas.org/content/97/1/262
[5] H. Nielsen, S. Brunak, and G. von Heijne, “Machine learning approaches for the prediction of signal peptides and other protein sorting signals,” Protein Engineering, Design and Selection, vol. 12, no. 1, pp. 3–9, 01 1999. [Online]. Available: https://doi.org/10.1093/protein/12.1.3
[6] M. Kubat, R. C. Holte, and S. Matwin, “Machine learning for the detection of oil spills in satellite radar images,” Machine Learning, vol. 30, no. 2, pp. 195–215, Feb 1998. [Online]. Available: https://doi.org/10.1023/A:1007452223027
[7] M. S. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I. Fasel, and J. Movellan, “Recog- nizing facial expression: machine learning and application to spontaneous behavior,” in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 2, June 2005, pp. 568–573 vol. 2.
[8] J. Perols, “Financial statement fraud detection: An analysis of statistical and machine learning algorithms,” AUDITING: A Journal of Practice & Theory, vol. 30, no. 2, pp. 19–50, 2011. [Online]. Available: https://doi.org/10.2308/ajpt-50009
[9] M.D.OdomandR.Sharda,“Aneuralnetworkmodelforbankruptcyprediction,”in1990 IJCNN International Joint Conference on Neural Networks, June 1990, pp. 163–168 vol. 2.
[10] V. S. Desai, J. N. Crook, and G. A. Overstreet, “A comparison of neural networks and linear scoring models in the credit union environment,” European Journal of Operational Research, vol. 95, no. 1, pp. 24 – 37, 1996. [Online]. Available: http://www.sciencedirect.com/science/article/pii/0377221795002464
[11] T.-S. Lee, C.-C. Chiu, C.-J. Lu, and I.-F. Chen, “Credit scoring using the hybrid neural discriminant technique,” Expert Systems with Applications, vol. 23, no. 3, pp. 245 – 254, 2002. [Online]. Available: http://www.sciencedirect.com/science/article/pii/ S0957417402000441
[12] E. Angelini, G. di Tollo, and A. Roli, “A neural network approach for credit risk evaluation,” The Quarterly Review of Economics and Finance, vol. 48, no. 4, pp. 733 – 755, 2008. [Online]. Available: http://www.sciencedirect.com/science/article/pii/ S1062976907000762
[13] V. Kozeny, “Genetic algorithms for credit scoring: Alternative fitness function performance comparison,” Expert Systems with Applications, vol. 42, no. 6, pp. 2998 – 3004, 2015. [Online]. Available: http://www.sciencedirect.com/science/article/pii/ S0957417414007143
[14] A. Blanco, R. Pino-MejÃas, J. Lara, and S. Rayo, “Credit scoring models for the microfinance industry using neural networks: Evidence from peru,” Expert Systems with Applications, vol. 40, no. 1, pp. 356 – 364, 2013. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0957417412009062
[15] R. Florez-Lopez and J. M. Ramon-Jeronimo, “Enhancing accuracy and interpretability of ensemble strategies in credit risk assessment. a correlated-adjusted decision forest proposal,” Expert Systems with Applications, vol. 42, no. 13, pp. 5737 – 5753, 2015. [Online]. Available: http://www.sciencedirect.com/science/article/pii/ S0957417415001499
[16] V. Srinivasan and Y. H. Kim, “Credit granting: A comparative analysis of classification procedures,” The Journal of Finance, vol. 42, no. 3, pp. 665–681, 1987. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1540-6261.1987.tb04576.x
[17] C. Cortes and V. Vapnik, “Support-vector networks,” Machine Learning, vol. 20, no. 3, pp. 273–297, Sep 1995. [Online]. Available: https://doi.org/10.1007/BF00994018
[18] C.-L. Huang, M.-C. Chen, and C.-J. Wang, “Credit scoring with a data mining approach based on support vector machines,” Expert Systems with Applications, vol. 33, no. 4, pp. 847 – 856, 2007. [Online]. Available: http://www.sciencedirect.com/science/article/pii/ S095741740600217X
[19] S. P. Dina, A. M. Nur, and S. Bagus, “Credit scoring analysis using lasso logistic re- gression and support vector machine (svm),” International Journal of Engineering and Management Research, 2017.
[20] T. Harris, “Credit scoring using the clustered support vector machine,” Expert Systems with Applications, vol. 42, no. 2, pp. 741 – 750, 2015. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0957417414005119
[21] L. G. Valiant, “A theory of the learnable,” in Proceedings of the Sixteenth Annual ACM Symposium on Theory of Computing, ser. STOC ’84. New York, NY, USA: ACM, 1984, pp. 436–445. [Online]. Available: http://doi.acm.org/10.1145/800057.808710
[22] H. Drucker, R. Schapire, and P. Simard, “Boosting performance in neural networks,” International Journal of Pattern Recognition and Artificial Intelligence, vol. 07, no. 04, pp. 705–719, 1993. [Online]. Available: https://doi.org/10.1142/S0218001493000352
[23] Y. Freund and R. E. Schapire, “A decision-theoretic generalization of on-line learning and an application to boosting,” Journal of Computer and System Sciences, vol. 55, no. 1, pp. 119 – 139, 1997. [Online]. Available: http://www.sciencedirect.com/science/article/pii/ S002200009791504X
[24] J. H. Friedman, “Greedy function approximation: A gradient boosting machine,” The Annals of Statistics, vol. 29, no. 5, pp. 1189–1232, 2001. [Online]. Available: http://www.jstor.org/stable/2699986
[25] T. Chen and C. Guestrin, “Xgboost: A scalable tree boosting system,” in Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ser. KDD ’16. New York, NY, USA: ACM, 2016, pp. 785–794. [Online]. Available: http://doi.acm.org/10.1145/2939672.2939785
[26] A. E. Khandani, A. J. Kim, and A. W. Lo, “Consumer credit-risk models via machine-learning algorithms,” Journal of Banking and Finance, vol. 34, no. 11, pp. 2767 – 2787, 2010. [Online]. Available: http://www.sciencedirect.com/science/article/pii/ S0378426610002372
[27] L. Breiman, J. Friedman, R. Olshen, and C. Stone, “Classification and regression trees,” 1984.
[28] G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y. Liu, “Lightgbm: A highly efficient gradient boosting decision tree,” in Advances in Neural Information Processing Systems 30, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds. Curran Asso- ciates, Inc., 2017, pp. 3146–3154. [Online]. Available: http://papers.nips.cc/paper/ 6907-lightgbm-a-highly-efficient-gradient-boosting-decision-tree.pdf
[29] I.-C. Yeh, “Default of credit card clients data set,” UC Irvine Machine Learning Repository, 2016. [Online]. Available: http://archive.ics.uci.edu/ml/datasets/default+of+ credit+card+clients