臺灣博碩士論文加值系統

Multi-class classification is one of the major challenges in machine learning and an on-going research issue. Classification algorithms are generally binary but they must be extended to multi-class problems for real-world application. Multi-class classification is more complex than binary classification. In binary classification, only the decision boundaries of one class are to be known whereas in multiclass classification, several boundaries are involved. The objective of this investigation is to propose a metaheuristic optimized multi-level classification model for forecasting in engineering problems. The proposed model integrates the firefly algorithm (FA), metaheuristic intelligence, decomposition approaches, the one-against-one (OAO), and the least squares support vector machine (LSSVM). The enhanced FA automatically fine-tunes the hyperparameters of the LSSVM to construct an optimized LSSVM classification model. The developed model is called the Optimized-OAO-LSSVM. Ten benchmark functions are used to evaluate the performance of the enhanced optimization algorithm. Two binary-class datasets related to geotechnical engineering, concerning seismic bumps and soil liquefaction, are then used to clarify the application of the proposed model to binary problems. Further, this investigation uses multi-class cases in civil engineering and construction management to verify the effectiveness of the model in the diagnosis of faults in steel plates, quality of water in a reservoir and determining urban land cover. The results revealed that the Optimized-OAO-LSSVM model predicted faults in steel plates with an accuracy of 91.085%, the quality of water in a reservoir with an accuracy of 93.650% and urban land cover with an accuracy of 87.274%. To demonstrate the effectiveness of the proposed model, its predictive accuracy was compared with that of a non-optimized baseline model (OAO-LSSVM), single multi-class classification algorithms (Sequential Minimal Optimization (SMO), the Multiclass Classifier, the Naïve Bayes, the Library Support vector machine (LibSVM) and Logistic). The analytical results showed that the proposed model is a promising tool to help decision-makers solve classification problems in civil engineering and construction management.

Multi-class classification is one of the major challenges in machine learning and an on-going research issue. Classification algorithms are generally binary but they must be extended to multi-class problems for real-world application. Multi-class classification is more complex than binary classification. In binary classification, only the decision boundaries of one class are to be known whereas in multiclass classification, several boundaries are involved. The objective of this investigation is to propose a metaheuristic optimized multi-level classification model for forecasting in engineering problems. The proposed model integrates the firefly algorithm (FA), metaheuristic intelligence, decomposition approaches, the one-against-one (OAO), and the least squares support vector machine (LSSVM). The enhanced FA automatically fine-tunes the hyperparameters of the LSSVM to construct an optimized LSSVM classification model. The developed model is called the Optimized-OAO-LSSVM. Ten benchmark functions are used to evaluate the performance of the enhanced optimization algorithm. Two binary-class datasets related to geotechnical engineering, concerning seismic bumps and soil liquefaction, are then used to clarify the application of the proposed model to binary problems. Further, this investigation uses multi-class cases in civil engineering and construction management to verify the effectiveness of the model in the diagnosis of faults in steel plates, quality of water in a reservoir and determining urban land cover. The results revealed that the Optimized-OAO-LSSVM model predicted faults in steel plates with an accuracy of 91.085%, the quality of water in a reservoir with an accuracy of 93.650% and urban land cover with an accuracy of 87.274%. To demonstrate the effectiveness of the proposed model, its predictive accuracy was compared with that of a non-optimized baseline model (OAO-LSSVM), single multi-class classification algorithms (Sequential Minimal Optimization (SMO), the Multiclass Classifier, the Naïve Bayes, the Library Support vector machine (LibSVM) and Logistic). The analytical results showed that the proposed model is a promising tool to help decision-makers solve classification problems in civil engineering and construction management.

ABSTRACT
TABLE OF CONTENT
LIST OF FIGURES
LIST OF TABLE
ABBREVIATIONS AND SYMBOLS
Chapter 1 : Introduction
1.1. Research background and motivation
1.2. Research objectives
1.3. Research outline
Chapter 2 : Literature review
2.1. Artificial intelligence for solving multi-class problems in civil engineering and construction management
2.2. Hybrid artificial intelligence-based multi-class classification model
2.3. Development of expert system for solving multi-classification problems
Chapter 3 : Methodology
3.1. Decomposing approaches
3.2. Metaheuristic optimization in machine learning
3.2.1. Least squares support vector machine for classification
3.2.2. Metaheuristic artificial firefly algorithm
3.2.3. Optimized LSSVM model with decomposition
3.3. Performance measures
3.3.1. Model evaluation
3.3.2. Hypothesis testing for model comparison
3.4. Programming techniques
Chapter 4 : Metaheuristic-optimized multi-level classification system
4.1. Swarm and metaheuristic optimization algorithm benchmarking
4.2. Systematization development
4.2.1. System framework
4.2.2. Systematization
Chapter 5 : Engineering applications
5.1. Numerical experiments
5.1.1. Data preprocessing
5.1.2. Benchmarking results
5.2. Solving engineering classification problems
5.2.1. Binary-class problems
5.2.2. Multi-class problems
5.3. Analytical results and discussion
Chapter 6 : Conclusion and recommendations
References
APPENDICES
APPENDIX A. User Interface Snapshot
A.1 Main user interface
A.2 Optimized-OAO-LSSVM interface (evaluation)
A.3 Optimized-OAO-LSSVM interface (prediction)
A.4 OAO-LSSVM interface (evaluation)
A.5 OAO-LSSVM interface (prediction)
APPENDIX B. Analysis Report of Optimized-OAO-LSSVM model
B.1 Analysis Report for Diagnosis of faults in steel plates Case (data set 1), Without Feature Scaling
B.2 Analysis Report for Diagnosis of faults in steel plates Case (data set 1), With Feature Scaling
B.3 Analysis Report for Quality of water in reservoir Case (data set 2), Without Feature Scaling
B.4 Analysis Report for Quality of water in reservoir Case (data set 2), With Feature Scaling
B.5 Analysis Report for Urban land cover Case (dataset 3), Without Feature Scaling
B.6 Analysis Report for Urban land cover Case (dataset 3), With Feature Scaling
APPENDIX C. Performance of the ROC
C.1 Diagnosis of faults in steel plates Case (data set 1), Without Feature Scaling
C.2 Diagnosis of faults in steel plates Case (data set 1), With Feature Scaling
C.3 Quality of water in reservoir Case (data set 2), Without Feature Scaling
C.4 Quality of water in reservoir Case (data set 2), With Feature Scaling
C.5 Urban land cover Case (data set 3), Without Feature Scaling
C.6 Urban land cover Case (data set 3), With Feature Scaling

[1] M.P. Sesmero, J.M. Alonso-Weber, G. Gutierrez, A. Ledezma, A. Sanchis, An ensemble approach of dual base learners for multi-class classification problems, Information Fusion, 24 (2015) 122-136.
[2] A. Bhardwaj, A. Tiwari, Breast cancer diagnosis using Genetically Optimized Neural Network model, Expert Systems with Applications, 42 (2015) 4611-4620.
[3] J. Iounousse, S. Er-Raki, A. El Motassadeq, H. Chehouani, Using an unsupervised approach of Probabilistic Neural Network (PNN) for land use classification from multitemporal satellite images, Applied Soft Computing, 30 (2015) 1-13.
[4] N. Huang, D. Xu, X. Liu, L. Lin, Power quality disturbances classification based on S-transform and probabilistic neural network, Neurocomputing, 98 (2012) 12-23.
[5] F. Kaefer, C.M. Heilman, S.D. Ramenofsky, A neural network application to consumer classification to improve the timing of direct marketing activities, Computers & Operations Research, 32 (2005) 2595-2615.
[6] M. Galar, A. Fernández, E. Barrenechea, H. Bustince, F. Herrera, An overview of ensemble methods for binary classifiers in multi-class problems: Experimental study on one-vs-one and one-vs-all schemes, Pattern Recognition, 44 (2011) 1761-1776.
[7] S. Karakatič, V. Podgorelec, Improved classification with allocation method and multiple classifiers, Information Fusion, 31 (2016) 26-42.
[8] Z.Z. Wang, Z. Zhang, Seismic damage classification and risk assessment of mountain tunnels with a validation for the 2008 Wenchuan earthquake, Soil Dynamics and Earthquake Engineering, 45 (2013) 45-55.
[9] J.-S. Chou, S.-C. Hsu, C.-W. Lin, Y.-C. Chang, Classifying Influential Information to Discover Rule Sets for Project Disputes and Possible Resolutions, International Journal of Project Management, 34 (2016) 1706-1716.
[10] H. Chih-Wei, L. Chih-Jen, A comparison of methods for multiclass support vector machines, IEEE Transactions on Neural Networks, 13 (2002) 415-425.
[11] R. Rifkin, A. Klautau, In Defense of One-Vs-All Classification, J. Mach. Learn. Res., 5 (2004) 101-141.
[12] A.C. Lorena, A.C.P.L.F. de Carvalho, J.M.P. Gama, A review on the combination of binary classifiers in multiclass problems, Artificial Intelligence Review, 30 (2009) 19.
[13] M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, I.H. Witten, The WEKA data mining software: an update, SIGKDD Explor. Newsl., 11 (2009) 10-18.
[14] M. Galar, A. Fernández, E. Barrenechea, F. Herrera, DRCW-OVO: Distance-based relative competence weighting combination for One-vs-One strategy in multi-class problems, Pattern Recognition, 48 (2015) 28-42.
[15] S. Kang, S. Cho, P. Kang, Constructing a multi-class classifier using one-against-one approach with different binary classifiers, Neurocomputing, 149, Part B (2015) 677-682.
[16] J.A. Balazs, J.D. Velásquez, Opinion Mining and Information Fusion: A survey, Information Fusion, 27 (2016) 95-110.
[17] W. Medhat, A. Hassan, H. Korashy, Sentiment analysis algorithms and applications: A survey, Ain Shams Engineering Journal, 5 (2014) 1093-1113.
[18] X. Yang, Q. Yu, L. He, T. Guo, The one-against-all partition based binary tree support vector machine algorithms for multi-class classification, Neurocomputing, 113 (2013) 1-7.
[19] K.-j. Kim, H. Ahn, A corporate credit rating model using multi-class support vector machines with an ordinal pairwise partitioning approach, Computers & Operations Research, 39 (2012) 1800-1811.
[20] J.-S. Chou, A.-D. Pham, Nature-inspired metaheuristic optimization in least squares support vector regression for obtaining bridge scour information, Information Sciences, 399 (2017) 64-80.
[21] T. van Gestel, J.A.K. Suykens, B. Baesens, S. Viaene, J. Vanthienen, G. Dedene, B. de Moor, J. Vandewalle, Benchmarking Least Squares Support Vector Machine Classifiers, Machine Learning, 54 (2004) 5-32.
[22] C.G. L. Qingjie, L. Xiaofang, Y. Qing, Genetic algorithm based SVM parameter composition optimization, Comput. Appl. Softw., 29 (04) (2012).
[23] H.C. C. Hang, Z. Yonggang, Constructed of SVM decision tree based on particle swarm optimization algorithm for gear box fault diagnosis, 33 (01) (2013) 2153-2156.
[24] S.K. Pal, Rai, C. S., and Singh, A. P. (2012). , Comparative study of firefly algorithm and particle swarm optimization for noisy non-linear optimization problems, Int. J. Intell. Syst. Appl., 4 (10) (2012) 50–57.
[25] J. Olamaei, M. Moradi, T. Kaboodi, A new adaptive modified Firefly Algorithm to solve optimal capacitor placement problem, in: 18th Electric Power Distribution Conference, 2013, pp. 1-6.
[26] J.-S. Chou, A.-D. Pham, Enhanced artificial intelligence for ensemble approach to predicting high performance concrete compressive strength, Construction and Building Materials, 49 (2013) 554-563.
[27] L.d.S. Coelho, V.C. Mariani, Improved firefly algorithm approach applied to chiller loading for energy conservation, Energy and Buildings, 59 (2013) 273-278.
[28] A. Verma, X. Wei, A. Kusiak, Predicting the total suspended solids in wastewater: A data-mining approach, Engineering Applications of Artificial Intelligence, 26 (2013) 1366-1372.
[29] J.-S. Chou, C. Lin, Predicting Disputes in Public-Private Partnership Projects: Classification and Ensemble Models, Journal of Computing in Civil Engineering, 27 (2013) 51-60.
[30] M.-Y. Cheng, N.-D. Hoang, Risk Score Inference for Bridge Maintenance Project Using Evolutionary Fuzzy Least Squares Support Vector Machine, Journal of Computing in Civil Engineering, 28 (2014) 04014003.
[31] Y. Liu, H. Wang, H. Zhang, K. Liber, A comprehensive support vector machine-based classification model for soil quality assessment, Soil and Tillage Research, 155 (2016) 19-26.
[32] R. Gholami, V. Rasouli, A. Alimoradi, Improved RMR Rock Mass Classification Using Artificial Intelligence Algorithms, Rock Mechanics and Rock Engineering, 46 (2013) 1199-1209.
[33] M. Wauters, M. Vanhoucke, Support Vector Machine Regression for project control forecasting, Automation in Construction, 47 (2014) 92-106.
[34] Y. Liao, J. Xu, W. Wang, A Method of Water Quality Assessment Based on Biomonitoring and Multiclass Support Vector Machine, Procedia Environmental Sciences, 10 (2011) 451-457.
[35] S. Du, C. Liu, L. Xi, A Selective Multiclass Support Vector Machine Ensemble Classifier for Engineering Surface Classification Using High Definition Metrology, Journal of Manufacturing Science and Engineering, 137 (2015) 011003-011003-011015.
[36] K. Polat, S. Güneş, A novel hybrid intelligent method based on C4.5 decision tree classifier and one-against-all approach for multi-class classification problems, Expert Systems with Applications, 36 (2009) 1587-1592.
[37] L.P.F. Garcia, J.A. Sáez, J. Luengo, A.C. Lorena, A.C.P.L.F. de Carvalho, F. Herrera, Using the One-vs-One decomposition to improve the performance of class noise filters via an aggregation strategy in multi-class classification problems, Knowledge-Based Systems, 90 (2015) 153-164.
[38] L. Zhou, K.P. Tam, H. Fujita, Predicting the listing status of Chinese listed companies with multi-class classification models, Information Sciences, 328 (2016) 222-236.
[39] W. Huang, Y. Nakamori, S.-Y. Wang, Forecasting stock market movement direction with support vector machine, Computers & Operations Research, 32 (2005) 2513-2522.
[40] T.V.G. J. A. K. Suykens, J. D. Brabanter, B. D. Moor, and J. Vandewalle, Least squares support vector machines, Singapore: World Scientific, (2002).
[41] V.N. Vapnik, The Nature of Statistical Learning Theory, New York: Springer, (2000).
[42] R. Khemchandani, Jayadeva, S. Chandra, Regularized least squares fuzzy support vector regression for financial time series forecasting, Expert Systems with Applications, 36 (2009) 132-138.
[43] W. Haifeng, H. Dejin, Comparison of SVM and LS-SVM for Regression, in: 2005 International Conference on Neural Networks and Brain, 2005, pp. 279-283.
[44] S.-H. Min, J. Lee, I. Han, Hybrid genetic algorithms and support vector machines for bankruptcy prediction, Expert Systems with Applications, 31 (2006) 652-660.
[45] P.-F. Pai, W.-C. Hong, Support vector machines with simulated annealing algorithms in electricity load forecasting, Energy Conversion and Management, 46 (2005) 2669-2688.
[46] W.-C. Hong, Electric load forecasting by seasonal recurrent SVR (support vector regression) with chaotic artificial bee colony algorithm, Energy, 36 (2011) 5568-5578.
[47] J. Wang, S. Jin, S. Qin, H. Jiang, Swarm Intelligence-Based Hybrid Models for Short-Term Power Load Prediction, Mathematical Problems in Engineering, (2014).
[48] X.-S. Yang, Firefly algorithm, Bristol, U.K: Luniver Press, (2008).
[49] H. Banati, M. Bajaj, Fire Fly Based Feature Selection Approach, International Journal of Computer Science Issues, 8 (2011) 473-479.
[50] I. Fister, I. Fister Jr, X.-S. Yang, J. Brest, A comprehensive review of firefly algorithms, Swarm and Evolutionary Computation, 13 (2013) 34-46.
[51] S.C. Aphirak Khadwilard, Thatchai Thepphakorn, Peeraya Thapatsuwan, Warattapop Chainate and Pupong Pongcharoen, Application of Firefly Algorithm and Its Parameter Setting for Job Shop Scheduling, Journal of Industrial Technology 8 (2012).
[52] P. Aungkulanon, N.Chai-ead, P. Luangpaiboon, Simulated Manufacturing Process Improvement via Particle Swarm Optimisation and Firefly Algorithms, Lecture Notes in Engineering and Computer Science, (2011).
[53] M. Aci, C. İnan, M. Avci, A hybrid classification method of k nearest neighbor, Bayesian methods and genetic algorithm, Expert Systems with Applications, 37 (2010) 5061-5067.
[54] J.-S. Chou, M.-Y. Cheng, Y.-W. Wu, Improving classification accuracy of project dispute resolution using hybrid artificial intelligence and support vector machine models, Expert Systems with Applications, 40 (2013) 2263-2274.
[55] Y. Lee, J. Lee, Binary tree optimization using genetic algorithm for multiclass support vector machine, Expert Systems with Applications, 42 (2015) 3843-3851.
[56] M. Seera, C.P. Lim, A hybrid intelligent system for medical data classification, Expert Systems with Applications, 41 (2014) 2239-2249.
[57] Y. Tian, M. Fu, F. Wu, Steel plates fault diagnosis on the basis of support vector machines, Neurocomputing, 151, Part 1 (2015) 296-303.
[58] N. García-Pedrajas, D. Ortiz-Boyer, An empirical study of binary classifier fusion methods for multiclass classification, Information Fusion, 12 (2011) 111-130.
[59] C. Cortes, V. Vapnik, Support-Vector Networks, Machine Learning, 20 (1995) 273-297.
[60] H. Zhang, F. Yang, Y. Li, H. Li, Predicting profitability of listed construction companies based on principal component analysis and support vector machine—Evidence from China, Automation in Construction, 53 (2015) 22-28.
[61] M.-Y. Cheng, N.-D. Hoang, Y.-W. Wu, Hybrid intelligence approach based on LS-SVM and Differential Evolution for construction cost index estimation: A Taiwan case study, Automation in Construction, 35 (2013) 306-313.
[62] H. Son, C. Kim, C. Kim, Hybrid principal component analysis and support vector machine model for predicting the cost performance of commercial building projects using pre-project planning variables, Automation in Construction, 27 (2012) 60-66.
[63] X.-S. Yang, Nature-Inspired Metaheuristic Algorithms, Luniver Press, (2008).
[64] A.H. Gandomi, X.S. Yang, S. Talatahari, A.H. Alavi, Firefly algorithm with chaos, Communications in Nonlinear Science and Numerical Simulation, 18 (2013) 89-98.
[65] W.-C. Hong, Y. Dong, L.-Y. Chen, S.-Y. Wei, SVR with hybrid chaotic genetic algorithms for tourism demand forecasting, Applied Soft Computing, 11 (2011) 1881-1890.
[66] R. Kohavi, A study of cross-validation and bootstrap for accuracy estimation and model selection, in: Proceedings of the 14th international joint conference on Artificial intelligence - Volume 2, Morgan Kaufmann Publishers Inc., Montreal, Quebec, Canada, 1995, pp. 1137-1143.
[67] J.-S. Chou, C.-F. Tsai, Y.-H. Lu, Project dispute prediction by hybrid machine learning techniques, Journal of Civil Engineering and Management, 19 (2013) 505-517.
[68] M. Sokolova, G. Lapalme, A systematic analysis of performance measures for classification tasks, Information Processing & Management, 45 (2009) 427-437.
[69] M. Jamil, X.-S. Yang, A Literature Survey of Benchmark Functions For Global Optimization Problems, Int. Journal of Mathematical Modelling and Numerical Optimisation, 4 (2013) 150-194.
[70] S.S.a.D. Bingham., Virtual library of simulation experiments: test functions and datasets, (2013).
[71] M. Lichman, UCI Machine Learning Repository,, University of California, School of Information and Computer Science, Irvine, CA, http://archive.ics.uci.edu/ml, (2013).
[72] M. Sikora, Ł. Wróbel, Application of rule induction algorithms for analysis of data collected by seismic hazard monitoring systems in coal mines, Archives of Mining Sciences, 55 (2010) 91-114.
[73] A.T.C. Goh, S.H. Goh, Support vector machines: Their use in geotechnical engineering as illustrated using seismic liquefaction data, Computers and Geotechnics, 34 (2007) 410-421.
[74] J.-S. Chou, J.P.P. Thedja, Metaheuristic optimization within machine learning-based classification system for early warnings related to geotechnical problems, Automation in Construction, 68 (2016) 65-80.
[75] S.S. Durduran, Automatic classification of high resolution land cover using a new data weighting procedure: The combination of k-means clustering algorithm and central tendency measures (KMC–CTM), Applied Soft Computing, 35 (2015) 136-150.
[76] N.T.U. Hydrotech Research Institute, Reservoir eutrophiction prediction and prevention by using remote sensing technique, Water Resources Agency (in Chinese) (2005).
[77] E.A. Wentz, W.L. Stefanov, C. Gries, D. Hope, Land use and land cover mapping from diverse data sources for an arid urban environments, Computers, Environment and Urban Systems, 30 (2006) 320-346.
[78] S. Crone, J. Guajardo, R. Weber, The impact of preprocessing on support vector regression and neural networks in time series prediction, in: Proceedings of the International Conference on Data Mining DMIN'06, Las Vegas, USA, 2006, pp. 37-42.