臺灣博碩士論文加值系統

Probabilistic Machine Learning Model: Extended Monte Carlo Simulation for Solving Geotechnical Engineering Problems

Thesis Advisor: Jui-Sheng Chou
Graduate Student: Lucia Dewi Santoso

ABSTRACT
Machine learning (ML) is a data mining technique that integrates the principles of statistics, pattern recognition in machine learning, and data base systems. However, prediction, a powerful function of ML, mostly uses deterministic inputs to develop a deterministic prediction model; this model definitely cannot deal with uncertainty of the deterministic output result. Input data commonly includes outliers that contaminates the predictors, raising a question: with what do the inputs support the probability or certainty of the predictions? Monte Carlo simulation (MCS) is a probability-informed approach that is used in analyzing system reliability and risk because most of expected assumptions are taken in account in simulating the risk of the system that reasonably to be believed. Owing to the great effectiveness of machine learning in forecasting, this study attempts to construct a method that is based on Monte Carlo simulation to demonstrate a probabilistic machine learning model. This novel approach integrates the pattern recognition aspect of machine learning, empirical statistical rules, curve fitting, and data base systems, which are constructed by random samplings in the MCS to predict the probabilities of the outputs. The results indicate that probabilistic machine learning not only can gives the output variable of interest, along with a probability distribution, but it also performs faultless classification prediction that is based on a resampled dataset. Three case studies in the field of geotechnical engineering, which involve the peak shear strength of reinforced soil, factor of safety of slope, and the stability of a slope are presented to establish the effectiveness of this simulation method. The results reveal that, in addition to linking probabilistic information with outputs, the probabilistic machine learning model can significantly improve all of the prediction models that are used in demonstrated geotechnical cases.

Keywords: probabilistic machine learning; Monte Carlo simulation; data mining; engineering prediction; regression; classification.

Probabilistic Machine Learning Model: Extended Monte Carlo Simulation for Solving Geotechnical Engineering Problems

Thesis Advisor: Jui-Sheng Chou
Graduate Student: Lucia Dewi Santoso

ABSTRACT
Machine learning (ML) is a data mining technique that integrates the principles of statistics, pattern recognition in machine learning, and data base systems. However, prediction, a powerful function of ML, mostly uses deterministic inputs to develop a deterministic prediction model; this model definitely cannot deal with uncertainty of the deterministic output result. Input data commonly includes outliers that contaminates the predictors, raising a question: with what do the inputs support the probability or certainty of the predictions? Monte Carlo simulation (MCS) is a probability-informed approach that is used in analyzing system reliability and risk because most of expected assumptions are taken in account in simulating the risk of the system that reasonably to be believed. Owing to the great effectiveness of machine learning in forecasting, this study attempts to construct a method that is based on Monte Carlo simulation to demonstrate a probabilistic machine learning model. This novel approach integrates the pattern recognition aspect of machine learning, empirical statistical rules, curve fitting, and data base systems, which are constructed by random samplings in the MCS to predict the probabilities of the outputs. The results indicate that probabilistic machine learning not only can gives the output variable of interest, along with a probability distribution, but it also performs faultless classification prediction that is based on a resampled dataset. Three case studies in the field of geotechnical engineering, which involve the peak shear strength of reinforced soil, factor of safety of slope, and the stability of a slope are presented to establish the effectiveness of this simulation method. The results reveal that, in addition to linking probabilistic information with outputs, the probabilistic machine learning model can significantly improve all of the prediction models that are used in demonstrated geotechnical cases.

Keywords: probabilistic machine learning; Monte Carlo simulation; data mining; engineering prediction; regression; classification.

TABLE OF CONTENT
ABSTRACT i
ACKNOWLEDGEMENT ii
TABLE OF CONTENT iv
LIST OF FIGURES vii
LIST OF TABLES viii
ABBREVIATIONS AND SYMBOLS x
Chapter 1 1
INTRODUCTION 1
1.1 Research Background 1
1.2 Research Objectives 3
1.3 Research Process 4
Chapter 2 5
LITERATURE REVIEW 5
2.1 Machine learning models in geotechnical engineering 5
2.1.1 Multiple linear regression 5
2.1.2 Artificial Neural Network (ANN) 6
2.1.3 Support Vector Machine (SVM) 6
2.1.4 Classification and Regression Tree (CART) 9
2.2 Geotechnical prediction model developed in WEKA 10
2.3 Implementation of Monte Carlo simulation in geotechnical engineering 11
2.4 Monte Carlo simulation in spreadsheet 13
Chapter 3 15
METHODOLOGY 15
3.1 Probabilistic machine learning framework 15
3.2 Using ML to develop deterministic model 15
3.3 Resampling insufficient data 17
3.4 Fit the distribution of the input variable with a PDF to generate realization dataset 18
3.5 Construct realization prediction model by using ML model 19
3.6 Develop probability mass function (PMF) and cumulative distribution function (CDF) using the best prediction model 20
3.7 Framework performance evaluation method 21
3.7.1 Accuracy on testing dataset 22
3.7.2 Confusion matrix for classification problem 25
Chapter 4 27
CASE STUDY AND PERFORMANCE EVALUATION 27
4.1 Prediction of peak shear strength of geosynthetically-reinforced soil 30
4.1.1 Database of peak shear strength of reinforced soil 31
4.1.2 Construction of model for predicting peak shear strength using probabilistic machine learning framework 31
4.1.3 Evaluation of performance of model for predicting peak shear strength 32
4.2 Prediction of factors of safety of slopes 40
4.2.1 Database of factor of safety of slopes 41
4.2.2 Construction of model for predicting factor of safety of slopes using probabilistic machine learning framework 42
4.2.3 Evaluation of performance of model for predicting factor of safety of slopes 42
4.3 Prediction of stability of slope based on soil properties 52
4.3.1 Database of stability of slopes 53
4.3.2 Construction of model for predicting stability of slopes prediction using probabilistic machine learning model 54
4.3.3 Evaluation of performance of model for classifying stability of slopes 57
Chapter 5 59
CONCLUSION 59
REFERENCES 61
APPENDIX A. ORIGINAL DATASET 68
APPENDIX B. DISTRIBUTION FITTING CURVE 78
APPENDIX C. PREDICTION RESULT ON TESTING DATASET 82
APPENDIX D. TUTORIAL FOR SYSTEM DEVELOPMENT 117

REFERENCES
Abu-Farsakh, M., Coronel, J., Tao, M., 2007. Effect of Soil Moisture Content and Dry Density on Cohesive Soil–Geosynthetic Interactions Using Large Direct Shear Tests. Journal of Materials in Civil Engineering, 19, 540-549.
Aiken, L.S., West, S.G., Pitts, S.C., 2003. Multiple Linear Regression, Handbook of Psychology, I. B. Weiner (Ed.). ed.
Amir, H.A., Amir, H.G., 2011. A robust data mining approach for formulation of geotechnical engineering systems. Engineering Computations, 28, 242-274.
Calamak, M., Yanmaz, A.M., Kentel, E., 2017. Probabilistic evaluation of the effects of uncertainty in transient seepage parameters. Journal of Geotechnical and Geoenvironmental Engineering, 143, 06017009.
Carlà, T., Intrieri, E., Farina, P., Casagli, N., 2017. A new method to identify impending failure in rock slopes. International Journal of Rock Mechanics and Mining Sciences, 93, 76-81.
Cheng, Y.M., Lau, C.K., 2014. Slope Stability Analysis and Stabilization: New Methods and Insight, Second Edition. CRC Press.
Choobbasti, A.J., Farrokhzad, F., Barari, A., 2009. Prediction of slope stability using artificial neural network (case study: Noabad, Mazandaran, Iran). Arabian Journal of Geosciences, 2, 311-319.
Chou, J.-S., Ngo, N.-T., 2016. Engineering strength of fiber-reinforced soil estimated by swarm intelligence optimized regression system. Neural Computing and Applications.
Chou, J.-S., Yang, K.-H., Lin, J.-Y., 2016. Peak Shear Strength of Discrete Fiber-Reinforced Soils Computed by Machine Learning and Metaensemble Methods. Journal of Computing in Civil Engineering, 30, 04016036.
Chou, J.-S., Yang, K.-H., Pampang, J.P., Pham, A.-D., 2015. Evolutionary metaheuristic intelligence to simulate tensile loads in reinforcement for geosynthetic-reinforced soil structures. Computers and Geotechnics, 66, 1-15.
Chou, J.S., Pham, A.D., 2015. Smart Artificial Firefly Colony Algorithm‐Based Support Vector Regression for enhanced forecasting in civil engineering. Computer-Aided Civil and Infrastructure Engineering, 30, 715-732.
Cox, D.R., 1958. The regression analysis of binary sequences. Journal of the Royal Statistical Society, 20, 215-242.
Das, S.K., Biswal, R.K., Sivakugan, N., Das, B., 2011. Classification of slopes and prediction of factor of safety using differential evolution neural networks. Environmental Earth Sciences, 64, 201-210.
Elía, R.d., Laprise, R., 2005. Diversity in Interpretations of Probability: Implications for Weather Forecasting. Monthly Weather Review, 133, 1129-1143.
Feng, X., 2000. Introduction of intelligent rock mechanics. Science Press, 239-241.
Feng, X., Li, S., Yuan, C., Zeng, P., Sun, Y., 2018. Prediction of slope stability using Naive Bayes classifier. KSCE Journal of Civil Engineering, 22, 941-950.
Frank, E., Lee, K.L., Mark, D.B., 1996. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequancy, and measuring and reducing errors. Statistics in Medicine, 15, 361-387.
Gadd, S.A., Leaming, D.G., Riley, T., 1998. Transport Risk at: the HSE quantified risk assessment tool for toxic and flammable dangerous goods transport by road and rail in Great Britain, 9Th International Symposium Loss Prevention and Safety Promotion in the Process Industries, Barcelona, Spain, pp. 308-317.
Gholamy, A., Kreinovich, V., Kosheleva, O., 2018. Why 70/30 or 80/20 relation between training and testing sets: a pedagogical explanation, Texas, United State.
Gordan, B., Jahed Armaghani, D., Hajihassani, M., Monjezi, M., 2016. Prediction of seismic slope stability through combination of particle swarm optimization and neural network. Engineering with Computers, 32, 85-97.
Grimit, E.P., Mass, C.F., 2007. Measuring the Ensemble Spread–Error Relationship with a Probabilistic Approach: Stochastic Ensemble Results. Monthly Weather Review, 135, 203-221.
Harsha, V., Sanandam, B., Akhil, G., Ankit, G., Sreedeep, S., 2017. Compressive strength analysis of soil reinforced with fiber extracted from water hyacinth. Engineering Computations, 34, 330-342.
Hatami, K., Bathurst, R.J., 2005. Development and verification of a numerical model for the analysis of geosynthetic-reinforced soil segmental walls under working stress conditions. Canadian Geotechnical Journal, 42, 1066-1085.
Hoang, N.-D., Pham, A.-D., 2016. Hybrid artificial intelligence approach based on metaheuristic and machine learning for slope stability assessment: a multinational data analysis. Expert Systems with Applications, 46, 60-68.
Huang, Z., Cui, J., Liu, H., 2004. Chaotic neural network method for slope stability prediction. Chinese Journal of Rock Mechanics and Engineering, 22, 015.
Hwang, S., Guevarra, I.F., Yu, B., 2009. Slope failure prediction using a decision tree: a case of engineered slopes in South Korea. Engineering Geology, 104, 126-134.
Kang, F., Han, S., Salgado, R., Li, J., 2015. System probabilistic stability analysis of soil slopes using Gaussian process regression with Latin hypercube sampling. Computers and Geotechnics, 63, 13-25.
Kang, F., Xu, B., Li, J., Zhao, S., 2017. Slope stability evaluation using Gaussian processes with various covariance functions. Applied Soft Computing, 60, 387-396.
Karstunen, M., Leoni, M., 2008. Geotechnics of soft soils: focus on ground improvement, 2nd International Workshop held in Glasgow. CRC Press, Scotland.
Li, D.-Q., Qi, X.-H., Cao, Z.-J., Tang, X.-S., Phoon, K.-K., Zhou, C.-B., 2016. Evaluating slope stability uncertainty using coupled Markov chain. Computers and Geotechnics, 73, 72-82.
Li, D.-Q., Zhang, F.-P., Cao, Z.-J., Zhou, W., Phoon, K.-K., Zhou, C.-B., 2015. Efficient reliability updating of slope stability by reweighting failure samples generated by Monte Carlo simulation. Computers and Geotechnics, 69, 588-600.
Li, J., Wang, F., 2010. Study on the forecasting models of slope stability under data mining.
Li, J., Wang, F., 2012. Study on the forecasting models of slope stability under data mining, 12th Biennial International Conference on Engineering, Construction, and Operations in Challenging Environments; and Fourth NASA/ARO/ASCE Workshop on Granular Materials in Lunar and Martian Exploration American Society of Civil Engineers, Honolulu, Hawaii, United States.
Liu, L.-L., Cheng, Y.-M., 2016. Efficient system reliability analysis of soil slopes using multivariate adaptive regression splines-based Monte Carlo simulation. Computers and Geotechnics, 79, 41-54.
Lu, P., Rosenbaum, M.S., 2003. Artificial Neural Networks and Grey systems for the prediction of slope stability. Natural Hazards, 30, 383-398.
Ma, J., Tang, H., Liu, X., Wen, T., Zhang, J., Tan, Q., Fan, Z., 2018. Probabilistic forecasting of landslide displacement accounting for epistemic uncertainty: a case study in the three Gorges reservoir area, China. Landslides.
Mahdiyar, A., Hasanipanah, M., Armaghani, D.J., Gordan, B., Abdullah, A., Arab, H., Majid, M.Z.A., 2017. A Monte Carlo technique in safety assessment of slope under seismic condition. Engineering with Computers, 33, 807-817.
Najjar, S.S., Sadek, S., Alcovero, A., 2013. Quantification of model uncertainty in shear strength predictions for fiber-reinforced sand. Journal of Geotechnical and Geoenvironmental Engineering, 139, 116-133.
Palisade, 2007. Guide to using risk analysis and simulation add-in for Microsoft Excel Palisade Corporation, Ithaca, New York.
Pham, B.T., Son, L.H., Hoang, T.-A., Nguyen, D.-M., Tien Bui, D., 2018. Prediction of shear strength of soft soil using machine learning methods. CATENA, 166, 181-191.
Pham, B.T., Tien Bui, D., Prakash, I., Dholakia, M.B., 2017. Hybrid integration of Multilayer Perceptron Neural Networks and machine learning ensembles for landslide susceptibility assessment at Himalayan area (India) using GIS. CATENA, 149, 52-63.
Popescu, R., Deodatis, G., Nobahar, A., 2005. Effects of random heterogeneity of soil properties on bearing capacity. Probabilistic Engineering Mechanics, 20, 324-341.
Prayogo, D., Susanto, Y.T.T., 2018. Optimizing the prediction accuracy of friction capacity of driven piles in cohesive soil using a novel self-tuning Least Squares Support Vector Machine. Advances in Civil Engineering, 2018, 9.
Radulovic, F., Poveda-Villalón, M., Vila-Suero, D., Rodríguez-Doncel, V., García-Castro, R., Gómez-Pérez, A., 2015. Guidelines for linked data generation and publication: an example in building energy consumption. Automation in Construction, 57, 178-187.
Rubinstein, R.Y., Kroese, D.P., 2016. Simulation and the Monte Carlo Method. Wiley.
Rubinstein, R.Y., Kroese, D.P., 2017. Simulation and the Monte Carlo method. John Wiley & Sons, Inc., Hoboken, New Jersey, Canada.
Rukhaiyar, S., Alam, M.N., Samadhiya, N.K., 2017. A PSO-ANN hybrid model for predicting factor of safety of slope. International Journal of Geotechnical Engineering, 1-11.
Sakellariou, M.G., Ferentinou, M.D., 2005. A study of slope stability prediction using neural networks. Geotechnical & Geological Engineering, 23, 419.
Samui, P., 2008. Slope stability analysis: a support vector machine approach. Environmental Geology, 56, 255.
Samui, P., 2012. Application of statistical learning algorithms to ultimate bearing capacity of shallow foundation on cohesionless soil. International Journal for Numerical and Analytical Methods in Geomechanics, 36, 100-110.
Samui, P., Kothari, D.P., 2011. Utilization of a least square support vector machine (LSSVM) for slope stability analysis. Scientia Iranica, 18, 53-58.
Sarkar, S., Vinay, S., Raj, R., Maiti, J., Mitra, P., 2018. Application of optimized machine learning techniques for prediction of occupational accidents. Computers & Operations Research.
Shirzadi, A., Shahabi, H., Chapi, K., Bui, D.T., Pham, B.T., Shahedi, K., Ahmad, B.B., 2017. A comparative study between popular statistical and machine learning methods for simulating volume of landslides. CATENA, 157, 213-226.
Song, Y., Gong, J., Gao, S., Wang, D., Cui, T., Li, Y., Wei, B., 2012. Susceptibility assessment of earthquake-induced landslides using Bayesian network: a case study in Beichuan, China. Computers & Geosciences, 42, 189-199.
Suman, S., Khan, S.Z., Das, S.K., Chand, S.K., 2016. Slope stability analysis using artificial intelligence techniques. Natural Hazards, 84, 727-748.
Thompson, K.M., 1999. Software review of distribution fitting programs: Crystal Ball and BestFit Add-In to @RISK. Human and Ecological Risk Assessment: An International Journal, 5, 501-508.
Timofeev, R., 2004. Classification and regression trees (CART) theory and applications. Humboldt University, Berlin.
Vapnik, V.N., 1995. The nature of statistical learning theory. Springer, New York.
Wang, G., Sassa, K., 2003. Pore-pressure generation and movement of rainfall-induced landslides: effects of grain size and fine-particle content. Engineering Geology, 69, 109-125.
Wang, Y., Cao, Z., Au, S.-K., 2010. Practical reliability analysis of slope stability by advanced Monte Carlo simulations in a spreadsheet. Canadian Geotechnical Journal, 48, 162-172.
Wilks, D.S., 2005. Statistical Methods in the Atmospheric Sciences. Academic Press.
Witten, I., Frank, E., Hall, M., 2016. Data mining: practical machine learning tools and techniques. Todd Green, United State.
Wu, X., Kumar, V., Ross Quinlan, J., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G.J., Ng, A., Liu, B., Yu, P.S., Zhou, Z.-H., Steinbach, M., Hand, D.J., Steinberg, D., 2008. Top 10 algorithms in data mining. Knowledge and Information Systems, 14, 1-37.
Wu, X.Z., 2013. Trivariate analysis of soil ranking-correlated characteristics and its application to probabilistic stability assessments in geotechnical engineering problems. Soils and Foundations, 53, 540-556.
Wu, X.Z., 2017. Implementing statistical fitting and reliability analysis for geotechnical engineering problems in R. Georisk: Assessment and Management of Risk for Engineered Systems and Geohazards, 11, 173-188.
Wu, Z.-Y., Shi, Q., Guo, Q.-q., Chen, J.-K., 2017. CST-based first order second moment method for probabilistic slope stability analysis. Computers and Geotechnics, 85, 51-58.
Xue, X., 2017. Prediction of slope stability based on hybrid PSO and LSSVM. Journal of Computing in Civil Engineering, 31, 04016041.
Zhou, X.-P., Huang, X.-C., 2018. Reliability analysis of slopes using UD-based response surface methods combined with LASSO. Engineering Geology, 233, 111-123.