臺灣博碩士論文加值系統

在工業生產環境中，機器設備屬於高成本的固定資產，因此故障診斷在製造業的設備維護中非常的重要。隨著物聯網與大數據等新興技術的發展，預測性維護透過機器學習對資料做分析，以進行設備的狀態監測與故障診斷等預測，而故障診斷當中的故障分類可以確定故障的發生、故障的類型以及故障的原因，因此在工業製造中能有效作為設備維護管理的參考依據。由於在工業製造過程中所收集到的數據普遍具有遺漏值、維度與數據不平衡等問題，加上以往的分類器皆建構在數據平衡的情況下，使分類錯誤率相當高且無法真正觀察到我們主要想關注的對象，這將使故障分類面臨了挑戰。

本研究以一組來自UCI資料庫具有高維且資料不平衡的資料集以及一組某工廠機器中實際收集的資料不平衡資料集作為研究對象，以減少多數法與SMOTE兩種採樣方法來處理資料不平衡的問題，搭配卡方特徵選取方法找出得分較高的重要特徵後，以10交叉驗證進行參數的調整，最後以決策樹(DT)、隨機森林(RF)、XGBoost與K近鄰(KNN)四種機器學習演算法進行分類模型的建立，經測試組資料的測試後，以正確率與AUC值作為績效評估的指標，最後結果顯示XGBoost的績效較其他三個模型好，在兩組資料集中皆獲得0.9以上的正確率與AUC值，進而探討機器設備故障的原因，作為設備維護的重要資訊。

In the industrial production environment, machinery equipments are the high-cost fixed asset, so fault diagnosis is a very important part of manufacturing in equipment maintenance. With the development of emerging technologies of the Internet of Things and Big Data and predictive maintenance is through machine learning for data analysis to make predictions such as equipment condition monitoring and fault diagnosis. The fault classification in fault diagnosis can determine the occurrence of the fault, the type of fault and the cause of the fault, thus it could be effectively used as a reference for equipment maintenance management in industrial manufacturing. Due to the data collected in the industrial processes generally have problems such as missing values, dimensions, and data imbalances, make the classification error rate is quite high, which the failure classification faces challenges.

This study applied the UCI database and collected in a factory machine data. To reduce the problem of data imbalance by the Under Sampling method and SMOTE method in two sampling methods, after using the chi-square feature selection method to find the important features with higher scores, that adjust the parameters with 10 cross-validations. Finally, the classification model is built with the decision tree (DT), random forest (RF), XGBoost, and K nearest neighbor (KNN). The accuracy and AUC are used as performance evaluation indicators. The final results show that the performance of XGBoost is better than the others, and the accuracy and AUC are obtained more than 90% in both data sets, and then the cause of machine equipment failure is discussed as important information for equipment maintenance.

摘要 i
Abstract ii
目錄 iii
表目錄 v
圖目錄 vi
第一章 緒論 1
1.1 研究背景 1
1.2 研究動機 3
1.3 研究目的 5
1.4 研究架構 6
第二章 文獻探討 8
2.1 預測性維護 10
2.1.1 設備維護類別 10
2.1.2 產品故障分類 12
2.2 資料不平衡 13
2.2.1 資料不平衡相關之問題 13
2.2.2 處理資料不平衡相關方法 15
2.3 高維數據 17
2.3.1 高維數據相關之問題 17
2.3.2 特徵選取相關方法 18
2.4 機器學習 19
2.4.1 集成式學習方法(Ensemble Learning Method) 19
2.4.2 決策樹(Decision tree，DT) 21
2.4.3 隨機森林(Random Forest，RF) 22
2.4.4 XGBoost(eXtreme Gradient Boosting) 23
2.4.5 K近鄰(K Nearest Neighbor，KNN) 24
第三章 研究方法 26
3.1 資料分析流程 26
3.2 資料說明 27
3.3 資料前處理 28
3.3.1 遺漏值 28
3.3.2 資料轉換 28
3.4 重採樣方法 29
3.4.1 減少多數法 29
3.4.2 SMOTE 29
3.5特徵選取方法 31
3.6 機器學習模型 32
3.6.1 決策樹模型 32
3.6.2 隨機森林模型 35
3.6.3 XGBoost模型 38
3.6.4 KNN模型 43
3.7 模型績效評估 45
3.8 模型驗證 48
第四章 實驗結果與分析 49
4.1 模型建立 49
4.1.1 資料拆分與10折交叉驗證 49
4.1.2 卡方特徵選取 49
4.1.3 模型參數設定 50
4.2 決策樹模型績效與測試 52
4.3 隨機森林模型績效與測試 57
4.4 XGBoost模型績效與測試 61
4.5 KNN模型績效與測試 64
4.6 模型比較與小結 66
第五章 討論 68
第六章 結論與未來研究 69
5.1 結論 69
5.2 未來研究 70
參考文獻 71

[1]方禹涵(2019)。開拓 AI、VR 潛能，看 5G 如何顛覆「未來製造業」。TechOrange。https://buzzorange.com/techorange/2019/08/05/5g-new-era-of-manufacturing/。
[2]張殷綺(2017)。預測性維護讓製造業降低營運成本。DIGITIMES。http://www.digitimes.com.tw/iot/article.asp?cat=158&cat1=&cat2=&id=0000519761_9348ENWW5PYQM72SCK7GT&social_share=y。
[3]金屬工業研究發展中心(2019)。傳產升級智慧製造第一步-模具與廢料處理成本如何聰明省下來？。TechOrange。 https://buzzorange.com/techorange/2019/05/15/smart-manufacturing-in-metal-industry/。
[4]廖家宜(2019)。設備維護管理從預防性走向預測性 可望減少70%故障機率。DIGITIMES。https://www.digitimes.com.tw/iot/article.asp?cat=158&cat1=20&cat2=10&id=0000568761_40A8EEYG5CPSR22T51KK6。
[5]Scully, P. (2019). Predictive Maintenance initiatives saved organizations $17B in 2018, as the number of vendors surges. IoT Analytics. https://iot-analytics.com/numbers-of-predictive-maintenance-vendors-surges/。
[6]Chuang, A. (2019)。AI驅動智慧製造加速發展。EE Times Taiwan 電子工程專輯網。https://www.eettaiwan.com/news/article/20190924NT11-AI-drive-smart-manufacturing-development。
[7]TrendForce(2019)。2022年智慧製造市場規模將近3,700億美元。EE Times Taiwan 電子工程專輯網。https://www.eettaiwan.com/news/article/20190823NT21-Smart-manufacturing
[8]Kennedy, S. (2017). RxM: What is prescriptive maintenance, and how soon will you need it?. Plant Services.
https://www.plantservices.com/articles/2017/rxm-what-is-prescriptive-maintenance-and-how-soon-will-you-need-it/。
[9]林以真(2013)。針對不平衡資料鑑別分析之二項隨機子空間分類法。輔仁大學統計資訊學系應用統計碩士班，碩士論文。
[10]林佳蒨(2012)。支援向量機於不平衡資料類別問題之應用。國立暨南國際大學資訊管理學系，碩士論文。
[11]林冠宇(2014)。發展改良式支持向量資料描述改善不平衡資料分類。國立臺北科技大學工業工程與管理系所，碩士論文。
[12]林鼎家(2013)。比較特徵選擇與特徵擷取方法在評估中文適讀性的效果。國立嘉義大學資訊工程學系研究所，碩士論文。
[13]許哲彰(2019)。不平衡數據的機器學習發展暨可視化辨識模型之應用。國立中央大學機械工程學系，博士論文。
[14]郭皇志(2018)。晶圓切割之不平衡資料集的分類。國立成功大學工業與資訊管理學系碩士在職專班，碩士論文。
[15]張智綸(2018)。引進預測性維護之探討- 以鍛造模具設備為例。國立臺灣科技大學工業管理系碩士班，碩士論文。
[16]張乃楠(2018)。基於混合方法探究不平衡數據分類問題。元智大學資訊工程學系，碩士論文。
[17]陳逸真(2017)。不平衡資料分類方法之比較，國立臺北大學統計學系。碩士論文。
[18]楊志隆(2016)。應用合議法改善不平衡資料分類流程之研究。國防大學運籌管理學系，碩士論文。
[19]鄧惟中(2019)。一個結合極限梯度提升分類模型與關鍵字抽出方法的釣魚網站偵測服務架構。國立臺灣科技大學資訊工程系，碩士論文。
[20]劉順富(2019)。用 XGBOOST 演算法預測台灣指數期貨。國立雲林科技大學財務金融系，碩士論文。
[21]蕭凱駿(2019)。應用集成方法於半導體蝕刻製程晶圓分類之研究。國立臺灣科技大學工業管理系，碩士論文。
[22]龔健生(2016)。分類技術於類別不平衡資料集之研究。國立中央大學資訊管理學系在職專班，碩士論文。
[23]Anukrishna, P. R., & Paul, V. (2017). A review on feature selection for high dimensional data. 2017 International Conference on Inventive Systems and Control (ICISC).
[24]Basak, S., Kar, S., Saha, S., Khaidem, L., & Dey, S. R. (2019). Predicting the direction of stock market prices using tree-based classifiers. The North American Journal of Economics and Finance, 47, 552-567.
[25]Bommert, A., Sun, X., Bischl, B., Rahnenführer, J., & Lang, M. (2020). Benchmark for filter methods for feature selection in high-dimensional classification data. Computational Statistics & Data Analysis, 143, 106839.
[26]Carvalho, T. P., Soares, F. A. A. M. N., Vita, R., Francisco, R. P., Basto, J., & Alcalá, S. G. S. (2019). A systematic literature review of machine learning methods applied to predictive maintenance. Computers & Industrial Engineering, 137, 106024.
[27]Chawla, V., Bowyer, W., Hall, O., & Kegelmeyer, W. (2002). SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research, 16, 321-357
[28]Chen, T., & Guestrin, C. (2016). XGBoost: A Scalable Tree Boosting System. KDD '16 Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 785-794.
[29]Chen, C., & Ge, Z. (2019). SVM-tree and SVM-forest algorithms for imbalanced fault classification in industrial processes. IFAC Journal of Systems and Control, 8, 100052.
[30]Dou, D., & Zhou, S. (2016). Comparison of four direct classification methods for intelligent fault diagnosis of rotating machinery. Applied Soft Computing, 46, 459-468.
[31]Galar, M., Fernandez, A., Barrenechea, E., Bustince, H. & Herrera, F. (2012). A Review on Ensembles for the Class Imbalance Problem: Bagging-, Boosting-, and Hybrid-Based Approaches. IEEE Transactions on Systems, Man, and Cybernetics, Part C, 42, 463-484.
[32]Ge, Z., Song, Z., Ding, S. X., & Huang, B. (2017). Data Mining and Analytics in the Process Industry: The Role of Machine Learning. IEEE Access, 5, 20590-20616.
[33]Hamerla, G., Meyer, H. J., Schob, S., Ginat, D. T., Altman, A., Lim, T., Gihr, G. A., Rizea, D. H., Hoffmann, K. T., & Surov, A. (2019). Comparison of machine learning classifiers for differentiation of grade 1 from higher gradings in meningioma: A multicenter radiomics study. Magnetic Resonance Imaging, 63, 224-249.
[34]Hosseini, E. S., & Moattar, M. H. (2019). Evolutionary feature subsets selection based on interaction information for high dimensional imbalanced data classification. Applied Soft Computing, 82, 105581.
[35]Hung, Y., & Lai, S. (2019). The Survey of Predictive Maintenance and Cloud Computing. 2019 International Conference on Business, Economics and Management in the Age of Intelligence, 58.
[36]Ikram, S., & Cherukuri, A. (2017). Intrusion detection model using fusion of chi-square feature selection and multi class SVM. Journal of King Saud University - Computer and Information Sciences, 29, 4, 462-472.
[37]Jing, C., & Hou, J. (2015). SVM and PCA based fault classification approaches for complicated industrial process. Neurocomputing, 167, 636-642.
[38]Jović, A., Brkić, K., & Bogunović, N. (2015). A review of feature selection methods with applications. 2015 38th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO).
[39]Karabadji, N. E, l., Seridi, H., Khelf, l., Azizi, N., & Boulkroune, R. (2014). Improved decision tree construction based on attribute selection and data sampling for fault diagnosis in rotating machines. Engineering Applications of Artificial Intelligence, 35, 71-83.
[40]Kim, A., Oh, K., Jung, J. Y., & Kim, B. (2017). Imbalanced classification of manufacturing quality conditions using cost-sensitive decision tree ensembles. International Journal of Computer Integrated Manufacturing, 31, 8, 701-717.
[41]Kumar, A., Chinnam, R., & Tseng, F. (2019). An HMM and polynomial regression based approach for remaining useful life and health state estimation of cutting tools. Computers & Industrial Engineering, 128, 1008-1014.
[42]Lee, J., Kao, H. A., & Yang, S. (2014). Service Innovation and Smart Analytics for Industry 4.0 and Big Data Environment. Procedia CIRP, 16, 3-8.
[43]Lee J., Wu, F., Zhao, W., Ghaffari, M., Liao, L., & Siegel, D. (2014). Prognostics and health management design for rotary machinery systems—Reviews, methodology and applications. Mechanical Systems and Signal Processing, 42, 1-2, 314-334.
[44]Liu, Y., & Ge, Z. (2019). Deep ensemble forests for industrial fault classification. IFAC Journal of Systems and Control, 10, 100071.
[45]Liu, Y., & Ge, Z. (2018). Weighted random forests for fault classification in industrial processes with hierarchical clustering model selection. Journal of Process Control, 64, 62-70.
[46]Luo, L., Bao, S., & Peng, X. (2019). Robust monitoring of industrial processes using process data with outliers and missing values. Chemometrics and Intelligent Laboratory Systems, 192, 103827.
[47]Malhotra, R., & Kamal, S. (2019). An empirical study to investigate oversampling methods for improving software defect prediction using imbalanced data. Neurocomputing, 343, 120-140.
[48]Nobre, R., & Neves, R. F. (2019). Combining Principal Component Analysis, Discrete Wavelet Transform and XGBoost to trade in the financial markets. Expert Systems with Applications, 125, 181-194.
[49]Patel, R., & Giri, V. K. (2016). Feature selection and classification of mechanical fault of an induction motor using random forest classifier. Perspectives in Science, 8, 334-337.
[50]Pappu, V., & Pardalos, P. (2014). High-Dimensional Data Classification. Clusters, Orders, and Trees: Methods and Applications, pp119-150.
[51]Pandya, D. H. Upadhyay, S. H., & Harsha, S. P. (2013). Fault diagnosis of rolling element bearing with intrinsic mode function of acoustic emission data using APF-KNN. Expert Systems with Applications, 40, 10, 4137-4145.
[52]Sakthivel, N. R., Sugumaran, V., & Babudevasenapati, S. (2010). Vibration based fault diagnosis of monoblock centrifugal pump using decision tree. Expert Systems with Applications, 37, 6, 4040-4049.
[53]Sarmiento, J. R. R., Monroy, J., Moreno, F. A., Galindo, C., Bonelo, J. M., & Jimenez, J. G. (2020). A predictive model for the maintenance of industrial machinery in the context of industry 4.0. Engineering Applications of Artificial Intelligence, 87, 103289.
[54]Susto, G. A., Schirru, A., Pampuri, S., McLoone, S., & Beghi, A. (2015). Machine Learning for Predictive Maintenance: A Multiple Classifier Approach. IEEE Transactions on Industrial Informatics, 11, 3, 812-820.
[55]Susto, G. A., Beghi, A., & Luca, C. (2012). A Predictive Maintenance System for Epitaxy Processes Based on Filtering and Prediction Techniques. IEEE Transactions on Semiconductor Manufacturing, 25, 638-649.
[56]Toroody, A., Abaei, M., Arzaghi, E., Toroody, F., Carloa, F., & Abbassi, R. (2019). Multi-level optimization of maintenance plan for natural gas system exposed to deterioration process. Journal of Hazardous Materials, 362, 15, 412-423.
[57]Wu, Z., Lin, W., & Ji, Y. (2018). An Integrated Ensemble Learning Model for Imbalanced Fault Diagnostics and Prognostics. IEEE Access, 6, 8394-8402.
[58]Zheng, J., Wang, H., Song, Z., & Ge, Z. (2019). Ensemble semi-supervised Fisher discriminant analysis model for fault classification in industrial processes. ISA Transactions, 92, 109-117.
[59]Zio, E., Baraldi, P., & Popescu, I, C. (2009). A fuzzy decision tree method for fault classification in the steam generator of a pressurized water reactor. Annals of Nuclear Energy, 36, 8, 1159-1169.
[60]UCI資料庫。http://archive.ics.uci.edu/ml/datasets/secom。