(34.237.124.210) 您好!臺灣時間:2021/03/02 07:26
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果

詳目顯示:::

我願授權國圖
: 
twitterline
研究生:劉思葦
研究生(外文):Szu-Wei Liu
論文名稱:以機器學習演算法預測銀行虛擬資源使用模式
論文名稱(外文):Predicting and Analyzing Resource Utilization for Bank’s Virtual Machines Using Machine Learning Algorithm
指導教授:曹承礎曹承礎引用關係
指導教授(外文):Seng-Cho Chou
口試委員:蔡益坤周子元
口試日期:2019-07-12
學位類別:碩士
校院名稱:國立臺灣大學
系所名稱:資訊管理學研究所
學門:電算機學門
學類:電算機一般學類
論文種類:學術論文
論文出版年:2019
畢業學年度:107
語文別:中文
論文頁數:40
中文關鍵詞:智能維運系統異常預測機器學習卷積神經網路長短期記憶模型
DOI:10.6342/NTU201901776
相關次數:
  • 被引用被引用:0
  • 點閱點閱:98
  • 評分評分:系統版面圖檔系統版面圖檔系統版面圖檔系統版面圖檔系統版面圖檔
  • 下載下載:0
  • 收藏至我的研究室書目清單書目收藏:0
隨著數位科技的進步與發展,產業開始應用大型且複雜系統,監控這些大型資源和突發事件會產生大量數據,再加上大數據分析技術普及,智能維運的概念隨之興起,效率管理、異常偵測與預測以及異常自動化處理為其的主要功能,能夠幫助企業降低維運成本、提升客戶體驗,因此導入智能維運技術為系統維運部門非常關切的議題,故本研究蒐集銀行虛擬資源監控資料,聚焦於資源使用率和異常預測。
過去的研究中,針對虛擬資源使用率和異常預測,主要透過蒐集監控資料,以統計模型和機器學習模型進行預測,而本研究則實驗適合處理時間序列資料的深度學習模型:卷積神經網路、長短期記憶模型以及將兩者結合的演算法,同時實驗不同特徵之下模型的預測效果,並與過去的研究方法進行比較,改善未來針對資源使用率以及異常預測模型。根據實驗結果,本研究採用長短期記憶模型結合卷積神經網路的演算法無論在資源使用率或異常預測上,相較於過去研究使用的機器學習演算法,都有更突出的表現。
透過本研究成果,希望能幫助銀行更有效率的管理內部資源,更能提前預測系統異常,降低服務中斷的機率,減少內部系統維運的人工成本、同時提升管理效率以及系統穩定度。
Owing to new technological advances, most of the industries have implemented some large-scale and complicated systems as their infrastructure. Because of the improvement of Big Data Analysis and the numerous data produced by monitoring these systems, the concept of Artificial Intelligence for IT Operations (AIOps) was born. The main purposes of AIOps are to efficiently manage resources, and predict abnormal of the system and automatically deal with emergencies. As a result, introducing AIOps into IT infrastructure is an urgent demand for many companies. In our work, we collected monitoring data of bank’s IaaS platform and focused on the methods to implement AIOps in the bank.
In the past, many research implemented statistical models and machine learning models to predict virtual resource utilization and abnormal situation. In our work, we applied deep learning models, Convolutional Neural Networks(CNN), Long Short-Term Memory(LSTM) and the combination of these two models, and experiment different length of features to improve the performance of the prediction. According to the result of the experiments, the combination of LSTM and CNN is the most effective model to predict utilization and irregular situation among the algorithm used in the previous research.
Our research has favorable results which are able to predict resource usage and unusual patterns. By achieving these goals, our work could help the bank to manage the resource more efficiently, reduce the possibility of the interruption of the services as well as the cost of the maintenance, and also ensure the stability of the systems.
口試委員會審定書 #
致謝 i
中文摘要 ii
ABSTRACT iii
目錄 iv
圖目錄 vi
表目錄 vii
第一章 緒論 1
1.1 研究背景與動機 1
1.2 研究目的 3
1.3 研究流程 3
第二章 文獻回顧 4
2.1 資訊系統維運發展 4
2.2 虛擬資源使用量預測 6
2.3 資源異常模式檢測與預測 7
2.3 深度學習演算法 8
2.2.1 卷積神經網絡 8
2.2.2 遞迴神經網絡和長短期記憶模型 9
2.4 小結 11
第三章 研究方法 12
3.1 研究流程 12
3.2 資料集 13
3.3 資料前處理 14
3.4 資源使用率預測模型 19
3.5 資源異常預測模型 21
第四章 實驗結果與分析 23
4.1 實驗評估指標 23
4.2 資源使用率預測結果 24
4.3 資源使用異常預測結果 29
第五章 結論與未來展望 36
5.1 結論 36
5.2 未來展望 37
參考文獻 38
[1]L. Yi, X. Deng, M. Wang, D. Ding, and Y. Wang, (2017) "Localized confident information coverage hole detection in internet of things for radioactive pollution monitoring, " IEEE Access, vol. 5, no. 18665–18674.
[2]Dang, Y., Lin, Q., & Huang, P. (2019). “AIOps: real-world challenges and research innovations”. In Proceedings of the 41st International Conference on Software Engineering: Companion Proceedings (pp. 4-5). IEEE Press.
[3]F. Qiu, B. Zhang, and J, Guo, “A deep learning approach for VM workload prediction in the cloud,” in Proc. IEEE/ACIS Int. Conf. Softw. Eng., Artif. Intell., Netw. Parallel/Distrib. Comput., 2016, pp. 319–324.
[4]An ENTERPRISE MANAGEMENT ASSOCIATES® (EMA™) White Paper. (2015). “Application Performance Monitoring. ” Retrieved from https://kapost-files-prod.s3.amazonaws.com/published/5554fae276c31e448f00017b/ema-report-application-performance-monitoring-apm-2015-industry-challenges-stat-of-the-art-and-the-case-for-unified-monitoring.pdf (Dec. 1,2018)
[5]裴丹、張聖林、裴昶華(2017)。基於機器學習的智能維運。中國計算機學會通訊,12,68-73。
[6]Gartner. (2018) “Market Guide for AIOps Platforms”. Retrieved from https://www.gartner.com/doc/reprints?id=1-5YRNF78&ct=181218&st=sb (June. 21,2019)
[7]Calheiros, R.N., Masoumi, E., Ranjan, R., Buyya, R. (2015) Workload prediction using arima model and its impact on cloud applications’ QoS. TCC 3(4)
[8]Khan, A., Yan, X., Tao, S., Anerousis, N. (2012), “Workload characterization and prediction in the cloud: a multiple time series approach”. In NOMS
[9]Z. Xiao, W. Song and Q. Chen. (2013) “Dynamic resource allocation using virtual machines for cloud computing environment”. Parallel and Distributed Systems, IEEE Transactions on, 24(6): 1107-1117.
[10]Xue, J., Yan, F., Birke, R., Chen, L.Y., Scherer, T., Smirni, E. (2015) “Practise: robust prediction of data center time series”. In CNSM
[11]Zhang, W., Li, B., Zhao, D., Gong, F. and Lu, Q. (2016). Workload Prediction for Cloud Cluster Using a Recurrent Neural Network. In 2016 International Conference on Identification, Information and Knowledge in the Internet of Things (IIKI).
[12]Adane, P. D., & Kakde, O. G. (2018). Predicting Resource Utilization for Cloud Workloads Using Machine Learning Techniques. In 2018 Second International Conference on Inventive Communication and Computational Technologies (ICICCT) (pp. 1372-1376). IEEE.
[13]Y. Tan and X. Gu, (2010) “On predictability of system anomalies in real world,” In Modeling, Analysis & Simulation of Computer and Telecommunication Systems (MASCOTS), 2010 IEEE International Symposium on, pp. 133–140, IEEE.
[14]Wang, T., Zhang, W., Wei, J., & Zhong, H. (2012). Workload-aware online anomaly detection in enterprise applications with local outlier factor. In Computer Software and Applications Conference (COMPSAC), 2012 IEEE 36th Annual(pp. 25-34). IEEE.
[15]Huang, S., Fung, C., Liu, C., Zhang, S., Wei, G., Luan, Z., & Qian, D. (2017). Arena: Adaptive real-time update anomaly prediction in cloud systems. In 2017 13th International Conference on Network and Service Management (CNSM) (pp. 1-9). IEEE.
[16]Hongbing Wang, Lei Wang, Qi Yu, Zibin Zheng, Athman Bouguettaya, and Michael R Lyu. (2017). “Online reliability prediction via motifs-based dynamic bayesian networks for service-oriented systems”. IEEE Transactions on Software Engineering 43, 6 (2017), 556–579.
[17]Chen, Y., Yang, X., Lin, Q., Zhang, H., Gao, F., Xu, Z., ... & Li, H. (2019). Outage Prediction and Diagnosis for Cloud Service Systems. In The World Wide Web Conference (pp. 2659-2665). ACM.
[18]Le Cun, B.B.; Denker, J.S.; Henderson, D.; Howard, R.E.; Hubbard, W.; Jackel, L.D. (1990). “Handwritten digit recognition with a back-propagation network.” In Advances in neural information processing systems 2; Morgan Kaufmann Publishers: San Francisco, CA, USA; pp. 396–404.
[19]Jarrett, K.; Kavukcuoglu, K.; Lecun, Y. (2009). “What is the best multi-stage architecture for object recognition? ” In IEEE 12th International Conference on Computer Vision, Kyoto, Japan; pp. 2146–2153.
[20]Kim, Y. Convolutional neural networks for sentence classification. (2014). arXiv; arXiv:1408.5882.
[21]Zhao, R.; Mao, K. “Topic-Aware Deep Compositional Models for Sentence Classification”. (2017).In IEEE/ACM Trans. Audio Speech Lang. Process. pp. 248–260.
[22]Y. Zheng, Q. Liu, E. Chen, Y. Ge, and J. L. Zhao. (2014) “Time series classification using multi-channels deep convolutional neural networks”. In Web-Age Information Management; Springer, pp 298–310.
[23]Sepp Hochreiter; Jürgen Schmidhuber. (1997) “Long Short-Term Memory”. Neural Computation. pp. 1735-1780.
[24]Klaus Greff; Rupesh Kumar Srivastava; Jan Koutník; Bas R. Steunebrink; Jürgen Schmidhuber (2015). “LSTM: A Search Space Odyssey”. IEEE Transactions on Neural Networks and Learning Systems. 28 (10): 2222. arXiv:1503.04069
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top
系統版面圖檔 系統版面圖檔