跳到主要內容

臺灣博碩士論文加值系統

(44.221.66.130) 您好!臺灣時間:2024/06/20 23:40
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果 :::

詳目顯示

我願授權國圖
: 
twitterline
研究生:駱郁雯
研究生(外文):Lo, Yu-Wen
論文名稱:基於強化學習於異構雲端資料中心的綠色虛擬機分配框架
論文名稱(外文):Green Virtual Machine Allocation Framework in Heterogeneous Cloud Data Center Using DQL
指導教授:古政元古政元引用關係
指導教授(外文):Ku, Cheng-Yuan
口試委員:何蕙萍莊詠婷
口試委員(外文):Ho, Hui-PingChuang, Yung-Ting
口試日期:2023-07-14
學位類別:碩士
校院名稱:國立陽明交通大學
系所名稱:資訊管理研究所
學門:電算機學門
學類:電算機一般學類
論文種類:學術論文
論文出版年:2023
畢業學年度:111
語文別:英文
論文頁數:44
中文關鍵詞:虛擬機分配強化學習異構資料中心降低能耗
外文關鍵詞:Virtual Machine AllocationDQLHeterogeneous Data CenterEnergy Consumption Reduction
相關次數:
  • 被引用被引用:0
  • 點閱點閱:88
  • 評分評分:
  • 下載下載:0
  • 收藏至我的研究室書目清單書目收藏:0
誌謝 i
摘要 ii
Abstract iii
Contents iv
List of Figures v
List of Tables vi
Chapter 1 Introduction 1
1.1 Background 1
1.2 Motivation 2
1.3 Research Method 3
1.4 Research Process 4
Chapter 2 Related Work 6
2.1 Non-Machine Learning Techniques 6
2.2 Heuristic and Metaheuristic VMA Techniques 6
2.3 Machine Learning VMA Techniques 7
Chapter 3 Proposed Framework 16
3.1 System Overview 16
3.2 Problem Formulation 19
3.3 Proposed DQL VMA Design 22
Chapter 4 Evaluation 25
4.1 Experiment Settings 25
4.2 Evaluation Metrics 30
4.3 Performance Comparison 31
Chapter 5 Conclusion and Future Work 35
References 37
Appendix 43
[1] P. Mell and T. Grance, "The NIST definition of cloud computing," 2011. [Online]. Available: http://faculty.winthrop.edu/domanm/csci411/Handouts/NIST.pdf.
[2] R. Buyya, A. Beloglazov, and J. Abawajy, "Energy-efficient management of data center resources for cloud computing: A vision, architectural elements, and open challenges," arXiv preprint arXiv:1006.0308, 2010. [Online]. Available: https://arxiv.org/abs/1006.0308.
[3] B. Wang, Z. Qi, R. Ma, H. Guan, and A. V. Vasilakos, "A survey on data center networking for cloud computing," Computer Networks, vol. 91, pp. 528-547, 2015. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S138912861500300X.
[4] M. Avgerinou, P. Bertoldi, and L. Castellazzi, "Trends in data centre energy consumption under the european code of conduct for data centre energy efficiency," Energies, vol. 10, no. 10, p. 1470, 2017. [Online]. Available: https://www.mdpi.com/1996-1073/10/10/1470.
[5] V. Petrucci, O. Loques, and D. Mossé, "A framework for dynamic adaptation of power-aware server clusters," in Proceedings of the 2009 ACM symposium on Applied Computing, 2009, pp. 1034-1039. [Online]. Available: https://dl.acm.org/doi/abs/10.1145/1529282.1529509.
[6] M. Zhou, R. Zhang, D. Zeng, and W. Qian, "Services in the cloud computing era: A survey," in 2010 4th International Universal Communication Symposium, 2010: IEEE, pp. 40-46. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/5666772.
[7] B. T. Ahmed, "Virtualization Mechanisms and Tools: A Comprehensive Survey."
[8] A. Mosa and R. Sakellariou, "Dynamic virtual machine placement considering CPU and memory resource requirements," in 2019 IEEE 12th International Conference on Cloud Computing (CLOUD), 2019: IEEE, pp. 196-198. [Online]. Available: https://ieeexplore.ieee.org/document/8814574.
[9] U. Arshad, M. Aleem, G. Srivastava, and J. C.-W. Lin, "Utilizing power consumption and SLA violations using dynamic VM consolidation in cloud data centers," Renewable and Sustainable Energy Reviews, vol. 167, p. 112782, 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1364032122006669.
[10] M. Ghobaei‐Arani, A. A. Rahmanian, M. Shamsi, and A. Rasouli‐Kenari, "A learning‐based approach for virtual machine placement in cloud data centers," International Journal of Communication Systems, vol. 31, no. 8, p. e3537, 2018. [Online]. Available: https://onlinelibrary.wiley.com/doi/full/10.1002/dac.3537.
[11] A. Beloglazov and R. Buyya, "Optimal online deterministic algorithms and adaptive heuristics for energy and performance efficient dynamic consolidation of virtual machines in cloud data centers," Concurrency and Computation: Practice and Experience, vol. 24, no. 13, pp. 1397-1420, 2012. [Online]. Available: https://onlinelibrary.wiley.com/doi/full/10.1002/cpe.1867.
[12] M. C. Silva Filho, C. C. Monteiro, P. R. Inácio, and M. M. Freire, "Approaches for optimizing virtual machine placement and migration in cloud environments: A survey," Journal of Parallel and Distributed Computing, vol. 111, pp. 222-250, 2018. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S074373151730240X.
[13] S. Long, Z. Li, Y. Xing, S. Tian, D. Li, and R. Yu, "A reinforcement learning-based virtual machine placement strategy in cloud data centers," in 2020 IEEE 22nd International Conference on High Performance Computing and Communications; IEEE 18th International Conference on Smart City; IEEE 6th International Conference on Data Science and Systems (HPCC/SmartCity/DSS), 2020: IEEE, pp. 223-230.
[14] R. Shaw, E. Howley, and E. Barrett, "Applying reinforcement learning towards automating energy efficient virtual machine consolidation in cloud data centers," Information Systems, vol. 107, p. 101722, 2022.
[15] A. Aghasi, K. Jamshidi, A. Bohlooli, and B. Javadi, "A decentralized adaptation of model-free Q-learning for thermal-aware energy-efficient virtual machine placement in cloud data centers," Computer Networks, vol. 224, p. 109624, 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1389128623000695.
[16] H. Hasselt, "Double Q-learning," Advances in neural information processing systems, vol. 23, 2010. [Online]. Available: https://proceedings.neurips.cc/paper/2010/hash/091d584fced301b442654dd8c23b3fc9-Abstract.html.
[17] Q. Chou, W. Fan, and J. Zhang, "A Reinforcement Learning Model for Virtual Machines Consolidation in Cloud Data Center," in 2021 6th international conference on automation, control and robotics engineering (CACRE), 2021: IEEE, pp. 16-21. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/9501288.
[18] E. Le Sueur and G. Heiser, "Dynamic voltage and frequency scaling: The laws of diminishing returns," in Proceedings of the 2010 international conference on Power aware computing and systems, 2010, pp. 1-8. [Online]. Available: https://www.usenix.org/legacy/events/hotpower/tech/full_papers/LeSueur.pdf.
[19] C.-M. Wu, R.-S. Chang, and H.-Y. Chan, "A green energy-efficient scheduling algorithm using the DVFS technique for cloud datacenters," Future Generation Computer Systems, vol. 37, pp. 141-147, 2014. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0167739X13001234#br000095.
[20] R. Nathuji and K. Schwan, "Virtualpower: coordinated power management in virtualized enterprise systems," ACM SIGOPS operating systems review, vol. 41, no. 6, pp. 265-278, 2007. [Online]. Available: https://dl.acm.org/doi/abs/10.1145/1323293.1294287.
[21] M. A. Khoshkholghi, M. N. Derahman, A. Abdullah, S. Subramaniam, and M. Othman, "Energy-efficient algorithms for dynamic virtual machine consolidation in cloud data centers," IEEE Access, vol. 5, pp. 10709-10722, 2017. [Online]. Available: https://ieeexplore.ieee.org/document/7937801.
[22] F. Alharbi, Y.-C. Tian, M. Tang, W.-Z. Zhang, C. Peng, and M. Fei, "An ant colony system for energy-efficient dynamic virtual machine placement in data centers," Expert Systems with Applications, vol. 120, pp. 228-238, 2019.
[23] P. Guo, M. Liu, and Z. Xue, "A PSO-based energy-efficient fault-tolerant static scheduling algorithm for real-time tasks in clouds," in 2018 IEEE 4th International Conference on Computer and Communications (ICCC), 2018: IEEE, pp. 2537-2541. [Online]. Available: https://ieeexplore.ieee.org/document/8781005.
[24] N. J. Kansal and I. Chana, "Energy-aware virtual machine migration for cloud computing-a firefly optimization approach," Journal of Grid Computing, vol. 14, pp. 327-345, 2016. [Online]. Available: https://link.springer.com/article/10.1007/s10723-016-9364-0.
[25] A. Y. Nikravesh, S. A. Ajila, and C.-H. Lung, "Towards an autonomic auto-scaling prediction system for cloud resource provisioning," in 2015 IEEE/ACM 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, 2015: IEEE, pp. 35-45. [Online]. Available: https://ieeexplore.ieee.org/document/7194655.
[26] W. Zhang, B. Li, D. Zhao, F. Gong, and Q. Lu, "Workload prediction for cloud cluster using a recurrent neural network," in 2016 International Conference on Identification, Information and Knowledge in the Internet of Things (IIKI), 2016: IEEE, pp. 104-109. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/8281183.
[27] S. Gupta and D. A. Dinesh, "Resource usage prediction of cloud workloads using deep bidirectional long short term memory networks," in 2017 IEEE international conference on advanced networks and telecommunications systems (ANTS), 2017: IEEE, pp. 1-6. [Online]. Available: https://ieeexplore.ieee.org/document/8384098.
[28] M. A. Wiering and M. Van Otterlo, "Reinforcement learning," Adaptation, learning, and optimization, vol. 12, no. 3, p. 729, 2012.
[29] I. R. Galatzer-Levy, K. V. Ruggles, and Z. Chen, "Data science in the Research Domain Criteria era: relevance of machine learning to the study of stress pathology, recovery, and resilience," Chronic Stress, vol. 2, p. 2470547017747553, 2018. [Online]. Available: https://journals.sagepub.com/doi/full/10.1177/2470547017747553.
[30] B. Jang, M. Kim, G. Harerimana, and J. W. Kim, "Q-learning algorithms: A comprehensive classification and applications," IEEE access, vol. 7, pp. 133653-133667, 2019. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/8836506.
[31] T. Fu, Y. Peng, P. Liu, H. Lao, and S. Wan, "Distributed reinforcement learning-based memory allocation for edge-PLCs in industrial IoT," Journal of Cloud Computing, vol. 11, no. 1, p. 73, 2022. [Online]. Available: https://link.springer.com/article/10.1186/s13677-022-00348-9.
[32] M. Tokic and G. Palm, "Value-difference based exploration: adaptive control between epsilon-greedy and softmax," in Annual conference on artificial intelligence, 2011: Springer, pp. 335-346. [Online]. Available: https://link.springer.com/chapter/10.1007/978-3-642-24455-1_33.
[33] P. Dayan and C. Watkins, "Q-learning," Machine learning, vol. 8, no. 3, pp. 279-292, 1992. [Online]. Available: https://d1wqtxts1xzle7.cloudfront.net/35466333/watkins92-libre.pdf?1415404402=&response-content-disposition=inline%3B+filename%3DTechnical_Note_Q_Learning.pdf&Expires=1683014043&Signature=MnkGbIHaCQ9SEHkdhg6GG-YfPuUwkEixFf4E~rEjFqAhah6KKRj-0DduwyFjARitRh4sD9Z95wS9Qx4km3dXsuwfuAAGWYLOdoo9uMOPWZz8MkybdG2IjHtdw0bl6upNTOoJh5M36H3SBPPFCfUxZXm0lcqQmQp1jhDA~B8E4AS6e3ycuFhEaToZ6c~sG9I-2JQal-4WsPpOHu98PLdmSqSvC5uUl~WhHK1DA4Vf1JSqoUm-2eRs90hnLlhXq6paVFc~CNab7JP-VI77fKwu6e38L57X0vTDTHjDCUTRbR8r5MVPf~nkQ44SY9Q10ijkXRGVPuLxPCTLh642z35E9Q__&Key-Pair-Id=APKAJLOHF5GGSLRBV4ZA.
[34] D. Precup, R. S. Sutton, and S. Dasgupta, "Off-policy temporal-difference learning with function approximation," in ICML, 2001, pp. 417-424.
[35] Q. Huang, "Model-based or model-free, a review of approaches in reinforcement learning," in 2020 International Conference on Computing and Data Science (CDS), 2020: IEEE, pp. 219-221. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/9275964.
[36] Y. Qin, H. Wang, S. Yi, X. Li, and L. Zhai, "Virtual machine placement based on multi-objective reinforcement learning," Applied Intelligence, vol. 50, pp. 2370-2383, 2020. [Online]. Available: https://link.springer.com/article/10.1007/s10489-020-01633-3.
[37] A. Jumnal and S. D. Kumar, "Optimal VM Placement Approach Using Fuzzy Reinforcement Learning for Cloud Data Centers," in 2021 Third International Conference on Intelligent Communication Technologies and Virtual Mobile Networks (ICICV), 2021: IEEE, pp. 29-35. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/9388424.
[38] J. Yan, J. Xiao, and X. Hong, "Dueling-DDQN Based Virtual Machine Placement Algorithm for Cloud Computing Systems," in 2021 IEEE/CIC International Conference on Communications in China (ICCC), 2021: IEEE, pp. 294-299. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/9580393.
[39] S. Kumar T, S. D. S. Mustapha, P. Gupta, and R. P. Tripathi, "Hybrid approach for resource allocation in cloud infrastructure using random forest and genetic algorithm," Scientific Programming, vol. 2021, pp. 1-10, 2021. [Online]. Available: https://www.hindawi.com/journals/sp/2021/4924708/.
[40] C. Wei, Z.-H. Hu, and Y.-G. Wang, "Exact algorithms for energy-efficient virtual machine placement in data centers," Future Generation Computer Systems, vol. 106, pp. 77-91, 2020. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0167739X19319594.
[41] M. L. Puterman, Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons, 2014.
[42] Q. Zheng, R. Li, X. Li, and J. Wu, "A multi-objective biogeography-based optimization for virtual machine placement," in 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, 2015: IEEE, pp. 687-696.
[43] M. Duggan, K. Flesk, J. Duggan, E. Howley, and E. Barrett, "A reinforcement learning approach for dynamic selection of virtual machines in cloud data centres," in 2016 sixth international conference on innovative computing technology (INTECH), 2016: IEEE, pp. 92-97. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/7845053.
[44] J. Zeng, D. Ding, K. Kang, H. Xie, and Q. Yin, "Adaptive DRL-based virtual machine consolidation in energy-efficient cloud data center," IEEE Transactions on Parallel and Distributed Systems, vol. 33, no. 11, pp. 2991-3002, 2022. [Online]. Available: https://ieeexplore.ieee.org/document/9698981.
[45] A. Beloglazov, J. Abawajy, and R. Buyya, "Energy-aware resource allocation heuristics for efficient management of data centers for cloud computing," Future generation computer systems, vol. 28, no. 5, pp. 755-768, 2012.
[46] K.-D. Lange, "Identifying shades of green: The SPECpower benchmarks," Computer, vol. 42, no. 03, pp. 95-97, 2009.
[47] R. N. Calheiros, R. Ranjan, A. Beloglazov, C. A. De Rose, and R. Buyya, "CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms," Software: Practice and experience, vol. 41, no. 1, pp. 23-50, 2011. [Online]. Available: https://onlinelibrary.wiley.com/doi/full/10.1002/spe.995.
[48] Z. Li, X. Yu, L. Yu, S. Guo, and V. Chang, "Energy-efficient and quality-aware VM consolidation method," Future Generation Computer Systems, vol. 102, pp. 789-809, 2020. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0167739X18324713.
[49] K. Park and V. S. Pai, "CoMon: a mostly-scalable monitoring system for PlanetLab," ACM SIGOPS Operating Systems Review, vol. 40, no. 1, pp. 65-74, 2006. [Online]. Available: https://dl.acm.org/doi/abs/10.1145/1113361.1113374.
電子全文 電子全文(網際網路公開日期:20280903)
連結至畢業學校之論文網頁點我開啟連結
註: 此連結為研究生畢業學校所提供,不一定有電子全文可供下載,若連結有誤,請點選上方之〝勘誤回報〞功能,我們會盡快修正,謝謝!
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top