跳到主要內容

臺灣博碩士論文加值系統

(44.200.169.3) 您好!臺灣時間:2022/12/01 02:34
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果 :::

詳目顯示

: 
twitterline
研究生:饒育睿
研究生(外文):Rao, Yu-Ruei
論文名稱:雲端計算中為MapReduce設計之精明的中間資料傳輸機制
論文名稱(外文):Smart Intermediate Data transfer for MapReduce on Cloud Computing
指導教授:黃祖基
指導教授(外文):Huang, Tzu-Chi
口試委員:蔡明峰黃祖基朱國志
口試委員(外文):Tsai, Ming-FongHuang, Tzu-ChiChu, Kuo-Chih
口試日期:2014-06-24
學位類別:碩士
校院名稱:龍華科技大學
系所名稱:電子工程系碩士班
學門:工程學門
學類:電資工程學類
論文種類:學術論文
論文出版年:2014
畢業學年度:102
語文別:中文
論文頁數:65
中文關鍵詞:MapReduce雲端中間資料SIDT壓縮Bzip2Gzip
外文關鍵詞:MapReduceCloudIntermediate dataSIDTcompressionBzip2Gzip
相關次數:
  • 被引用被引用:0
  • 點閱點閱:184
  • 評分評分:
  • 下載下載:14
  • 收藏至我的研究室書目清單書目收藏:0
MapReduce是Google所提出的一套程式架構,用於在雲端中做大規模數據的處理。MapReduce經常需要在節點間傳送大量的中間資料,這使得應用程式的處理需要花費更多的時間。MapReduce可以被加強藉由本論文所提出的Smart Intermediate Data Transfer(SIDT)方法來對中間資料進行整理,進而改善應用程式執行所花費的時間。本論文中證明在與Bzip2和Gzip等壓縮方法比較之下,雖然SIDT對中間資料的壓縮效果不是最好的,但是MapReduce能藉著SIDT的整理可以減少系統執行的時間在大部分情況下能達到比其他編碼更好的效能。
MapReduce is a programming model proposed by Google to process large datasets in clusters. However, MapReduce often needs to transfer much intermediate data among nodes, which is harmful to performances of an application. MapReduce can be enhanced by using the proposed Smart Intermediate Data Transfer (SIDT) in the runtime system to smartly arrange intermediate data. Although SIDT does not reduce intermediate data to the minimal size in comparison with other intermediate data arrangement procedures such as Huffman coding, Bzip2, and Gzip, MapReduce is proved to get a better performance from SIDT than from others in the experiments of this paper.
摘要 i
ABSTRACT ii
誌謝 iii
目錄 iv
表目錄 vi
圖目錄 vii
第一章 緒論 1
1.1 介紹 1
1.2 論文流程與架構 2
第二章 背景知識 3
2.1 MapReduce 3
2.2 常用壓縮編碼方法 4
2.2.1 Gzip壓縮 4
2.2.2 Bzip2壓縮 6
第三章 SMART INTERMEDIATE DATA TRANSFER 8
3.1 概論 8
3.2 SIDT Encoder 9
3.3 SIDT Decoder 11
第四章 系統實作 12
4.1 MapReduce系統 12
4.2 SIDT on MapReduce 15
第五章 實驗 17
5.1 實驗設置 17
5.2 中間資料的壓縮與解壓縮 18
5.3 系統分析 21
5.4 Word Count效能分析 22
5.5 Quick Sort效能分析 29
5.6 Bubble Sort效能分析 36
5.7 GREP效能分析 43
5.8 Inverted Index效能分析 49
第六章 相關工作 56
第七章 結論 62
參考文獻 63

[1] J. Dean, S. Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters", Communications of the ACM, Volume 51, Issue 1, 2008, pp. 107-113
[2] S. P. Ahuja, A. C. Rolli, "Survey of the State-of-the-Art of Cloud Computing", International Journal of Cloud Applications and Computing, Volume 1 Issue 4, 2011, pp.34-43
[3] B. P. Rimal, E. Choi, I. Lumb, "A Taxonomy and Survey of Cloud Computing Systems", in Proceedings of Fifth International Joint Conference on INC, IMS and IDC, 2009, pp. 44-51
[4] Patel, R.A., Yao Zhang, Mak,J., Davidson,A., Owens,J.D., "Parallel Lossless Data Compression on the GPU", Innovative Parallel Computing (InPar), 2012, pp.1-9
[5] M. F. Nowlan, B. Ford, R. Gummadi, "Non-linear compression: Gzip Me Not!", in Proceedings of the 4th USENIX conference on Hot Topics in Storage and File Systems, 2012, pp. 11-11
[6] M. Sharma, "Compression Using Huffman Coding", IJCSNS International Journal of Computer Science and Network Security, Vol. 10, No. 5, 2010, pp. 133-141
[7] Giovanni Manzini, "An Analysis of the Burrows-Wheeler Transform" , Journal of the ACM (JACM), Volume 48 Issue 3, May 2001, pp.407-430
[8] C. A. R. Hoare, "Quicksort", the Computer Journal, Vol. 5, Issue 1, 1962, pp.10-16
[9] Owen Astrachan, "Bubble Sort: An Archaeological Algorithmic Analysis",in Proceedings of the 34th SIGCSE technical symposium on Computer science education, pp.1-5
[10] Jacopo Urbani, Jason Maassen, Henri Bal, "Massive Semantic Web data compression with MapReduce", in Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing, pp.795-802
[11] M. Liroz-Gistau, R. Akbarinia, D. Agrawal, E. Pacitti, and P. Valduriez, "Data Partitioning for Minimizing Transferred Data in MapReduce", in Proc. Globe, 2013, pp.1-12
[12] Po-Cheng Chen, Yen-Liang Su1, Jyh-Biau Chang2, Ce-Kuen Shieh, "Variable-Sized Map and Locality-Aware Reduce on Public-Resource Grids", Future Generation Computer Systems,Volume 27 Issue 6, June, 2011,pp.843-849
[13] M. Hammoud,Majd F. Sakr, "Locality-Aware Reduce Task Scheduling for MapReduce", in Proceedings of the 2011 IEEE Third International Conference on Cloud Computing Technology and Science, pp.570-576
[14] Shadi Ibrahim, Hai Jin, Lu Lu, Song Wu, Bingsheng He, Li Qi, "LEEN: Locality/Fairness- Aware Key Partitioning for MapReduce in the Cloud", in Proceedings of the 2010 IEEE Second International Conference on Cloud Computing Technology and Science, pp.17-24
[15] Hong-Yi Chang, Nen-Fu Huang, Kwei-Bor Chen, Yih-Jou Tzang, Yuan-Wei Lin, "Saving Streaming Bandwidth via Wireless Sharing for a Tree-based Live Streaming System on Public-Shared Network", in Proceedings of the 2012 8th International Conference on Computing Technology and Information Management, pp.88-94
[16] Hyuck Han, Young Choon Lee, Seungmi Choi, Heon Y. Yeom, Albert Y. Zomaya, "Cloud-Aware Processing of MapReduce-Based OLAP Applications", in Proceedings of the Eleventh Australasian Symposium on Parallel and Distributed Computing - Volume 140, pp.31-38
[17] Athicha Muthitacharoen, Benjie Chen, David Mazières, "A Low-bandwidth Network File System", in Proceedings of the eighteenth ACM symposium on Operating systems principles, pp.174-187
[18] Mohammad Hammoud, M. Suhail Rehman, Majd F. Sakr, " Center-of-Gravity Reduce Task Scheduling to Lower MapReduce Network Traffic", in Proceedings of the 2012 IEEE Fifth International Conference on Cloud Computing pp.49-58
[19] D. S. Hirschberg, D. A. Lelewer, "Efficient Decoding of Prefix Codes", Communications of the ACM, Vol. 33, Issue. 4, 1990, pp. 449-459
[20] S. Trent, M. Tatsubori, T. Suzumura, A. Tozawa, T. Onodera, "Performance comparison of PHP and JSP as server-side scripting languages", in Proceedings of the 9th ACM/IFIP/USENIX International Conference on Middleware, 2008, pp. 164-182

QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top