跳到主要內容

臺灣博碩士論文加值系統

(34.204.198.73) 您好!臺灣時間:2024/07/21 16:52
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果 :::

詳目顯示

: 
twitterline
研究生:楊貴安
研究生(外文):Kuei-an Yang
論文名稱:Hadoop雲端平台在工程應用之探討研究
論文名稱(外文):Study on the Hadoop Cloud Computing Platform for Engineering Applications
指導教授:周建成周建成引用關係
指導教授(外文):Chien-Cheng Chou
學位類別:碩士
校院名稱:國立中央大學
系所名稱:土木工程研究所
學門:工程學門
學類:土木工程學類
論文種類:學術論文
論文出版年:2012
畢業學年度:100
語文別:中文
論文頁數:185
中文關鍵詞:雲端運算Hadoop分散式系統虛擬叢集
外文關鍵詞:Distributed SystemsHadoopCloud Computing
相關次數:
  • 被引用被引用:12
  • 點閱點閱:1060
  • 評分評分:
  • 下載下載:325
  • 收藏至我的研究室書目清單書目收藏:0
雲端運算是一種新的網路概念,是藉由網路串聯不同電腦之間的相互合作,藉由網路的概念來產生相應的技術,本質來自於分散式運算與網格運算。分散式運算是將大型工作分成很多小型工作,再分別分配給眾多的電腦運算,最後再匯集所有的結果來完成單機無法完成的工作。網格運算則是分散式運算的一種延伸,主要特色是將不同平台、不同等級、不同架構的電腦藉由分散式運算來做整合,所以雲端運算與網格運算都是一種分散式運算的延伸。網格運算是強調整合眾多不同的平台,而雲端運算是強調在本機有限的資源利用網路來取得運算資源。因此,建置分散式運算的雲端平台研究有其必要性。本研究建置雲端分散式檔案系統Hadoop,Hadoop Distributed File System(HDFS),使用四台實體電腦來架設四台虛擬叢集環境與八台虛擬叢集環境。四台虛擬叢集電腦架設方式是在每台實體電腦各虛擬一台電腦出來,共四台虛擬叢集環境,而八台虛擬叢集電腦架設方式是在每台實體電腦各虛擬兩台電腦出來,共八台虛擬叢集環境。經本研究結果實現,一台實體電腦可以虛擬兩台以上電腦,符合雲端虛擬化上百台或上千台的叢集環境。其次,雲端分散式系統是來處理大量的運算,本研究藉由矩陣大量的運算來測試Hadoop分散式檔案系統。而矩陣運算在工程應用是常見且重要的,不過目前矩陣運算都是以MPI(Message Passing Interface)來實現,並無在雲端平台上來實現,因此本研究藉由雲端平台來實施矩陣運算。
Cloud computing is a new concept of networking. Cloud computing is a co-operation of the network which allows several computers to work together. The nature of cloud computing are from distributed computing and grid computing. Distributed computing is a large work divided into several small parts. Then it will be handed over to several computers to do the computing process. Finally, to bring all the results together to complete the stand-alone computing could not be done. Grid computing is an extension of the distributed computing. The main features of different platforms, different levels of the different computer structure integrated by distributed computing. Cloud computing and grid computing are extensions of a distributed computing. Grid computing is the emphasis on the integration of many different platforms. While cloud computing is the emphasis on limited resources in the machine, which use internet to obtain the computing resources. Therefore, build distributed computing cloud platform is necessary. The objective of this study is to build a cloud distributed systems Hadoop, Hadoop Distributed File System (HDFS).We use four physical computers to host four virtual cluster environments with eight virtual cluster environments. The four cluster computers set up in each physical computer, with one virtual computer exist for each physical computer, so there are four virtual cluster environments. Then each of the four physical computers is built two virtual computers inside, so there are eight virtual cluster environments. It proves that a physical computer can has two or more virtual computers. Comply with the cloud virtualization of hundreds or thousands of cluster environments. Moreover, cloud distributed systems can deal with a lot of computing that we used in computing matrix. Matrix computing is important of engineering application. But usually implement MPI(Message Passing Interface). So this paper implements matrix computing of cloud platform.
摘要 i
Abstract ii
誌謝 iv
目錄 v
圖目錄 viii
表目錄 xiv
第一章、緒論 1
1.1 研究背景與動機 1
1.2 研究目的 3
1.3 研究範圍 3
1.4 研究流程 4
1.5 論文架構 5
第二章、文獻回顧 7
2.1 雲端運算 7
2.1.1 Microsoft 13
2.1.2 IBM 13
2.1.3 Oracle 14
2.1.4 Amazon 14
2.1.5 Google 14
2.1.6 雲端平台選擇 17
2.2 Hadoop簡介 17
2.2.1 HDFS 17
2.2.2 MapReduce 25
2.2.3 HBase 30
2.3 矩陣運算平行化回顧 37
2.3.1 矩陣加法 37
2.3.2 矩陣乘法 38
2.3.3 反矩陣 41
第三章、系統架構 42
3.1 Hadoop環境介紹 42
3.1.1 整體環境說明 45
3.1.2 四台節點的叢集環境 46
3.1.3 八台節點的叢集環境 51
3.2 Hadoop-0.20.2範例測試 55
3.2.1 執行成功範例 55
3.2.2 執行異常範例 69
3.3 HBase-0.20.6範例測試 70
3.3.1 執行成功範例 70
3.3.2 執行異常範例 78
3.4 測試總結 78
3.4.1 適合於Hadoop實作之探討 78
3.4.2 不適合於Hadoop實作之探討 79
3.4.3 適合於HBase實作之探討 79
3.4.4 不適合於HBase實作之探討 79
第四章、矩陣運算與評估 80
4.1 矩陣運算 80
4.1.1 矩陣相加減運算 80
4.1.2 矩陣相乘運算 85
4.1.3 反矩陣運算 91
4.2 矩陣運算評估 97
4.2.1 矩陣相加減評估 97
4.2.2 矩陣相乘評估 104
4.2.3 反矩陣評估 112
第五章、結論與建議 115
5.1 結論 115
5.2 建議 116
5.3 貢獻 116
參考文獻 118
附錄一:VirtualBox-4.0.8安裝與設定 120
附錄二:Ubuntu-9.10安裝 130
附錄三:Ubuntu-12.04安裝 138
附錄四:Hadoop-0.20.2與HBase-0.20.6設定 145
附錄五:Hadoop-1.0.3與HBase-0.94.0設定 172
Agarwal, C., R., Bale, M., S., Gustavson, G., F., Joshi, M., and Palkar, P. (1995) “A threedimensional approach to parallel matrix multiplication.” IBM Journal of Research and Development, 39(5), 575-582.
Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., and Zaharia, M. (2010) “A View of Cloud Computing.” Communications of the ACM, 53(4), 50-58.
Apache (2012). HDFS Architecture Guide. Retrieved February 17, 2012, from http://hadoop.apache.org/common/docs/current/hdfs_design.html
Chang, F., Dean, J., Ghemawat, S., Hsieh, C., W., Wallach, A., D., Burrows, M., Chandra, T., Fikes, A., and Gruber, E., R. (2008) “Bigtable: A Distributed Storage System for Structured Data.” ACM Transactions on Computer Systems, 26(2).
Cusumano, M. (2010) “Cloud Computing and SaaS as New Computing Platforms.” Communications of the ACM, 53(4), 27-29.
Dean, J., and Ghemawat, S. (2008) “MapReduce: Simplied Data Processing on Large Clusters.” Communications of the ACM, 51(1), 107-113.
Ghemawat, S., Gobioff, H., and Leung, T., S. (2003) “The Google File System.” Proceedings of the nineteenth ACM symposium on Operating systems principles, 29-43.
Krumins, P. (2009). MIT''s Introduction to Algorithms, Lectures 20 and 21: Parallel, Retrieved February 18, 2012, April 27, 2012 from http://www.catonmat.net/blog/mit-introduction-to-algorithms-part-thirteen/
Lars, G. (2011) “HBase: The Definitive Guide.” Oreilly and Associates Inc.
Pease, C., M. (1967) “Matrix Inversion Using Parallel Processing.” Journal of the Association for Computing Machinery, 14(4), 757-764.
Ricky, H. (2008). Hadoop Map/Reduce Implementation. Retrieved February 2, 2012, from http://horicky.blogspot.com/2008/11/ hadoop-mapreduce-implementation.html
Shvachko, K., Kuang, H., Radia, S., and Chansler, R. (2010) “The Hadoop Distributed File System.” Mass Storage Systems and Technologies, 1-10.
Seo, S., Yoon, J. E., Kim, J., Jin, S., Kim, S. J., Maeng, S. (2010) “HAMA: An Efficient Matrix Computation with the MapReduce Framework. ” International Conference on Cloud Computing Technology and Science, 721-726.
Tom, W. and Doug, C. (2010) “Hadoop: The Definitive Guide.” Oreilly and Associates Inc.
Wikipedia (2011). Apache Hadoop. Retrieved February 2, 2012, from http://en.wikipedia.org/wiki/Apache_Hadoop
Wikipedia (2010). Windows Azure. Retrieved February 2, 2012, from http://zh.wikipedia.org/wiki/Windows_Azure
Wikipedia (2012). MapReduce. Retrieved February 18, 2012, from http://en.wikipedia.org/wiki/MapReduce
Zhang, C., and Sterck, D. H. (2010) “Supporting Multi-row Distributed Transactions with Global Snapshot Isolation Using Bare-bones HBase” In Proceedings of the 11th ACM/IEEE International Conference on Grid Computing (Grid), 177-184.
王鵬(2010)。雲端運算的關鍵技術與應用實例。佳魁資訊。
國際商業機器股份有限公司(2010)。精選雲端解決方案。西元2012年2月3日,取自:http://www-03.ibm.com/systems/tw/promotion/2010_q1_hot/cloud.html
黃淑玲(2010)。雲端辦公室Google Office簡介。國立台灣大學計算機及資訊網路中心電子報第15期。民國101年2月2日,取自:http://www.cc.ntu.edu.tw/chinese/epaper/0015/20101220_1506.htm
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top
無相關期刊