跳到主要內容

臺灣博碩士論文加值系統

(44.192.22.242) 您好!臺灣時間:2021/08/05 12:50
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果 :::

詳目顯示

我願授權國圖
: 
twitterline
研究生:紀瑤君
研究生(外文):Yao-Chun Chi
論文名稱:運用動態預測調整機制於資料格網中之平行檔案傳輸
論文名稱(外文):Redundant Parallel File Transfer with Anticipative Recursively-Adjusting Mechanism in Data Grids
指導教授:楊朝棟楊朝棟引用關係
指導教授(外文):Chao-Tung Yang
學位類別:碩士
校院名稱:東海大學
系所名稱:資訊工程與科學系碩士在職專班
學門:工程學門
學類:電資工程學類
論文種類:學術論文
論文出版年:2007
畢業學年度:95
語文別:英文
論文頁數:54
中文關鍵詞:資料網格協同配置動態協同配置平行傳輸
外文關鍵詞:Data GridCo-allocationDynamic Co-allocationPartial Transfer
相關次數:
  • 被引用被引用:0
  • 點閱點閱:156
  • 評分評分:
  • 下載下載:0
  • 收藏至我的研究室書目清單書目收藏:0
資料網格(Data Grid)使得分散在不同區域上的計算及儲存資源(同質或異質),可以達到分享、選擇、以及相互的溝通的應用。尤其是需要分析大量且密集資料的科學實驗,諸如高能物理、生物資訊的運用、以及氣象的模擬等,透過應用資料網格的都獲得良好的問題解決方式。
在資料網格(Data Grid)環境中,資料集被複製為複本且分送到多重的站台。由於資料集的檔案通常很大,如何有效率的存取及傳輸成為重大的課題。因此先前有學者發展出協同配置的架構(Co-allocation Architecture),使得同時從多重站台平行下載資料變成可能,且發展出數種協同配置的策略被使用來解決傳輸時本地端與伺服端網路傳輸率會變動的問題。例如將欲傳輸的檔案切割成數個均等的檔案大小,或是將檔案切割成相同大小置於工作佇列,透過連線品質較佳者傳送佇列中末完成傳輸的檔案區塊,來解決網路變動的問題。
無論各個下載連線的效率為何,前述中各傳輸伺服器每次所被分配到的每一個檔案區塊大小是一樣的,將使傳送最後一個檔案區塊時,發生速度快的伺服器通常需要耗用較多的等侯時間在等侯最慢的伺服器完成最後一個檔案區塊的傳送,或是因為不同伺服器傳送相同的檔案區塊,造成網路資源的浪費,因此,如何減少各伺服器間完成傳輸時間的差異,且避免傳送相同檔案區塊所形成的網路資源浪費,將成為重要的工作。
在這個研究中,我們提出了一個動態預測調整的方式稱為預測性遞迴調整的協同配置(Anticipative Recursively-Adjusting Mechanism),來改善資料網格(Data Grid)中資料傳輸的效能。
我們的方法有效地減少了快速伺服器與慢速伺服器間資料傳輸完成之閒置時間,也減少了整體資料傳輸之完成時間,且避免發生重覆傳送相同檔案區塊的情況。
More and more applications emphasize analysis of a huge amount of data and depend on the transmission of them. Data Grids enable the sharing, selection, and connection of a wide variety of geographically distributed computational and storage resources for content that the large-scale data-intensive application needs, such as high-energy physics, bioinformatics, and virtual astrophysical observatories. Data grids consist of scattered computing and storage resources located in different countries/regions yet accessible to users. The co-allocation architecture was developed to enable the parallel download of datasets/servers from selected replica servers, and the bandwidth performance is the main factor that affects the internet transfer between the client and the server. In previous works, exist a drawback of idle time or degrade the network performance for transferring the same block. Hence, it is important to reduce the difference of finished time between each selected replica server, and avert the bandwidth traffic congestion from transferring the same block in the link among the servers and clients, and manage changeful network performance during the term of transferring as well. In this thesis, we proposed Anticipative Recursively-Adjusting Mechanism scheme to adjust the workload of each selected replica server, which handles unwarned variant network performances of the selected replica servers. The algorithm is based on finished rate of the previous assigned transfer size to anticipate the bandwidth status on the next section to adjust the workload, and further, to reduce file transfer time in a grid environment. Our approach is useful in unstable gird environments and it not only reduces the wasted idle time from waiting the slowest server but also decreases the completion time for file transfers.
摘要 ii
Abstract iii
Acknowledgements iv
Contents v
List of Tables vii
List of Figures viii
Chapter 1 Introduction 1
1.1 Motivation 1
1.2 Contribution 3
1.3 Thesis Organization 4
Chapter 2 Background 5
2.1 Data Grid 5
2.2 Replica Management 9
2.3 Replica Catalog 10
2.4 Replica Selection 12
2.5 Globus Toolkit and GridFTP 13
2.6 Peer-to-Peer System 15
Chapter 3 Co-Allocation Architecture and Related Work 16
3.1 Multi-Source Data Transfer 19
3.2 Tuned Conservative scheduling technique 19
3.3 Co-Allocation Architecture 19
3.3.1 Brute-Force Co-Allocation 20
3.3.2 History-based Co-Allocation 20
3.3.3 Conservative Load Balancing 21
3.3.4 Aggressive Load Balancing 21
3.4 Dynamic Co-allocation Scheme with Duplicate Assignments 22
Chapter 4 System Portal 25
4.1 Components 25
4.2 Transaction Flow 26
4.3 Improvements 28
Chapter 5 Dynamic Co-allocation Sheme 29
5.1 Assumptions 29
5.2 Anticipative Recursively-Adjusting Mechanism 30
5.3 Parameters and Evaluation Model 34
5.4 Simple Example 38
5.5 Algorithm 40
Chapter 6 Experimental Results and Analysis 41
6.1 Simulations 41
6.2 Results and Analysis 42
Chapter 7 Conclusions and Future Works 49
References 51
[1]B. Allcock, J. Bester, J. Bresnahan, A. Chervenak, I.Foster, C. Kesselman, S. Meder, V. Nefedova, D.Quesnel, and S. Tuecke, “Data Management and Transfer in High-Performance Computational Grid Environments,” Parallel Computing, 28(5):pp.749-771, May 2002.
[2]A. Chervenak, E. Deelman, I. Foster, L. Guy, W. Hoschek, A. Iamnitchi, C. Kesselman, P. Kunszt, and M. Ripeanu, “Giggle: A Framework for Constructing Scalable Replica Location Services,” Proceedings of the 2002 ACM/IEEE conference on Supercomputing, pp.1-17, November 2002.
[3]A. Chervenak, I. Foster, C. Kesselman, C. Salisbury, and S. Tuecke, “The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific Datasets,” Journal of Network and Computer Applications, 23: pp.187-200, 2001 (based on conference publication from Proceedings of NetStore Conference 1999).
[4]K. Czajkowski, S. Fitzgerald, I. Foster, and C. Kesselman, “Grid Information Services for Distributed Resource Sharing,” Proceedings of the Tenth IEEE International Symposium on High-Performance Distributed Computing (HPDC-10’01), 181-194, August 2001.
[5]K. Czajkowski, I. Foster, and C. Kesselman. “Resource Co-Allocation in Computational Grids,” Proceedings of the Eighth IEEE International Symposium on High Performance Distributed Computing (HPDC-8’99), August 1999.
[6]F. Donno, L. Gaido, A. Ghiselli, F. Prelz, M. Sgaravatto, “DataGrid Prototype 1,” TERENA Networking Conference, http://www.terena.nl/conferences/tnc2002/Papers/p5a2-ghiselli.pdf, June 2002
[7]I. Foster, C. Kesselman, S. Tuecke. “The Anatomy of the Grid: Enabling Scalable Virtual Organizations.” Int. J. of Supercomputer Applications and High Performance Computing, 15(3), pp. 200-222, 2001.
[8]I. Foster, C. Kesselman, “Globus: A Metacomputing Infrastructure Toolkit,” Intl J. Supercomputer Applications, 11(2), pp. 115-128, 1997.
[9]Global Grid Forum, http://www.ggf.org/
[10]C.H. Chen, C.T. Yang, and C.L. Lai, “Towards an Efficient Replica Selection for Data Grid,” Proceedings of the First Workshop on Grid Technologies and Applications (WoGTA'04), pp. 89-94, December 2004.
[11]W. Hoschek, J. Jaen-Martinez, A. Samar, H. Stockinger, and K. Stockinger, “Data Management in an International Data Grid Project,” Proceedings of the First IEEE/ACM International Workshop on Grid Computing - Grid 2000, Bangalore, India, December 2000.
[12]IBM Red Books, “Introduction to Grid Computing with Globus,” IBM Press, www.redbooks.ibm.com/redbooks/pdfs/sg246895.pdf
[13]C.M. Wang, C.C. Hsu, H.M. Chen, and J.J. Wu, “Efficient Multi-Source Data Transfer in Data Grids,” Proceedings of the Sixth IEEE International Symposium on Cluster Computing and the Grid (CCGRID'06), pp 421-424, 16-19 May 2006.
[14]H. Stockinger, A. Samar, B. Allcock, I. Foster, K. Holtman, and B. Tierney, “File and Object Replication in Data Grids,” Journal of Cluster Computing, 5(3):305-314, 2002.
[15]R. S. Bhuvaneswaran, Y. Katayama, and N. Takahashi, “Dynamic Co-allocation Scheme for Parallel Data Transfer in Grid Environment,” Proceedings of First International Conference on Semantics, Knowledge, and Grid (SKG 2005), pp. 17, 2005.
[16]R. S. Bhuvaneswaran, Y. Katayama, and N. Takahashi, “A Framework for an Integrated Co-allocator for Data Grid in Multi-Sender Environment,” IEICE TRANSACTIONS on Communications, Vol.E90-B No.4 pp.742-749, 2007.
[17]The Globus Alliance, http://www.globus.org/
[18]C.T. Yang, I.H. Yang, and C.H. Chen, “Improve Dynamic Adjustment Mechanism in Co-Allocation Data Grid Environments,” Proceedings of the 11th Workshop on Compiler Techniques for High-Performance Computing (CTHPC-11’ 05), pp. 189-194, 17-18 March 2005.
[19]C.T. Yang, C.H. Chen, K.C. Li, and C.H. Hsu, “Performance Analysis of Applying Replica Selection Technology for Data Grid Environments,” PaCT 2005, Lecture Notes in Computer Science, vol. 3603, pp. 278-287, Springer-Verlag, September 2005.
[20]C. T. Yang, I.H. Yang, K.C. Li, and C.H. Hsu “A Recursive-Adjustment Co-Allocation Scheme in Data Grid Environments,” ICA3PP 2005 Algorithm and Architecture for Parallel Processing, Lecture Notes in Computer Science, vol. 3719, pp. 40-49, Springer-Verlag, October 2005.
[21]C. T. Yang, I.H. Yang, K.C. Li, and S.Y. Wang, “Improvements on Dynamic Adjustment Mechanism in Co-Allocation Data Grid Environments,” accepted and to appear in The Journal of Supercomputing, December 2006.
[22]C. T. Yang, S.Y. Wang, C.H. Lin, M.H Lee, and T.Y. Wu, “Cyber-Transformer: A Toolkit for Files Transfer with Replica Management in Data Grid Environments,” Proceedings of the Second Workshop on Grid Technologies and Applications (WoGTA’05), pp. 73-80, December 2005.
[23]C. T. Yang, S.Y. Wang, and C.P. Fu, “A Dynamic Adjustment Mechanism for Data Transfer in Data Grids,” accepted and to appear in the Proceeding of Network and Parallel Computing (NPC2006), October 2006.
[24]S. Vazhkudai, “Enabling the Co-Allocation of Grid Data Transfers,” Proceedings of Fourth International Workshop on Grid Computing, pp. 44 – 51, 17 November 2003.
[25]S. Vazhkudai, S. Tuecke, and I. Foster, “Replica Selection in the Globus Data Grid,” Proceedings of the 1st International Symposium on Cluster Computing and the Grid (CCGRID 2001), pp. 106-113, May 2001.
[26]S. Vazhkudai and J. Schopf, “Using Regression Techniques to Predict Large Data Transfers,” International Journal of High Performance Computing Applications (IJHPCA), 17:249-268, August 2003
[27]S. Vazhkudai and J. Schopf, “Predicting Sporadic Grid Data Transfers,” Proceedings of 11th IEEE International Symposium on High Performance Distributed Computing (HPDC-11 ‘02), pp. 188-196, July 2002.
[28]S. Vazhkudai, J. Schopf, and I. Foster, “Predicting the Performance of Wide Area Data Transfers,” Proceedings of the 16th International Parallel and Distributed Processing Symposium (IPDPS 2002), pp.34-43, April 2002.
[29]S. Venugopal, R. Buyya, and K. Ramamohanarao, “A Taxonomy of Data Grids for Distributed Data Sharing, Management, and Processing,” Proceedings of the ACM Computing Surveys, Vol.38 Article 3, March 2006.
[30]L. Yang, J. M. Schopf, and I. Foster, “Improving Parallel Data Transfer Times Using Predicted Variances in Shared Networks,” Proceedings of the fifth IEEE International Symposium on Cluster Computing and the Grid, (CCGrid’05’), pp 734- 742, 9-12 May 2005.
[31]X. Zhang, J. Freschl, and J. Schopf, “A Performance Study of Monitoring and Information Services for Distributed Systems”, Proceedings of 12th IEEE International Symposium on High Performance Distributed Computing (HPDC-12 ‘03), pp. 270-282, August 2003.
連結至畢業學校之論文網頁點我開啟連結
註: 此連結為研究生畢業學校所提供,不一定有電子全文可供下載,若連結有誤,請點選上方之〝勘誤回報〞功能,我們會盡快修正,謝謝!
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top
1. 施教裕(1997),<民間福利機構團體因應民營化之現況、問題及策略>,《社區發展季刊》,第80 期,頁43。
2. 陳恆鈞(1997), <由「公私部門合夥」觀念談民眾參與政府建設>,《人力發展月刊》,第47 期,頁32-41。
3. 曾思瑜(2002),<北歐高齡者住宅、設施政策與體系建構之研究─以瑞典和丹麥為例>。《建築學報》,第41期,2002年9月,頁28-29.
4. 莫永榮(2004)。政府服務業務委託外包的理論與實務:臺灣經驗。《行政暨政策學報》,38期,頁75-103。
5. 林萬億(2003),論我國的社會住宅政策與社會照顧的結合,行政院研究發展考核委員會《國家政策季刊》,第二卷,第四期,2003.12,頁77-80
6. 郭登聰(2005),推動社會福利民營化相關法制的析論。《社區發展季刊》,108期。
7. 蘇昭如(1993),<政府委託民間辦理社會福利服務之條件與方式>,《社區發展季刊》,第63 期,頁68。
8. 陳菊(1997),<台北市政府社會局公設民營現況,面臨問題與因應之道>,《社區發展季刊》,第80 期,頁19。
9. 龐建國、劉錦常(1990),<福利國家的爭議>,《中山社會科學季刊,第5 卷第3 期,頁2。
10. 劉坤億(2003b),<全球治理的理想與現實>,《中國行政評論》,第13 卷,第1 期,頁 29~56 。
11. 劉坤億(2003a),<地方治理與地方政府角色職能的轉變>,國立空中大學:《空大行政學報》,第13 期,頁 233~268 。
12. 陳維萍(1999),<從「臺北市陽明老人公寓」的成立淺談銀髮族的居住環境>,《福利社會》,第75期,頁28-31。
13. 陳金貴(1992),<公民參與的研究>,《公共行政學報》,第24期,頁95-128。
14. 趙碧華(1996),國家社會福利民營化之社政工作初探,《社區發展季刊》,75期,48-56。
15. 王仕圖,1999,〈「公設民營」的迷思:非營利組織理論觀點的反省〉,《社區發展季刊》,95期,頁156-165。