臺灣博碩士論文加值系統

在現今微型企業的興起中，小規模的企業也建置相當多的資訊系統，產生出大量的運算，以及大量的儲存資料。在人力資源與預算有限的前提下，企業內部的資訊單位必須有能力處理龐大的數據資料，並妥善做好資料備份。為了解決企業龐大的儲存需求，雲端儲存服務（Cloud Storage Services）成為重要的解決方案之一，先將儲存資源虛擬化後，再建置成儲存池（storage pool），並提供線上資料存取與備份之網路服務。本研究主要探討三種不同的方式建置檔案系統，從中分析比較出具有低成本且高效能的最佳檔案系統配置，讓資源有限的小企業，也可以滿足高效能的儲存需求。依實驗結果顯示，在低成本的硬體及網路環境下，GlusterFS、Lustre、Hadoop三種不同的系統格式再不同的檔案大小下個有不同的優缺點。

The micro-enterprises and small-scale enterprises are raised in recent decade. They build a considerable number of information systems, resulting in a large number of operations, as well as a large amount of data stored. In the human resources and budget conscious premise, internal IT unit must have the ability to handle large data, and make proper data backup. To solve the huge demand for storage and cost, cloud storage services have become one of the important solutions. These solutions include first storage resource virtualization, the construction of storage pools, the supplies of online data access and the backup of web services. In this study, we compare the time cost and the efficiency with three common file systems, namely GlusterFS, Lustre, and Hadoop. Experimental results show as follows. First, the Hadoop has the highest performance when the storage is larger than 200MB file in these three systems. Second, the Lustre has the highest performance when the storage is less than 200MB and more than 4MB file. Final, the GlusterFS has the highest performance when the storage is less than 4MB file.

摘　要 i
ABSTRACT ii
誌 謝 iii
目 錄 iv
表目錄 vi
圖目錄 vii
第一章 緒論 1
1.1 研究背景 1
1.2 研究動機與目的 1
1.2.1 研究動機 1
1.2.2 研究目的 3
1.3 論文架構 3
第二章 背景說明 5
2.1 雲端運算 5
2.2 雲端儲存服務 10
2.3 元資料 11
2.4 使用者空間檔案系統 13
2.5日誌檔案系統 14
2.6 EXT4 15
2.7 XFS 17
2.8 IOZone 18
第三章 系統介紹 20
3.1 系統架構 20
3.2 系統資訊 21
3.3 系統環境介紹及配置 23
3.3.1Gluster 23
3.3.2 Lustre 33
3.3.3 Hadoop 37
第四章 實驗分析 42
4.1 測試項目 42
4.2 分析比較 43
4.2.1 GlusterFS、Lustre和Hadoop在單機環境使用EXT4格式的比較 43
4.2.2 GlusterFS、Lustre和Hadoop在多機環境使用EXT4格式的比較 55
4.2.3 GlusterFS、Lustre和Hadoop在多機環境比較EXT4格式以及XFS格式 66
第五章 結論與未來展望 74
5.1 結論 74
5.2 未來展望 74
參考文獻 76
附錄 80
A. 儲存伺服器硬體規格表 80
B. 測試客戶端硬體規格表 80
C. 網路交換器硬體規格比較表 81

[1]Cloud storage, http://en.wikipedia.org/wiki/Cloud_storage, 2014
[2]Amazon Glacier, http://aws.amazon.com/glacier/, 2014
[3]微企業當道 高市數創3年育成25家，http://www.epochtimes.com/b5/15/12/27/n4605369.htm, 2015
[4]推特3200萬帳密外洩 最多人用這密碼
http://www.appledaily.com.tw/realtimenews/article/new/20160609/882483/
[5]IOzone Filesystem Benchmark, http://www.iozone.org/, 2014
[6]陳瀅、王慶波、金涬、趙陽、何樂、鄒志樂、吳玉會、楊林，雲端策略：雲端運算與虛擬化技術，臺北市：天下雜誌股份有限公司，2010，第24-25頁
[7]Cloud computing, http://en.wikipedia.org/wiki/Cloud_computing, 2014
[8]Cloud Computing Concerns and Issues,
http://www.onbile.com/info/what-cloud-computing-means/, 2014
[9]Dikaiakos, M.D., Katsaros, D., Mehra, P., Pallis, G., and Vakali, A., “Cloud Computing: Distributed Internet Computing for IT and Scientific Research,” Internet Computing, IEEE, vol. 13, no. 5, 2009, pp. 10-13.
[10]Bare-Metal Deployment — or as elasticity of a cloud in System Center 2012 SP1 Virtual Machine Manager are provid, http://sysmagazine.com/posts/178117/, 2014
[11]OneDrive, https://onedrive.live.com/, 2014
[12]Google App Engine, https://appengine.google.com/, 2014
[13]Amazon Web Services, http://aws.amazon.com/, 2014
[14]淺談雲端儲存與服務， http://www.leosys.com/, 2014
[15]中央研究院數位典藏與數位學習國家型科技計劃後設資料工作組, http://metadata.teldap.tw/share/share-frame.html, 2014
[16]元數據，
http://baike.baidu.com/view/107838.htm?fromtitle=metadata&fromid=8567615&type=syn, 2014
[17]Yu QianCheng, “Metadata integration architecture in enterprise data warehouse system,” 2010 2nd International Conference on Information Science and Engineering (ICISE), Hangzhou, China, 2010, pp. 340-343.
[18]Junsup Song, and Dongkun Shin, “Performance improvement with zero copy technique on FUSE-based consumer devices,” 2014 IEEE International Conference on Consumer Electronics, Las Vegas, NV, 2014, pp. 434-435.
[19]Filesystem in Userspace, http://fuse.sourceforge.net/, 2014
[20]Xiangyong Ouyang, Raghunath Rajachandrasekar, Xavier Besseron, Hao Wang, Jian Huang, Dhabaleswar K. Panda , “CRFS: A Lightweight User-Level Filesystem for Generic Checkpoint/Restart,” 2011 International Conference on Parallel Processing, Taipei City, 2011, pp. 375-384.
[21]FUSE, http://en.wikipedia.org/wiki/Filesystem_in_Userspace, 2014
[22]Jun Lu, Bin Du, Yi Zhu, Ly Ren, and Daiw Li, “A High Performance Cluster File System with Standard Network File System Interface,” International Forum on Information Technology and Applications, Chengdu, 2009, pp. 397-400.
[23]Kejiang Ye, Dawei Huang, Xiaohong Jiang, Huajun Chen,and Shuang Wu, “Virtual Machine Based Energy-Efficient Data Center Architecture for Cloud Computing: A Performance Perspective” IEEE/ACM International Conference on Green Computing and Communications & IEEE/ACM International Conference on Cyber, Physical and Social Computing, 2010, pp.171-178.
[24]Parissis, G., and Apostolopoulos, T., “A Distributed Hash Table-based Approach for Providing a File System Neutral, Robust and Resilient Storage Infrastructure,” Next Generation Internet Networks, Aveiro, 2009, pp. 1-8.
[25]Rajan, A., and Joshi, B.K., “Performance comparison of 20 Gbps and 40 Gbps InfiniBand interconnect,” 2014 International Conference on Computing for Sustainable Global Development, New Delhi, 2014, pp.253-258.
[26]Tiezhu Zhao, and Jinlong Hu, “Performance Evaluation of Parallel File System Based on Lustre and Grey Theory,” 2010 9th International Conference on Grid and Cooperative Computing, Nanjing, 2010, pp. 118-123.
[27]GlusterFS, http://www.gluster.org/, 2014
[28]Introduction to Lustre, http://wiki.lustre.org/manual/LustreManual18_HTML/IntroductionToLustre.html, 2014
[29]Hadoop分布式文件系统：架构和设计, https://hadoop.apache.org/docs/r1.0.4/cn/hdfs_design.html, 2014
[30]Zhonglei Fan, and Xiangmo Zhao, “An enhanced journaling method for clustered file system with shared storage,” 2010 3rd International Conference on Advanced Computer Theory and Engineering, Chengdu, 2010, pp. 78-81.
[31]Anatomy of Linux journaling file systems, http://www.ibm.com/developerworks/library/l-journaling-filesystems/, 2014
[32]Journaling file system, http://en.wikipedia.org/wiki/Journaling_file_system, 2014
[33]Anatomy of ext4, http://www.ibm.com/developerworks/library/l-anatomy-ext4/, 2014
[34]XFS, http://en.wikipedia.org/wiki/XFS, 2014
[35]GlusterFS 集群文件系统研究， http://www.oschina.net/question/12_25567, 2014
[36]Logan, J., and Dickens, P., “Towards an understanding of the performance of MPI-IO in Lustre file systems,” 2008 IEEE International Conference on Cluster Computing, Tsukuba, 2008, pp. 330-335.
[37]黃威誌，雲端運下具有服務品質之資源分配，碩士論文，國立台北科技大學電資學院，台北，2014