臺灣博碩士論文加值系統

在這篇論文中，我們描述 FileFarm: 一個建構於現有雲端儲存服務之上，為防止機密資料外洩、提升可靠性並去除對單一雲端依賴而設計的雲中雲儲存系統。 為了解決既有雲中雲設計因集中式資料庫而造成的一致性和負載平衡問題，FileFarm 採取端對端(P2P)的解決方案。在FileFarm中，每個雲端服務皆為可獨立運作的單元，對客戶端提供相同的服務。這些被稱為Farmer的單元相互合作，共同組成一個端對端儲存網路。FileFarm可容忍同時發生於至多K-1個Farmer上的錯誤，其中K為一個可調整的系統參數。當任何Farmer發生問題而無法提供服務時，FileFarm系統會自動開啟一個修補機制，將資料備份到剩餘存活的Farmer上，確保每個資料區塊都在網路中被儲存了至少K份。為了在端對端網路中有效率地尋找資源，FileFarm實作了Kademlia分散式雜湊表協定。 FileFarm 從Kademlia 中繼承了許多重要的特性，包含： (1) 備份數維護、 (2) 高效率搜尋、 (3) 負載平衡的設計。除此之外，作為一個企業級儲存系統，FileFarm還需滿足以下四項條件：(1) 資料機密性 (2) 權限管理 (3) 成本效益 (4) 可存取性。 為此，FileFarm以此四個條件為面向分別設計對應的機制： (1) 加密與資訊分散演算法 (2) 分散式認證 (3) 儲存空間釋放與下載次序差異化 (4) 公有雲ID指定規則。我們基於系統所提供的特性將FileFarm與相關文獻進行比較，同時我們實作了一個系統原型並利用此原型進行一系列實驗以驗證我們聲稱的特性。此系統原型同時也是我們所提出的結構化端對端資料儲存解決方案之產品原型。

In this thesis, we describe FileFarm: a secured storage overlay that leverages existing cloud services to form a cloud-of-clouds storage system with better robustness, no single-point-of-failure and minimal data leakage concerns. To resolve the consistency and load-balancing issues caused by a centralized database design in conventional cloud-of-clouds work, FileFarm adopts a P2P strategy, in which each cloud operates as an independent node providing identical service for clients. The storage nodes, called farmers, cooperate with each other to form a peer-to-peer network, which tolerates concurrent failures occurring at any K-1 farmers, where K is a configurable system-wise parameter. In case of failure occurring at any farmer, a storage repair procedure will be triggered automatically, which backs up data to surviving farmers and maintain K copies of each piece of data. To lookup resources efficiently in a P2P network, FileFarm implements Kademlia DHT(Distributed Hash Table) protocol. Several desired properties of FileFarm are inherited from Kademlia: (1) redundancy maintenance, (2) efficient search and (3) load-balancing design. However, in order to serve as an enterprise-level storage, 4 further properties are required: (1) data confidentiality, (2) access management, (3) cost-efficiency, (4) retrievability. FileFarm meets these requirements by designing corresponding mechanisms, which collectively make FileFarm a robust, secure and cost-efficient storage solution: (1) Encryption and Information Dispersal Algorithm, (2) Decentralized Authentication, (3) Storage Release and Prioritized Download, (4)Public Farmer ID Assignment. We compare FileFarm with related implementations in various aspects of properties. We also implement a proof-of-concept and perform a series of experiments on it to verify our claims. The proof-of-concept not only confirms our claims but also served as a product prototype of our structured P2P file storage solution.

口試委員會審定書 iii
誌謝 v
摘要 vii
Abstract ix
1 Introduction 1
1.1 Rise and Prominence of Cloud Storage 1
1.2 Concerns of Single Cloud Solution 1
1.3 FileFarm Overview 2
2 Related Work 5
2.1 Redundancy and Confidentiality 6
2.2 LocationQuery 7
2.3 StorageRepair 8
3 Background 11
3.1 Cloud Computing 11
3.2 Cloud Storage Service 12
3.3 Cloud-of-Clouds 13
3.4 Hybrid Cloud 13
3.5 Peer-to-Peer Systems 14
3.6 Distributed Hash Table 15
3.7 Kademlia 15
3.7.1 Load Balancing 15
3.7.2 Efficient Search 16
3.7.3 Redundancy Maintenance 20
3.8 Information Dispersal Algorithm 21
3.9 Public Key Infrastructure 22
4 Methodology 25
4.1 System Architecture 25
4.2 Application Models and Process Flows 26
4.3 DHT-Based Approach 29
4.4 Beyond Kademlia 29
4.5 Data Confidentiality 30
4.6 Access Management 32
4.6.1 Decentralized Authentication 32
4.7 Cost Efficiency 35
4.7.1 Storage Release 35
4.7.2 Prioritized Download 36
4.8 Retrievability 38
4.8.1 Public Farmer ID Assignment 38
5 Experiments and Results 41
5.1 Environment 41
5.2 Experiment: NODE_LOOKUP Efficiency 42
5.3 Experiment: VALUE_LOOKUP Efficiency 43
5.4 Experiment: Retrievability 44
5.5 Experiment: Throughput 46
5.6 Experiment: Cost – Storage Release 48
5.7 Experiment: Cost – Prioritized Download 49
6 Conclusion 51
Bibliography 53

[1] The state of software-defined storage, hyperconverged and cloud storage. https://s3.amazonaws.com/ydtimages/~yourdai7/wp-content/uploads/ 2017/08/18133654/SDS_2016_Report_FINAL.pdf, 2017. [Online; accessed 27-June-2019].
[2] Amazon Simple Storage Service. https://aws.amazon.com/s3/, 2019. [Online; accessed 27-June-2019].
[3] aMule. http://www.amule.org, 2019. [Online; accessed 02-July-2019].
[4] Apache ZooKeeper. https://zookeeper.apache.org, 2019. [Online; accessed 10-July-2019].
[5] BitTorrent. https://www.bittorrent.com/bittorrent-free, 2019. [Online; accessed 02-July-2019].
[6] eMule. https://www.emule-project.net/home/perl/general.cgi?l=1, 2019. [Online; accessed 02-July-2019].
[7] Ethereum. https://www.ethereum.org, 2019. [Online; accessed 02-July-2019].
[8] Google Cloud Storage. https://cloud.google.com/storage/, 2019. [Online; accessed 27-June-2019].
[9] IPFS. https://ipfs.io, 2019. [Online; accessed 02-July-2019].
[10] Microsoft Azure Cloud Storage. https://azure.microsoft.com/en-us/ product-categories/storage/, 2019. [Online; accessed 27-June-2019].
[11] Napster. https://us.napster.com, 2019. [Online; accessed 02-July-2019].
[12] RFC 4158. https://tools.ietf.org/html/rfc4158, 2019. [Online; accessed 02-July-2019].
[13] Storij. https://storj.io, 2019. [Online; accessed 02-July-2019].
[14] H. Abu-Libdeh, L. Princehouse, and H. Weatherspoon. RACS: a case for cloud storage diversity. In Proceedings of the 1st ACM symposium on Cloud computing, pages 229–240. ACM, 2010.
[15] M. A. AlZain, E. Pardede, B. Soh, and J. A. Thom. Cloud computing security: From single to multi-clouds. In 2012 45th Hawaii International Conference on System Sciences, pages 5490–5499, Jan 2012.
[16] M.A.AlZain,B.Soh,andE.Pardede.MCDB:usingmulti-cloudstoensuresecurity in cloud computing. In 2011 IEEE Ninth International Conference on Dependable, Autonomic and Secure Computing, pages 784–791. IEEE, 2011.
[17] A. Bessani, M. Correia, B. Quaresma, F. André, and P. Sousa. DepSky: depend- able and secure storage in a cloud-of-clouds. ACM Transactions on Storage (TOS), 9(4):12, 2013.
[18] K. D. Bowers, A. Juels, and A. Oprea. HAIL: A high-availability and integrity layer for cloud storage. In Proceedings of the 16th ACM conference on Computer and communications security, pages 187–198. ACM, 2009.
[19] C. Cachin, R. Haas, and M. Vukolic. Dependable storage in the intercloud. IBM research, 3783:1–6, 2010.
[20] X. S. Cai and L. Devroye. A probabilistic analysis of Kademlia networks. In In- ternational Symposium on Algorithms and Computation, pages 711–721. Springer, 2013.
[21] D. Dobre, P. Viotti, and M. Vukolić. Hybris: Robust hybrid cloud storage. In Pro- ceedings of the ACM Symposium on Cloud Computing, pages 1–14. ACM, 2014.
[22] Y. Hu, H. C. Chen, P. P. Lee, and Y. Tang. NCCloud: applying network coding for the storage repair in a cloud-of-clouds. In FAST, page 21, 2012.
[23] L. Lamport, R. Shostak, and M. Pease. The Byzantine generals problem. ACM Transactions on Programming Languages and Systems (TOPLAS), 4(3):382–401, 1982.
[24] M. Li, C. Qin, and P. P. Lee. CDStore: Toward reliable, secure, and cost-efficient cloud storage via convergent dispersal. In 2015 {USENIX} Annual Technical Con- ference ({USENIX}{ATC} 15), pages 111–124, 2015.
[25] P. Maymounkov and D. Mazieres. Kademlia: A peer-to-peer information system based on the xor metric. In International Workshop on Peer-to-Peer Systems, pages 53–65. Springer, 2002.
[26] P. Mell, T. Grance, et al. The NIST definition of cloud computing. 2011.
[27] M.O.Rabin.Efficientdispersalofinformationforsecurity,loadbalancing,andfault
tolerance. Journal of the ACM (JACM), 36(2):335–348, 1989.
[28] K. Ren, C. Wang, and Q. Wang. Security challenges for the public cloud. IEEE
Internet Computing, 16(1):69–73, Jan 2012.
[29] A. Rowstron and P. Druschel. Pastry: Scalable, decentralized object location, and routing for large-scale peer-to-peer systems. In IFIP/ACM International Conference on Distributed Systems Platforms and Open Distributed Processing, pages 329–350. Springer, 2001.
[30] A. Shamir. How to share a secret. Communications of the ACM, 22(11):612–613, 1979.
[31] I.Stoica,R.Morris,D.Karger,M.F.Kaashoek,andH.Balakrishnan.Chord:Ascal- able peer-to-peer lookup service for internet applications. ACM SIGCOMM Com- puter Communication Review, 31(4):149–160, 2001.
[32] C. Taylor. Survey reveals tech trends reshaping data storage.
https://www.enterprisestorageforum.com/storage-management/ survey-reveals-tech-trends-reshaping-data-storage.html, 2018. [Online; accessed 27-June-2019].
[33] M.Vukolic.TheByzantineempireintheintercloud.SIGACTNews,41(3):105–111, 2010.
[34] B. Y. Zhao, L. Huang, J. Stribling, S. C. Rhea, A. D. Joseph, and J. D. Kubiatowicz. Tapestry: A resilient global-scale overlay for service deployment. IEEE Journal on selected areas in communications, 22(1):41–53, 2004.