臺灣博碩士論文加值系統

Apache HBase 與Apache Cassandra 是常見的雲端計算大型分散式的NoSQL資料庫系統，兩者皆具有高彈性和高擴充性的特性，適合使用在巨量資料的環境下。本研究的目的是實現Apache HBase 與Apache Cassandra雲端資料中心的異地備援系統，並且設計一個具高度親和性的圖形使用者介面(Graph User Interface, GUI)。在傳統上大型分散式架構資料庫採用主從式(master-slave)或同儕式(peer-to-peer)等方式備援資料，以降低資料遺失的風險，但是在資料庫的資料完整性表現，如一致性 (Consistency)、高可用性 (Availability) 或分區容忍性 (Partition tolerance) 等往往無法達成所要的需求。因此，本研究首先實現具有上述資料完整性的雲端資料中心的異地備援系統，並且對許多典型資料庫進行效能及效率的測試，再給予性價比 (Cost-Performance Ratio)的評估。再者，應用Apache Thrift的二進制通信協定的技術，建立易於使用的圖形使用者介面，可實現跨平台的大量資料操作，如資料讀取、檔案寫入與二級索引等，以取代不便利的命令列操作方式，提高使用者對於NoSQL資料庫操作的親和性。

Apache HBase and Cassandra are a common cloud computing NoSQL databases for large distributed systems, both with high flexibility and high scalability characteristics, suitable for use in the context of huge amounts of data. The objective of this study is to realize the Apache HBase and Cassandra cloud data centers remote backup systems, and designing a graphical user interface with high affinity (Graph User Interface, GUI). Large distributed architecture database used in traditional master-slave (master-slave) or peers (peer-to-peer) and other ways to backup data in order to reduce the risk of data loss, but the data integrity of the database, such as coherence (Consistency), high availability (Availability) or partition tolerance (Partition tolerance) are often unable to reach the desired demand. Therefore, this study first to achieve the data integrity of cloud data centers remote backup systems, and the effectiveness and efficiency of many typical database testing, then granting performance (Cost-Performance Ratio) assessment. In addition, Apache Thrift binary communications protocol technology, build easy to use graphical user interface that enables large volumes of data across platforms operations, such as data read, file write, and secondary indexes, and so on, to replace an unwieldy command line operations, improving user affinity for NoSQL database operations.

摘要
ABSTRACT
致謝
Directory
List of Figures
List of Tables
Chapter 1. Introduction
Chapter 2. Background and Related Work
2.1 Apache HBase Database
2.2 Apache Cassandra Database
2.3 Apache Thrift
2.4 List of Various Remarkable Databases
Chapter 3. Research Method
3.1 Concept
3.2 Stress Test
3.2.1 Average Data Access
3.2.3 Performance index
3.2.4 Total Cost of ownership 15
.2.5 Cost-performance ratio 16
3.3 Cloud Data Center of Remote Backup Architecture 16
3.4 Build Cloud Data Center and Remote Backup Mechanism 18
3.4.1 System Firewall Settings 18
3.4.2 Build HBase and Cassandra Remote Backup Environment 19
3.5 Graph User Interface of Big Data Environments 24
3.5.1 Graph User Interface Processes 24
3.6 Compile HBase Library to Thrift API 26
3.7 Graph User Interface Operate Database System 27
3.7.1 Login Interface 27
3.7.2 Form Information 29
3.7.3 External File Import 30
3.7.4 Data Write 32
3.7.5 Secondary Index 33
3.7.6 Data Read 34
Chapter 4. Experimental Results and Discussion 35
4.1 Instrument and Software Specifications 35
4.2 Test of Data Center Remote Backup System 36
4.2.1 Read Big Data for Data center 36
4.2.2 Read Big Data of Data center 38
4.2.3 Remoter Backup of Data center 39
4.3 Graph User Interface Test 41
4.3.1 Data Import Function Test 41
4.3.2 Single/Multiple Read Function Test 42
4.3.3 Condition Read Function Test 44
4.4 Cost-Performance Ratio Evaluation 45
4.4.1 Performance Index Evaluation 45
4.4.2 5-Year Term the overall cost calculation 47
4.4.3 Result and Conclusion 47
Chapter 5. Conclusion 51
References 53

[1] B. R. Chang, H.-F. Tsai, C.-F. Huang, and H.-C. Huang, “Private Small-Cloud Computing in Connection with WinCE Thin Client,” Lecture Notes in Artificial Intelligence, Vol. 6422, pp. 172-182, 2010.
[2] C. Franke, S. Morin, A. Chebotko, J. Abraham, “Distributed Semantic Web Data Management in HBase and MySQL Cluster,”, Proceeding of 2011 IEEE International Conference on Cloud Computing, pp. 105-112, 2011.
[3] K. Y. Cheng, Y. L. Pan, C. H. Wu, H. E. Yu, H. S. Chen, W. H. Huang, “Ezilla Cloud Service with Cassandra Databasefor Sensor Observation System,” World Academy of Science, Engineering & Technology, Iss. 70, pp. 132, 2012.
[4] A.Boicea, F. Radulescu, L. L. Agapin, “MongoDB vs Oracle -- Database Comparison,” Emerging Intelligent Data and Web Technologies, pp. 330-335, Sept. 19-21, 2012.
[5] K. Kambatla, A. Pathak, H. Pucha, “Towards Optimizing Hadoop Provisioning in the Cloud,” Proceeding of the 2009 conference on Hot topics in cloud computing, Article No. 22, 2009.
[6] M. Maurya, S. Mahajan, “Performance analysis of MapReduce programs on Hadoop cluster,” Proceeding of World Congress on Information and Communication Technologies, pp. 505-510, 2012.

[7] S. Ramanathan, S. Goel, S. Alagumalai, “Comparison of Cloud database: Amazon's SimpleDB and Google's Bigtable,” Proceeding of World Congress on Information and Communication Technologies, pp. 165-168, 21-23, 2011.
[8] E. Brewer, “CAP twelve years later: How the “rules” have changed,” Proceeding of 2012 IEEE International Conference on Cloud Computing, pp. 23-29, Feb. 2012.
[9] G. Lu, Y. J. Nam, “BloomStore: Bloom-Filter based memory-efficient key-value store for indexing of data deduplication on flash,” Proceeding of 2012 IEEE 28th Symposium on Mass Storage Systems and Technologies, pp. 1-11, 2012.
[10] N. Saxena, N. Bhargava, U. Mahor, N. Dixit, “An Efficient Technique on Cluster Based Master Slave Architecture Design,” Proceeding of 2012 Fourth International Conference on Computational Intelligence and Communication Networks, pp. 561-565, 2012.
[11] N. Liu, F. Liu, “A Bidirectional Ring Cluster-based Peer to Peer System,” Proceeding of 2007 ISITAE '07. First IEEE International Symposium on Information Technologies and Applications in Education, pp. 568-571, 2007.
[12]S. Skeirik, R. B. Bobba, J. Meseguer, “Formal Analysis of Fault-tolerant Group Key Management Using ZooKeeper, Cluster,” Proceeding of 2013 13th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, pp. 636-641, 2013.
[13] S. Kodithuwakku, L. Padmakumara, I. Premadasa, S. Rathnayaka, V. Nanayakkara, S. Perera, “GajaNindu: A Distributed System Management Framework with User-Defined Management Logic,” Proceeding of 2013 Tenth International Conference on Information Technology: New Generations, pp. 83-88, 2013.
[14] D. Borthakur, J. Gray, J. S. Sarma, K. Muthukkaruppan, N. Spiegelberg, H. Kuang, K. Ranganathan, D. Molkov, A. Menon, S. Rash, R. Schmidt, A. Aiyer, “Apache hadoop goes realtime at Facebook,” Proceeding of 2014 International Conference on Signal Propagation and Computer Technology, pp. 1071-1080, 2011.
[15] C. Zheng, H. Xue, W. Shen, Q. Hao, “A heterogeneous sensors integration platform for independent living spaces,” Proceeding of 2012 IEEE 16th International Conference on Computer Supported Cooperative Work in Design, 616-621, 2012.
[16] A. S. John, HBase – Secondary Index, Huawei Technologies, 2013. http:// http://www.thebigdata.cn/upload/2013-10/13101415173453.pdf
[17] A. Prunicki, Apache Thrift, Senior Software Engineer Object Computing. http://jnb.ociweb.com/jnb/jnbJun2009.html
[18] Eucalyptus, 2012. http://open.eucalyptus.com/
[19] Google App Engine, 2012. http://groups.google.com/group/google-appengin
[20] Apache Thrift wiki. http://wiki.apache.org/cassandra/ThriftExamples.
[21] IBM Cloud Computing, 2012. http://www.ibm.com/ibm/cloud/
[22] HBase replication. https://hbase.apache.org/replication.html
[23]Apache Cassandra 1.0 Documentation. http://www.datastax.com/docs/1.0/cluster_architecture/replication
[24] OpenNebula, 2012. http://www.opennebula.org/
[25] Sector/Sphere, National Center for Data Mining, 2012. http://sector.sourceforge.net/
[26] Windows Azure- A Microsoft Solution to Cloud, 2012. http://tech.cipper.com/index.php/archives/332
[27] Welcome to Apache Hadoop, 2012. http://hadoop.apache.org/
[28] Amazon Web Services (AWS), 2012. http://aws.amazon.com/
[29] HBase Architecture. http://www.larsgeorge.com/2010/01/hbase-architecture-101-write-ahead-log.html