臺灣博碩士論文加值系統

製造執行系統主要用來收集製造現場物品的生產資訊，讓使用者能即時得到相關資訊進行分析，以提高生產效率。但是隨著生產資訊與日俱增，達到巨量資料，本論文討論如何以具有NoSQL資料庫特性的HBase搭配Java程式，實作出具有生產查詢功能的系統。本論文提出了兩種定義資料庫schema的方式。第一種方法是將物品型號和序號合併為Row Key，共同儲存在同一表格，此方法稱為『mixed方法』。我們提出的第二種方法，是將個別型號獨立建成新的表格，並以物品序號為Row Key，此方法稱為『assorted方法』。我們實作此二方法，透過一系列的實驗以及不同數量的資料集，比較其查詢效率。實驗結果顯示，assorted方法在point query中的查詢效率遠比mixed方法好，而在scan query中，兩種方法並沒有很大差別，且隨著資料量呈現線性成長。綜合實驗發現，HBase 對於單一資料以列為查詢條件的處理非常快速且穩定，對於掃瞄整個表格的查詢句則會隨著資料量呈現線性成長。 整體而言，assorted方法的執行效率比mixed方法來得好。

The Manufacturing Execution Systems (MES) are used in factories to provide the information about objects during the production process in real time. Traditionally, the MES is constructed using relational databases, but the efficiency might decay when dealing with big data. In this thesis, we study how to use HBase and Java to build a system for supporting the querying on manufacturing process. We propose two methods of defining schemas in HBase. In the “mixed” method, the information of all objects are stored in the same table where the product number and the serial number of an object are combined to form the Row Key. We then propose another method, called the “assorted” method. It will create a new table for each product number and the serial number of each object is the Row Key. We have performed a series of experiments based on a variety of datasets to compare the efficiency of the two proposed methods. Experimental results show that HBase performs very well if it only needs to retrieve a single row, but the execution time will increase linearly if it needs to examine a set of data. Overall, the assorted method is better than the mixed method.

目錄
摘要 III
Abstract IV
誌謝 V
圖目錄 VIII
表目錄 IX
第1章 序論 1
1.1 研究動機與目的 1
1.2 研究方法與貢獻 2
1.3 相關研究 3
1.4 論文架構 5
第2章 系統功能介紹 6
2.1 HBase 資料庫特性 6
2.2 系統架構與功能 9
第3章 以型號與序號合併之方法 12
3.1 資料定義 12
3.2 功能實作 13
第4章 以型號分類之方法 24
4.1 資料定義 24
4.2 功能實作 25
第5章 實驗 29
5.1 資料集和查詢句設計 29
5.2 不同查詢句的效率評估 31
5.3 Scan Query和Column Family的分析 34
5.4 Point Query的分析 37
5.5 將型號建立成獨立一個Column Family之實驗 38
5.6 改變Hadoop中Block Size對執行效率的影響之分析 40
第6章 結論與未來方向 41
參考文獻 42

[ABCD*12] Amitanand Aiyer, Mikhail Bautin, Guoqiang Jerry Chen, Pritam Damania Prakash Khemani, Kannan Muthukkaruppan, Karthik Ranganathan Nicolas Spiegelberg, Liyin Tang, Madhuwanti Vaidya, “Storage Infrastructure Behind Facebook Messages Using HBase at Scale”, Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, 2012.
[AHadoop] Apache Hadoop project, http://hadoop.apeche.org
[AHBase] Apache HBase Reference Guide, http://hbase.apache.org/book.html
[AK12] Amandeep Khurana, “Introduction to HBase Schema”, The Usenix Magazine, Vol.37, No.5, 2012.
[CSDS11] Zhao Cao, Charles Sutton, Yanlei Diao, Prashant Shenoy, “Distributed inference and query processing for RFID tracking and monitoring”, Proceedings of the VLDB conference, 2011.
[EMJ15] Ahmed Eldawy，Mohamed F. Mokbel， Christopher Jonathan，“HadoopViz: A MapReduce Framework for Extensible Visualization of Big Spatial Data”, Proceedings of the ICDE Conference, 2015.
[JOSW10] Dawei Jiang, Beng Chin Ooi, Lei Shi, Sai Wu, “The Performance of
MapReduce: An In-depth Study”, Proceedings of the VLDB Endowment, Vol. 3, No. 1, 2010.
[LWST13] Peng Lu, Sai Wu, Lidan Shou, Kian-Lee Tan, “An Efficient and Compact Indexing Scheme for Large-scale Data Store”, Proceedings of the ICDE Conference, 2013.
[MNV11] Mehul Nalin Vora, “Hadoop-HBase for Large-Scale Data”, Proceedings of the International Conference on Computer Science and Network Technology, 2011.
[OMES] Oracle MES User Guide, https://docs.oracle.com/cd/B34956_01/
current/acrobat/120gmomesug.pdf.
[SWL13] Bin Shao, Haixun Wang, Yatao Li, “Trinity: A Distributed Graph Engine on a Memory Cloud”, Proceedings of the ACM SIGMOD International Conference on Management of Data, Pages 505-516, 2013.
[TCCY13] Yung-Shun Tsai, Ruey-Shun Chen, Yeh-Cheng Chen, Chun-Ping Yeh, “An RFID-based Manufacture Process Control and Supply Chain Management in the Semiconductor Industry”, International Journal of Technology Management, Vol.12, No. 1/2, pp. 85-105, 2013.
[TP4.0I] 行政院生產力 4.0 發展方案，http://www.bost.ey.gov.tw/Upload/UserFiles/%E8%A1%8C%E6%94%BF%E9%99%A2%E7%94%9F%E7%94%A2%E5%8A%9B4_0%E7%99%BC%E5%B1%95%E6%96%B9%E6%A1%88.pdf
[ZCTW13] X. Zhang, L. Chen, Y. Tong, and M. Wang, “EAGRE: Towards Scalable I/O Efficient SPARQL Query Evaluation on the Cloud”, Proceedings of the ICDE Conference, 2013.
[李15] 李韋錫，“快速處理生產流程查詢與控管之研究” ，碩士論文，國立台灣海洋大學資工系研究所，2015。