臺灣博碩士論文加值系統

近年來，廣泛的智能電錶已經廣泛應用於各個領域的時間序列工程參數化;導致處理大數據的問題。大量的數據需要被傳輸，存儲，處理和檢索。存儲和訪問這些大數據在時間，空間和帶寬上變得昂貴。研究的目的是找出解決問題的辦法。研究中開發的一個解決方案是壓縮/解壓縮工程參數。變量的數據格式有三（03）部分：128位全局唯一標識符（GUID），64位時間戳參數和64位浮點值參數。已經應用和實現了三種編碼/解碼算法。這些方法已經將原始歷史數據大小減少了40％，以及存儲成本。算法的性能：壓縮比，節省百分比和壓縮/減壓時間和速度已被測量。解壓縮過程已被證明比基於歷史數據的壓縮過程更快。

In recent years, numerous smart meters have been widely installed to aggregate time series engineering parameters over fields; it has led to problems of handling big data. The huge volumes of data need to be transmitted, stored, processed as well as retrieved. Storing and accessing these big data have become expensive in time, space and bandwidth. The aim of the study is to find a solution for the problems. One solution developed in the study is to compress/decompress the engineering parameters. The data format of the variables has three (03) portions: 128-bit Global Unique Identifier (GUID), 64-bit time stamp parameter, and 64-bit floating point value parameter. Three encoding/decoding algorithms have been applied and implemented. The approaches have reduced the original historical data size 40% off as well as the storage cost. The algorithms’ performances: the compression ratio, the saving percentage and the compression/decompression time and speed have been measured. The decompression process has been proved faster than the compression process based on the historical data.

Chinese Abstract i
English Abstract ii
Acknowledgement iii
Chapter 1: Introduction 1
1.1 Problem Statements 1
1.2 Objectives 2
1.3 Scope 2
Chapter 2: Background 3
2.1 Concepts of compression 3
2.1.1 Run-length encoding 4
2.1.2 Huffman Coding 4
2.1.3 Lempel Ziv Welch Algorithm 5
2.2 Concepts of GUID 6
2.3 Related Work 7
Chapter 3: Methodologies 10
3.1 Data 10
3.1.1 Anasystem 12
3.1.2 Data Model 13
3.2. Compression 14
3.2.1. Data pre-processing 14
3.2.2 GUID Compression 15
3.2.3 Timestamp compression 18
3.2.4 Value compression 21
Chapter 4: Implementations and Results 24
4.1 Environment Setting 24
4.2 Data and Compression Performance Evaluation 24
4.3 GUID Algorithm Implementation and Results 25
4.4 Timestamp Compression Algorithm Implementation and Results 27
4.5 Value Parameter Compression Algorithm Implementation and Results 29
4.5 Compression and decompression performance verification 31
Chapter 5: Conclusion and Discussion 34
Bibliography 36
Appendix 1 I
Appendix 2 I
Appendix 3 II
Appendix 4 II
Appendix 5 III
Appendix 6 III

[1]
R. A. A. R. Al-Ali, Role of Internet of Things in the Smart Grid Technology, UAE: scientific research publishing, 2015.
[2]
"Business Analytics," [Online]. Available: https://practicalanalytics.co/2013/10/23/market-sizing-analytics-and-big-data/ . [Accessed 2 7 2015].
[3]
L. e. al., "Improving Floating Point Compression through Binary Masks," IEEE, 2013.
[4]
F. a. Sullivan, "World’s Top Global Mega Trends To 2025 and Implications to Business, Society and Cultures," 2015.
[5]
NASSCOM, "Big Data-The next big thing".
[6]
WKIBON, "Big data vendor revenue and market forcast," [Online]. Available: https://wikibon.com/executive-summary-big-data-vendor-revenue-and-market-forecast-2011-2026/. [Accessed 2 07 2017].
[7]
M. Meeker, "INTERNET TRENDS 2016-CODE CONFERENCE," 2016.
[8]
WIKIBON, "Big Data Revenue by Sub-Type," [Online]. Available: http://wikibon.org/w/images/1/1b/BigDataRevenueBySubType2013.png. [Accessed 2 07 2017].
[9]
ETHW, "Historical of lossless data compression algorithm," [Online]. Available: http://ethw.org/History_of_Lossless_Data_Compression_Algorithms . [Accessed 2 07 2017].
[10]
A. K. a. R. Poli, "Evolutionary Synthesis of Lossless Compression Algorithms with GP-zip3," I.
[11]
K. Sayood, in Introduction to Data Compression, USA, 2006.
[12]
S. S. e. al., "Performance Measurement and Comparison of Lossless Compression Algorithms," Global Journal Enterprise Information System, vol. 3, 2011.
[13]
G. E. Blelloch, in Itroduction to Data Compression, USA, 2013.
[14]
GeeksforGeeks, "A computer science portal for geeks," [Online]. Available: http://www.geeksforgeeks.org/lzw-lempel-ziv-welch-compression-technique/. [Accessed 02 07 2017].
[15]
P. L. e. al., in A Universally Unique Identifier (UUID) URN Namespace, RFC4122, 2005.
[16]
A. e. al., in generating and compressing universally unique identifiers (UUIDs) using counter having high order bit to low-order bit, USA, 1999.
[17]
C. e. al., "Efficient Use of GUIDs," IEEE, 2008.
[18]
OpenTSDB, "overview," 2010. [Online]. Available: http://opentsdb.net/overview.html. [Accessed 02 07 2017].
[19]
F. H. Mathis, in A Generalized Birthday Problem, vol. 33, Society for Industrial and Applied Mathematics, 1991.
[20]
P. e. al., "Fast Lossless Compression of Scientific Floating-Point Data," IEEE, 2006.
[21]
S. e. al, "Gorilla: A Fast, Scalable, In-Memory Time Series Database," vol. 8, 09 2015.
[22]
A. Kattan, "Evolutionary Synthesis of Lossless Compression Algorithms: the GP-zip Family," Thesis, 2010.
[23]
S. Lawson, 2013. [Online]. Available: https://trackpal.com/blog/historical-data-in-a-real-time-world/. [Accessed 02 07 2017].
[24]
Anasystem. [Online]. Available: http://www.anasystem.com.tw/home/. [Accessed 02 07 2017].
[25]
D. e. al., "A Proposal of Substitute for Base85/64 – Base91".
[26]
Josefsson, "The Base16, Base32, and Base64 Data Encodings," 2006.
[27]
J. e. al., "Comparison of Brotli, Deflate, Zopfli, LZMA, LZHAM and Bzip2 Compression Algorithms".