跳到主要內容

臺灣博碩士論文加值系統

(100.28.2.72) 您好!臺灣時間:2024/06/13 13:39
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果 :::

詳目顯示

: 
twitterline
研究生:楊承羲
研究生(外文):YANG, CHENG-XI
論文名稱:植基於 Hadoop 叢集互動式網頁應用系統開發 - 以線上測驗為例
論文名稱(外文):Interactive web application system development based on Hadoop cluster - A Case Study of Online Examination
指導教授:陳彼得陳彼得引用關係
指導教授(外文):CHEN, PE-DE
口試委員:陳振楠左豪官陳彼得
口試委員(外文):CHEN, JEN-NANTso, Hao-KuanCHEN, PE-DE
口試日期:2019-06-21
學位類別:碩士
校院名稱:中國科技大學
系所名稱:資訊工程系資訊科技應用碩士在職專班
學門:工程學門
學類:電資工程學類
論文種類:學術論文
論文出版年:2019
畢業學年度:107
語文別:中文
論文頁數:54
中文關鍵詞:雲端運算HadoopDocker多層式架構線上測驗
外文關鍵詞:Cloud ComputingHadoopDockerMulti-Layer ArchitectureOnline Examination System
相關次數:
  • 被引用被引用:0
  • 點閱點閱:239
  • 評分評分:
  • 下載下載:36
  • 收藏至我的研究室書目清單書目收藏:0
傳統三層式(3-Tier)架構是目前Web應用程式最普遍使用的架構;分別是表示層、應用層,與資料層。此架構若應用於雲端運算平台,具有以下缺點;增加系統開發人員學習大數據資料分析工具時間;應用系統開發套件與雲端運算平台之專案容易發生衝突;替換或升級元件會直接影響應用系統執行效率。本研究開發互動式網頁應用系統結合Hadoop 雲端運算平台(以下簡稱: Hadoop),運用Docker軟體貨櫃技術建立與管理 Hadoop,相對於實體機或傳統虛擬化技術,可減少開發、測試與部署的時間,並以多層式架構(N-Tier)為基礎,提出 Hadoop為核心的五層系統架構,由上至下分別是:負載平衡層(Load Balancing tier)、使用者介面層(UI tier)、應用層(Application tier)、資料存取層(Data Access tier),及資料湖層(Data Lake tier)。
如前所述,本研究,開發Hadoop互動式網頁應用系統,以線上測驗系統為例,有效降低建置 Hadoop 環境的成本。並以HDFS 檔案瀏覽介面,簡化使用者在Hadoop繁複的操作流程並可執行大數據分析工具;Pig、Hive、HBase,及 Phoenix。最後,本研究之系統架構,前端、後端,與 Hadoop均可獨立開發與部署,具有良好的擴充性。

The traditional three-tier (3-Tier) architecture is the most commonly used architecture for Web applications: including presentation layer, application layer, and data layer. If this architecture is applied to the cloud computing platform, it has the following disadvantages: increasing time for system developers to learn big data analysis tools; application development suite and cloud computing platform projects are prone to conflict; replacing or upgrading components directly affects the performance of the application system. This study develops an interactive web application system combined with Hadoop cloud computing platform (hereinafter referred to as: Hadoop), and uses Docker container technology to build and manage Hadoop, compared to the physical machine or traditional virtualization technology, the study can reduce the time of the development, testing, and deployment, Based on the multi-tier architecture (N-Tier), a five-tier architecture with Hadoop as the core is proposed. From top to bottom, it is Load Balancing Tier, UI Tier, Application Tier, Data Access Tier and Data Lake Tier.
As mentioned above, the study develop an interactive web application system based on Hadoop, taking online examination system as an example, and reduce the cost of Hadoop. This study uses the HDFS file browsing interface to simplify the user's complicated operation process in Hadoop and perform big data analysis tools; Pig, Hive, HBase, and Phoenix. Finally, the system architecture of this study, front-end, back-end, and Hadoop can be independently developed and deployed, and has with good scalability.
中文摘要 i
Abstract ii
謝誌 iii
目 錄 iv
表目錄 v
圖目錄 vi
第壹章緒論 1
第一節研究背景 1
第二節研究動機與目的 2
第三節論文章節概要 5
第貳章文獻探討 6
第一節雲端運算 6
第二節 Hadoop 雲端運算平台 7
第三節非關聯式資料庫 14
第四節Docker 軟體貨櫃技術 18
第參章系統架構與設計 21
第一節多層式系統架構 21
第二節Hadoop雲端運算平台 26
第三節Docker Compose 編排 Hadoop 29
第肆章系統功能設計與實作 32
第一節系統功能設計 32
第二節互動式網頁應用系統介面 33
第三節系統實作 34
第伍章結論與未來研究方向 40
第一節結論 40
第二節未來研究方向 40
參考文獻 41
[1]Amazon EMR – Amazon Web Services, Retrieved May 11, 2019, from https://aws.amazon.com/emr/
[2]Anderson, C. (2015). Docker [Software engineering]. IEEE Software, 32(3), 102–c3. doi:10.1109/ms.2015.62
[3]Apache Hadoop, Retrieved May 11, 2019, fromhttps://hadoop.apache.org/
[4]Apache HBase, Retrieved May 11, 2019, fromhttps://hbase.apache.org/
[5]Apache Hive TM, Retrieved May 11, 2019, fromhttps://hive.apache.org/
[6]Apache Phoenix, Retrieved May 11, 2019, fromhttps://phoenix.apache.org/
[7]Azure HDInsight - Hadoop, Spark, & Kafka Service, Retrieved May 11, 2019, from https://azure.microsoft.com/en-us/services/hdinsight/
[8]Baldini, I., Castro, P., Chang, K., Cheng, P., Fink, S., Ishakian, V., … Suter, P. (2017). Serverless Computing: Current Trends and Open Problems. Research Advances in Cloud Computing, 1–20. doi:10.1007/978-981-10-5026-8_1
[9]Bhimani, J., Yang, Z., Leeser, M., & Mi, N. (2017). Accelerating big data applications using lightweight virtualization framework on enterprise cloud. 2017 IEEE High Performance Extreme Computing Conference (HPEC). doi:10.1109/hpec.2017.8091086
[10]Boettiger, C. (2015). An introduction to Docker for reproducible research. ACM SIGOPS Operating Systems Review, 49(1), 71–79. doi:10.1145/2723872.2723882
[11]Borthakur, D., Rash, S., Schmidt, R., Aiyer, A., Gray, J., Sarma, J. S., … Menon, A. (2011). Apache hadoop goes realtime at Facebook. Proceedings of the 2011 International Conference on Management of Data - SIGMOD ’11. doi:10.1145/1989323.1989438
[12]Chang, F., Dean, J., Ghemawat, S., Hsieh, W. C., Wallach, D. A., Burrows, M., Chandra, T.,Fikes, A., and Gruber, R. E. (2008). Bigtable: A distributed storage system for structured data. ACMTrans. Comput. Syst. 26, 2, Article 4 (June 2008), 26 pages. DOI = 10.1145/1365815.1365816.
[13]Cloud Dataproc - Cloud-native Apache Hadoop, Retrieved May 11, 2019, from https://cloud.google.com/dataproc/
[14]Cloudera, Retrieved May 11, 2019, fromhttps://www.cloudera.com/
[15]DB-Engines Ranking, Retrieved May 11, 2019, fromhttps://db-engines.com/en/ranking
[16]DEAN, J., AND GHEMAWAT, S. (2004). Mapreduce: simplified data processing on large clusters. In OSDI’04: Proceedings ofthe 6th conference on Symposium on Opearting Systems Design & Implementation (Berkeley, CA, USA, 2004), USENIXAssociation, pp. 10–10.
[17]Fang, H. (2015). Managing data lakes in big data era: What’s a data lake and why has it became popular in data management ecosystem. 2015 IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER). doi:10.1109/cyber.2015.7288049
[18]Gates, A. F., Natkovich, O., Chopra, S., Kamath, P., Narayanamurthy, S. M., Olston, C., … Srivastava, U. (2009). Building a high-level dataflow system on top of Map-Reduce. Proceedings of the VLDB Endowment, 2(2), 1414–1425. doi:10.14778/1687553.1687568
[19]Ghemawat, S., Gobioff, H., & Leung, S.-T. (2003). The Google file system. Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles - SOSP ’03. doi:10.1145/945445.945450
[20]Hortonworks Acquires SequenceIQ to Provide Automated Deployment of “Hadoop Everywhere” i, Retrieved May 11, 2019, fromhttps://hortonworks.com/blog/hortonworks-acquires-sequenceiq-to-provide-automated-deployment-of-hadoop-everywhere/
[21]Hortonworks, Retrieved May 11, 2019, fromhttps://hortonworks.com/
[22]Ismail, B. I., MostajeranGoortani, E., Ab Karim, M. B., Ming Tat, W., Setapa, S., Luke, J. Y., & Hong Hoe, O. (2015). Evaluation of Docker as Edge computing platform. 2015 IEEE Conference on Open Systems (ICOS). doi:10.1109/icos.2015.7377291
[23]Jadeja, Y., & Modi, K. (2012). Cloud computing - concepts, architecture and challenges. 2012 International Conference on Computing, Electronics and Electrical Technologies (ICCEET). doi:10.1109/icceet.2012.6203873
[24]Liu, D., & Zhao, L. (2014). The research and implementation of cloud computing platform based on docker. 2014 11th International Computer Conference on Wavelet Actiev Media Technology and Information Processing(ICCWAMTIP). doi:10.1109/iccwamtip.2014.7073453
[25]Mehul Nalin Vora. (2011). Hadoop-HBase for large-scale data. Proceedings of 2011 International Conference on Computer Science and Network Technology. doi:10.1109/iccsnt.2011.6182030
[26]Mell, P. M., &Grance, T. (2011). The NIST definition of cloud computing. doi:10.6028/nist.sp.800-145
[27]Moniruzzaman ABM, Hossain SA (2013) Nosql database: new era of databases for big dataanalytics-classification, characteristics and comparison. arXiv preprint arXiv:1307.0191Pang B, Lee L (2008) Opinion mining and sentiment analysis. Found Trends Inf Retrieval 2(1–2):1–135
[28]Olston, C., Reed, B., Srivastava, U., Kumar, R., & Tomkins, A. (2008). Pig latin. Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data - SIGMOD ’08. doi:10.1145/1376616.1376726
[29]Philip Chen, C. L., & Zhang, C.-Y. (2014). Data-intensive applications, challenges, techniques and technologies: A survey on Big Data. Information Sciences, 275, 314–347. doi:10.1016/j.ins.2014.01.015
[30]PoweredBy - Hadoop Wiki - Apache Wiki, Retrieved May 11, 2019, fromhttps://wiki.apache.org/hadoop/PoweredBy
[31]Qi Zhang · Lu Cheng · Raouf Boutaba (2010). Cloud computing: state-of-the-art and research challenges, J Internet Serv Appl (2010) 1: 7–18. DOI 10.1007/s13174-010-0007-6
[32]R. P Padhy, M. R. Patra, S. C. Satapathy, “RDBMS to NoSQL:Reviewing Some Next-Generation Non-Relational Database’s”,International Journal of Advance Engineering Sciences andTechnologies, Vol. 11, Issue No. 1, 015-030, 2011.
[33]Thusoo A, Sarma JS, Jain N, Shao Z, Chakka P, Anthony S,Liu H, Wyckoff P, Murthy R. (2009). Hive - A Warehousing Solution Over a Map-ReduceFramework. Proc VLDB Endowment2(2):1626–1629
[34]V. K. Vavilapalli, A. C. Murthy, C. Douglas, S. Agarwal,M. Konar, R. Evans, T. Graves, J. Lowe, H. Shah, S. Seth,B. Saha, C. Curino, O. O’Malley, S. Radia, B. Reed, andE. Baldeschwieler. (‎2013). Apache Hadoop YARN: Yet AnotherResource Negotiator. In Proc. ACM Symp. on CloudComputing (SoCC), Santa Clara, CA, USA, 2013.
[35]Zhang, R., Li, M., & Hildebrand, D. (2015). Finding the Big Data Sweet Spot: Towards Automatically Recommending Configurations for Hadoop Clusters on Docker Containers. 2015 IEEE International Conference on Cloud Engineering. doi:10.1109/ic2e.2015.101
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top
無相關期刊