臺灣博碩士論文加值系統

近年來，巨量資料分析 (big data analytics)的興起，解決了部份資訊超載的問題。然而，目前社群媒體所提供的資訊檢索方式，仍然尚未能夠針對巨量資料，有效地讓使用者找到目標社群貼文。
甚且，過往對於灰色系統理論的應用研究鮮少應用在處理巨量資料上。為了解決上述的問題，本研究之目的乃提出並開發一個巨量的模糊灰關聯分析社群貼文檢索方法。我們將資料ETL載入到Spark巨量資料平台上，使用者透過設定關鍵字來讓此方法過濾候選貼文集，進而針對社群媒體的非功能性屬性設定其條件(criteria)、屬性目標 (goal)及權重 (weight)等，利用灰色關聯分析進行平行且分散式的運算，依照灰關聯分數的排名推薦前N名 (top-N)社群貼文給使用者。
最後，我們邀請使用者來評估此方法。其檢索的績效顯示相關度 (correlation)為63.91%、查準率 (precision)為95.89%、查全率 (recall)為71.62%及F-measure為77.97％。

Recently, the emerging big data analytics resolve the most problem of information overload. However, the current technologies of information retrieval still are unable to find user’s target posts form big data effectively.
Furthermore, the application of grey system theory was rarely used to deal with big data. To resolve the problem, this study suggests and develops a big data fuzzy grey relational analysis retrieval method for social posts.
After data's ETL (Extract, Transform, Load) into Spark big data platform, then user sets the criteria, goal and weight for the nonfunctional properties of the social media and sets the keywords to method for filter the candidate posts. This method use Grey Relational Analysis to proceed the parallel and distributed computing. And then, it recommends top N (top-N) social posts for users per each post’s grey relational grade.
Finally, we invited 30 users to evaluate this method. The performance shows 63.91% of the correlation and 95.89% of the precision averagely. The recall is 71.62%, and the F-measure is 77.97% averagely.

摘 要 i
ABSTRACT ii
致謝 iii
目錄 iv
圖目錄 v
表目錄 v
第壹章 緒 論 1
第一節 背景與動機 1
第二節 現有問題 2
第三節 研究目的 3
第貳章 文獻探討 4
第一節 Apache Spark 4
第一段 Spark原理 4
第二段 Spark核心 5
第三段 Spark SQL 5
第四段 Hadoop vs Spark 6
第二節 社群媒體現有的檢索方式及主題標籤 7
第三節 灰色理論與灰色關聯分析 8
第四節 模糊集 (Fuzzy Set) 9
第參章 研究方法 10
第一節 社群貼文ETL程式 10
第二節 社群貼文過濾器 11
第三節 巨量灰色關聯分析演算法 12
第四章、實驗結果分析 16
第一節、實驗設計 16
第二節、評估指標 16
第三節 結果 17
表4 實驗結果 17
第四節 建議 18
第伍章 結論 18
參考文獻 20

[1]. 林大貴. (2015). 大數據巨量分析與機器學習整合開發實戰 (Vol. 4).
[2].Chou, S.-Y., Chang, Y.-H., and Shen, C.-Y. 2008. "A fuzzy simple additive weighting system under group decision-making for facility location selection with objective/subjective attributes," European Journal of Operational Research (189:1) 8/16/, pp 132-145.
[3].Deng, J. L. 1989. "Introduction to Grey system theory," J. Grey Syst. (1:1), pp 1-24.
[4].FaceBook 2016. "如何使用主題標籤（Hashtag）？."
[5].Hsu, C.-L., and Liu, C.-M. 2015. "A Fuzzy Cloud Service Selection Using Grey Relational Analysis with objective and subjective attributes,").
[6].Hung Tien, T., Hiep Tuan, N., and Viet-Trung, T. Year. "Large-scale geographically weighted regression on Spark," 2016 Eighth International Conference on Knowledge and Systems Engineering (KSE)2016, pp. 127-132.
[7].Yunus, M., and Alsoufi, M. S. 2016. "Multi-output optimization of tribological characteristics control factors of thermally sprayed industrial ceramic coatings using hybrid Taguchi-grey relation analysis," Friction (4:3), pp 208-216.
[8].Jia, Z., Xue, C., Chen, G., Zhan, J., Zhang, L., Lin, Y., and Hofstee, P. Year. "Auto-tuning Spark big data workloads on POWER8: Prediction-based dynamic SMT threading," 2016 International Conference on Parallel Architecture and Compilation Techniques (PACT)2016, pp. 387-400.
[9]. Klir, G., and Yuan, B. 1995. Fuzzy sets and fuzzy logic, (Prentice hall New Jersey.
[10].Kelkar, A., and Kulkarni, S. Year. "Value of facebook for job search: Languishing present to a lucrative future," International Conference on Information Society (i-Society 2013)2013, pp. 222-226.
[11].Muley, A., and Bajaj, V. 2009. "Applications of fuzzy multiple attribute decision making method solving by interval numbers," a a (1:2), pp 1-2.
[12].Qu, L., Wang, Y., and Orgun, M. A. Year. "Cloud service selection based on the aggregation of user feedback and quantitative performance assessment," Services Computing (SCC), 2013 IEEE International Conference on, IEEE2013, pp. 152-159.
[13].Sallehuddin, R., Shamsuddin, S. M. H., and Hashim, S. Z. M. Year. "Application of grey relational analysis for multivariate time series," 2008 Eighth International Conference on Intelligent Systems Design and Applications, IEEE2008, pp. 432-437.
[14].Spark, A. 2016.
[15].Sobotta, N. Year. "A Systematic Literature Review on the Relation of Information Technology and Information Overload," 2016 49th Hawaii International Conference on System Sciences (HICSS)2016, pp. 858-867.
[16].Toffler, A. 1970. "Future shock."
[17]. Guan, K., Zhang, Y., and Song, P. Year. "A personalization recommendation method with time characteristics," 2016 IEEE International Conference on Information and Automation (ICIA)2016, pp. 2012-2015.
[18]. Wei, G. h., Feng, L., and Liang, M. Year. "Fuzzy optimization of water resources project scheme based on improved grey relation analysis," 2011 3rd International Conference on Computer Research and Development2011, pp. 333-336.