臺灣博碩士論文加值系統

近年來深度學習引領了的圖像辨識的高度成長，也廣泛應用在自然語言辨識與文字探勘。本研究使用遷移學習技術去分析網路評論，再透過特徵工程提取並擴增關鍵詞。此研究使用深度學習網路加載遷移學習機制後，以進行文本分類研究。此研究會將一般的深度神經網路，加入Attention Layer的深度神經網路，再透過多個深度神經網路，以及Stacking技術，融合成最終模型並進行最終偵測。偵測偏激性言論實驗結果顯示，使用Attention Layer的深度神經網路，其偵測結果可獲得F1 measure 67.69%，Auc：96.05%。結合深度神經網路與Stacking，其偵測結果可獲得F1 measure 69.96%，Auc：96.17%。
此研究其參與Kaggle自然與語言競賽，針對Quora上偏激性言論的偵測，最終結果於全球競賽排名為前16%以內，F1 measure 70.13%，與最佳獲勝成績結果71.32%，僅差距1.2%。

In recent years, deep learning technology has been highly developed in image recognition, and is also widely used in natural language recognition and word exploration. This study uses migration learning techniques to analyze online reviews and then extract and amplify keywords through feature engineering. This study uses deep learning techniques to load the migration learning mechanism and text classification study. This study will add an Attention Layer into general deep neural network, then through combining multiple deep neural networks and Stacking technology, a final model is developed as for comments detection. The experimental results of detect the extreme comments show that using the deep neural network with Attention Layer, the detection results can be 66.19% in F1 measure and Auc: 96.05%. The combined deep neural network with Stacking technology approach can obtain F1 measure 69.96% and Auc: 96.17%.
This study involved Kaggle nature language competition of extreme contents detection on Quora. The results of this study ranked within the top 16% of the global competition, F1 measure 70.13%, and the best winning result of 71.32%, with only 1.2% difference.

書名頁 i
論文口試委員審定書 ii
中文摘要 iii
英文摘要 iv
誌 謝 v
目 錄 vi
表目錄 viii
圖目錄 ix
第一章、緒論 1
第二章、文獻探討 4
2.1特徵工程 4
2.1.1 TFIDF 6
2.2機器學習相關分類技術 6
2.2.1 Liner SVM 6
2.2.2 Logistic regression 6
2.3深度學習相關分類技術 7
2.3.1 RNN 7
2.3.2 CNN 7
2.3.3遷移學習 8
2.3.4 Attention Mechanism 8
2.4 Stacking 多重模型融合技術 9
第三章、研究方法 13
3.1研究架構 13
3.2特徵工程 14
3.2.1去除標點符號 15
3.2.2數字均質化 15
3.2.3詞彙修訂 16
3.2.4擴增特徵 17
3.2.5清除辨識度低的資料 17
3.3 應用的機器學習與深度學習演算法 18
3.3.1應用機器學習演算法 18
3.3.2應用深度學習演算法 18
3.4 多重模型融合演算法 19
第四章、實驗結果 20
4.1資料描述 20
4.2預測方式 21
4.2.1單一演算法預測結果 21
4.2.2多重模型融合預測結果 22
第五章、討論 23
5.1 詞彙修訂討論 23
5.2 多重融合技術討論 24
5.3 融合遷移學習詞語庫 24
第六章、結論與未來展望 27
6.1未來展望 27
參考文獻 28

中文文獻
[1]. JustGoodThemes, 2017, 特徵工程特徵工程中常見的方法, https://vinta.ws/code/feature-engineering.html
[2]. Lqfarmer, (2018) 模型汇总24 - 深度学习中Attention Mechanism详细介绍：原理、分类及应用, 知呼, https://zhuanlan.zhihu.com/p/31547842, Data Accessed 2019/03/02
[3]. TRANSBIZ, (2016), Quora：美國知識問答網站，內容行銷建立品牌權威的秘境, https://transbiz.com.tw/quora-content-marketing/, Data Accessed 2019/03/18
[4]. Xin-Dong Wu、Yang-Lan Gan、Hao Wang、Xue-Gang Hu，2006，Feature Selection: Algorithms and Challenges，南昌工程學院學報 25卷2期28-34
[5]. 王韋翔(2018)，深度學習結合基因演算法在遊戲App的生命週期預測，元智大學，碩士論文。
[6]. 高翊瑋、侯藹玲、葉玲瓏、姜葉飛 (2016) ，醫療大數據概述，Journal of Data Analysis11卷5期， P1 - 11。
[7]. 陳雲濤的部落格 , 2019 , http://violin-tao.blogspot.com/2018/01/ml-ensemble.html
[8]. 黃瀚萱和陳信希(2016) ，醫療大數據及其應用，台灣醫學20卷6期， P589 - 594。
[9]. 梁春泉、張陽、劉全中，2009，基于堆叠集成的數據流分類，計算機應用研究26卷5期，P1716 - 1718。
[10]. 程式前沿，2018，
https://codertw.com/%E7%A8%8B%E5%BC%8F%E8%AA%9E%E8%A8%80/457901/
[11]. 潘浙楠, 2007, 主成份分析法在制定多變量製程能力指標上之應用研究, 品質學報14卷3期

英文文獻
[1]. Agrawal, R.K., Muchahary, F., Tripathi, M.M., 2019, Ensemble of relevance vector machines and boosted trees for electricity price forecasting, Applied Energy Vol. 250, 15 September 2019, pp. 540-548.
[2]. Alice Zheng & Amanda Casari, 2018, Book [ Feature Engineering for Machine Learning: Principles and Techniques for Data Scientists ], Publisher：O’Reilly.
[3]. Badshah, A. M., Ahmad, J., Rahim, N., & Baik, S. W. (2017, February). Speech emotion recognition from spectrograms with deep convolutional neural network. 2017 In Platform Technology and Service (PlatCon), 1-5.
[4]. Bai, X., Shi, B., Zhang, C., Cai, X., & Qi, L. (2017). Text/non-text image classification in the wild with convolutional neural network, Pattern Recognition, 66, pp. 437-446.
[5]. Christopher Olah (2015), Understanding LSTM Networks,
URL:http://colah.github.io/posts/2015-08-Understanding-LSTMs/, Data Accessed 2018/06/17.
[6]. Gajbhiye, A., Jaf, S., Moubayed, N.A., Bradley, S., McGough, A.S., 2019, CAM: A Combined Attention Model for Natural Language Inference, Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018, 8622057, pp. 1009-1014.
[7]. Graves, A., & Jaitly, N. (2014, January). Towards end-to-end speech recognition with recurrent neural networks. In International Conference on Machine Learning , 32, pp. 1764-1772.
[8]. Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), pp. 1735-1780.
[9]. Li, W., Ding, P., Zhang, X., Duan, C., Qiu, R., Lin, J., Shi, X., 2019, Ensemble learning methodologies to improve core power distribution abnormal detectability, Nuclear Engineering and Design 351 pp. 160-166.
[10]. Liu, G., Guo, J., 2019, Bidirectional LSTM with attention mechanism and convolutional layer for text classification, Neurocomputing 337 pp. 325-338
[11]. Hu, K., Luo, Q., Qi, K., Yang, S., Mao, J., Fu, X.,Zheng, J., Wu, H., Guo, Y., Zhu, Q., 2019, Understanding the topic evolution of scientific literatures like an evolving city: Using Google Word2Vec model and spatial autocorrelation analysis, Information Processing and Management56(4) pp. 1185-1203
[12]. Lee, D. D., & Seung, H. S. (1999). Learning the parts of objects by non-negative matrix factorization. Nature, 401(6755), 788.
[13]. Jason Yosinski, et al. (2014), How transferable are features in deep neural networks? Cornell University, https://arxiv.org/abs/1411.1792, Data Accessed 2019/03/02.
[14]. Kruger, J., 2019, Are you talking about software product lines? An analysis of developer communities, ACM International Conference Proceeding Series.
[15]. Ma, S., Chen, X., 2019, A data mining approach to predict risk of cardiovascular, AIP Conference Proceedings, 2058, 020014
[16]. Malave, N., Dhage, S.N., 2018, Sarcasm detection on twitter: User behavior approach, Advances in Intelligent Systems and Computing 910, pp. 65-76
[17]. Malik, M., Naaz, S., Ansari, I.R., 2019, Sentiment analysis of twitter data using big data tools and hadoop ecosystem, Lecture Notes in Computational Vision and Biomechanics 30, pp. 857-863.
[18]. Rekha, G., Tyagi, A.K., Krishna Reddy, V., 2020, A novel approach to solve class imbalance problem using noise filter method, Advances in Intelligent Systems and Computing 940 pp. 486-496.
[19]. Sun, Y., Wang, Y., Guo, L., Ma, Z., Jin, S., Wang, H., 2018, Researching on Multiple Machine Learning for Anomaly Detection, Journal of Physics: Conference Series 1169(1), 012002
[20]. SA，2017, https://stats.stackexchange.com/questions/18891/bagging-boosting-and-stacking-in-machine-learning
[21]. Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pp. 3104-3112.
[22]. Tenderich, A., Tenderich, B., Barton, T., Richards, S.E., 2019, What Are PWDs (People With Diabetes) Doing Online? A Netnographic Analysis, Journal of Diabetes Science and Technology 13(2), pp. 187-197
[23]. Turing, A. M., (1950) Computing Machinery and Intelligence. Mind 49, pp. 433-460.
[24]. Will Koehrsen, 2018, Towards Data Science, https://towardsdatascience.com/ feature- engineering-what-powers-machine-learning-93ab191bcc2d
[25]. Wu, Y., Ma, Y., Liu, J., Du, J., Xing, L., 2019, Self-attention convolutional neural network for improved MR image reconstruction, Information Sciences Vol 490, July 2019, pp. 317-328.
[26]. Zhang, Q., Yang, L. T., Chen, Z., & Li, P., 2018, A survey on deep learning for big data. Information Fusion, Vol. 42, pp. 146-157.
[27]. Zhang, W., Li, Y., Wang, S., 2019, Learning document representation via topic-enhanced LSTM model, Knowledge-Based Systems Vol. 174, 15 June 2019, pp. 194-204.