臺灣博碩士論文加值系統

隨著巨量資料領域的蓬勃發展，對於文本資料的處理需求與日俱增，然而如何有效將大量的文本自動化分類，仍是目前自然語言處理的熱門研究。在傳統文字探勘處理時，我們常採用“詞袋”（Bag-of-Words）模型，其捨棄了文句中字詞的順序及語意上的共線性，主要只關注在字詞出現的頻率上，另外運用一種文本探勘特徵提取常用的技術──詞頻與逆向文件頻率（term frequency–inverse document frequency; TF-IDF），將其轉換成特徵向量後再進行各分類器的訓練。本論文採用預先訓練好的詞向量模型(GloVe)，結合卷積神經網路與遞迴神經網路來結讀文字上的語意以及順序，並利用20Newsgroups新聞群分類做為測試資料， 經分類結果顯示最佳模型為合併卷積神經網路與遞迴神經網路模型，該模型於測試資料中獲得86.3%的準確率，並且比起傳統的詞袋模型成長了3%左右的準確程度。

With the rapid development of huge data research area, the demand for processing textual information is increasing. Text classification is still a hot research in the field of natural language processing. In the traditional text mining process, we often use the "Bag-of-Words" model, which discards the order of the words in the sentence, mainly concerned with the frequency of occurrence of the words. TF-IDF (term frequency–inverse document frequency) is one of the techniques for feature extraction commonly used in text exploration and classification. Therefore, we combine convolutional neural network and recurrent neural network to consider the semantics and order of the words in the sentence for text classification. We apply 20Newsgroups news group as our test dataset. The performance of the result achieves an accuracy of 86.3% on the test set and improves about 3% comparing with the traditional model.

目錄
誌謝.........................................................................i
中文摘要....................................................................ii
Abstract...................................................................iii
目錄........................................................................iv
圖目錄......................................................................vi
表目錄.....................................................................vii
第一章 緒論................................................................. 1
1.1研究動機與目的................................................... 1
1.2論文架構......................................................... 2
第二章 文獻探討............................................................. 3
2.1自動文本分類(Automatic Text Classification)...................... 3
2.2停用詞(Stop Words) .............................................. 3
2.3詞袋模型(Bag-of-words model) .................................... 4
2.4TF-IDF (Term Frequency-Inverse Document)........................ 6
2.5樸素貝氏分類器(Naïve Bayes, NB).................................. 7
2.6支援向量機(Support Vector Machine,SVM)........................... 7
2.7詞向量(Word2vec) ................................................ 8
第三章 研究方法與架構...................................................... 13
3.1研究流程與架構................................................... 13
3.2Global Vectors for Word Representation(GloVe)................... 15
3.3卷積神經網路模型(Convolutional Neural network,CNN)............... 19
3.4遞歸神經網路模型(Recurrent neural network,RNN) .................. 22
3.5長短期記憶(Long Short-Term Memory,LSTM)......................... 24
第四章 實驗設計與結果...................................................... 26
4.1實驗資料........................................................ 26
4.2實驗流程........................................................ 28
4.3實驗結果........................................................ 29
第五章 結論............................................................... 30
參考文獻................................................................... 31

[1] Aaron Jaech,George Mulcaire,Shobhit Hathi,Mari Ostendorf,and Noah A. Smith, “Hierarchical Character-Word Models for Language Identification”,2016.
[2]Bengio, Yoshua, et al. ,“A neural probabilistic language model.” ,Journal of machine learning research 3.Feb ,pp.1137-1155 ,2003.
[3] Carl-Xie, “LDA(Latent Dirichlet Allocation)主題模型”,2016
https://www.analyticsvidhya.com/blog/2016/08/beginners-guide-to-topic-modeling-in-python/
[4] C.Cortes and V.Vapnik, “Support-vector networks”, Machine Learning, vol.20, no.3,pp.273–297,1995.
[5]Christopher D.Manning,Jeffrey Pennington,and Richard Socher, “GloVe: Global Vectors for Word Representation”,2014.
https://nlp.stanford.edu/projects/glove/
[6] C. Olah, “Understanding lstm networks”,2015.
http://colah.github.io/posts/2015-08-Understanding-LSTMs/.
[7]DSDN, “理解GloVe模型（Global vectors for word representation）”, 2017.
https://blog.csdn.net/coderTC/article/details/73864097
[8] R. Feldman & J.Sanger,,“The text mining handbook: advanced approaches in analyzing unstructured data.” ,2007.
[9] F. Sebastiani, “Machine learning in automated text categorization”, ACM Comput. Surv., vol. 34, no. 1, pp. 1-47, 2002.
[10]G.Salton and M.McGill, “Introduction to Modern Information Retrieval” ,Mc Graw Hill Book Company,1984.
[11] Hayes-Roth, Frederick, Donald Waterman, and Douglas Lenat., “Building expert systems.” ,1984.
[12] Lawrence, Steve, et al.,“Face recognition: A convolutional neural-network approach”, IEEE transactions on neural networks, 1997.
[13] D. D. Lewis, “Naive (Bayes) at forty: The independence assumption in
information retrieval”, European conference on machine learning. Springer,
Berlin, Heidelberg, 1998.
[14]ITREAD, “詞袋模型和詞向量模型”,2018
https://www.itread01.com/content/1545232687.html
[15]M. E. Maron, “Automatic indexing: an experimental inquiry”, Journal of the ACM (JACM), vol. 8, pp. 404-417, 1961.
[16] Mikolov, Tomas, et al. ,“Efficient estimation of word representations in vectorspace ”, arXiv preprint arXiv:1301.3781,2013.
[17] Sepp Hochreiter and Jürgen Schmidhuber, “Long short-term memory.” Neural computation,vol. 9, no. 8, pp. 1735–1780, 1997.
[18] Shivam Bansal, “Beginners Guide to Topic Modeling in Python ”,Analytics Vidhya,2016
https://www.analyticsvidhya.com/blog/2016/08/beginners-guide-to-topic-modeling-in-python/
[19] V. Vapnik, “Pattern recognition using generalized portrait method”, Automation and remote control, vol. 24, pp. 774-780, 1963.
[20]Zhang Jian, “20Newsgroups文檔分類”,2018
https://zhangjian.site/2018/07/06/nlp-20-newsgroups-document-classification/