臺灣博碩士論文加值系統

隨社會發展，心理健康和心理疾病越來越受到人們的重視。當人們情緒低落或者受心理疾病困擾時，求助於網路獲得相關知識來緩解自己精神上的痛苦已經成為一種高效且有效的方法。許多心靈健康網站建立了論壇和博客，方便民眾向其他用戶和心理醫生分享他們的心理問題。其他用戶和心理醫生能相應的給出建議來回應這些心理問題。心靈健康網站積累了大量的心理疾病描述文本，這些文本包含了豐富的表達不同心理疾病的情感標籤。自動辨識這些心理疾病標籤能使線上心靈健康服務更高效。本論文使用了一種組合型深度神經網路框架BLSTM_CNN模型來自動從文本中提取特徵，然後進行多標籤分類。BLSTM被用來針對每個詞提取整句話的特徵，CNN被用來提取一句話的局部特徵，通過組合BLSTM和CNN，针对长文本多标签分类任务，能避免LSTM和CNN的缺点，并组合它们的优点。實驗結果顯示，BLSTM_CNN模型優於單獨使用CNN，LSTM以及LSTM_CNN模型。

With the development of society, more and more attention has been paid to psychiatric health and psychiatric illness. When people are depressed or suffer from psychiatric illness, it is an efficient and effective way to seek help from the Internet to help them to alleviate their suffering. Many psychiatric health websites have established forums and blogs to help people share their psychological problems with other users and psychologists. Other users and psychologists can give advice on how to respond to these psychological problems. Psychiatric health Web site has accumulated a large number of descriptions of psychological illness, which contains a wealth of emotion labels to express different psychological illness. Automatic identification of these mental illness labels can make online mental health services more efficient. In this paper, we propose a combined depth neural network framework BLSTM_CNN model to extract features from the text automatically. BLSTM is used to extract the sentence for each word, CNN is used to extract local features in a word, through the combination of BLSTM and CNN, according to the different emotion labels, implied more useful features can be extracted. The experimental results show that the BLSTM_CNN model is better than the CNN, LSTM and LSTM_CNN models.

目錄
摘要 iii
ABSTRACT iv
誌 謝 v
目錄 vi
表目錄 viii
圖目錄 ix
第一章、 緒論 1
1.1 研究背景和意義 1
1.2 本文的主要研究內容 2
1.3 本文的組織結構 3
第二章、背景知識 4
2.1 詞嵌入技術 4
2.1.1 word2vec方法 4
2.1.2 GloVe方法 5
2.2 面向文本處理的深度學習演算法 6
2.2.1 深度神經網路 6
2.2.2 卷積神經網路 8
2.2.3 長短期記憶 9
第三章、應用BLSTM_CNN模型于文本多標籤分類 12
3.1 CNN分类器 12
3.2 LSTM分类器 13
3.5 BLSTM_CNN模型 14
第四章、實驗內容 18
4.1 實驗資料 18
4.2 實驗步驟 19
4.3 多標籤分類評估方法 20
4.4 實驗結果與分析 22
第五章、總結與展望 29
參考資料 30


表目錄
表 1、心理學文本和情感標籤示意表。 2
表 2、總文件類別分佈比例。 19
表 3、斷詞。 20
表 4、混淆矩陣。 21
表 5、wiki和wiki+domain語料庫實驗結果對比。 23
表 6、實驗文本特點示例。 26
表 7、testing data結果。 27


圖目錄
圖 1、文本上下文視窗展示圖。 5
圖 2、深度神經網路結構示意 7
圖 3、DNN中相鄰兩層輸出值之間的關係 7
圖 4、卷積神經網路示意圖 9
圖 5、LSTM網路中的一個神經元示意圖 10
圖 6、CNN分類器框架圖。 12
圖 7、LSTM模型 13
圖 8、LSTM分類器框架圖。 14
圖 9、BLSTM模型。 14
圖 10、LSTM_CNN分類器架構圖。 15
圖 11、BLSTM_CNN分類器框架圖。 16
圖 12、實驗步驟圖。 20
圖 13、wiki和wiki+domain語料庫實驗結果對比圖。 23
圖 14、BLSTM_CNN和LSTM_CNN結果對比圖。 24
圖 15、BLSTM_CNN和CNN結果對比圖。 24
圖 16、BLSTM_CNN和LSTM結果對比圖。 25

[1]Rohde,Gonnerman,Plaut.(2005).An Improved Model of Semantic Similarity Based on Lexical Co-Occurrence.
[2]Rumelhart,Hinton,Williams.(1986).Learning representations by back-propagation. Nature 323,533-536.
[3]Bengio,Ducharme,Vincent,et al.(2003). A Neural Probabilistic Language Model. Journal of Machine Learning Research 3,1137–1155.
[4]Collobert,Weston,Bottou,et al.(2011). Journal of Machine Learning Research 12,2493-2537.
[5]Mikolov,Sutskever,Chen,et al.(2013).Distributed Representations of Words and Phrases and their Compositionality.NIPS,3111-3119.
[6]Mikolov,Chen,Corrado,et al.(2013).Efficient estimation of word representations in vector space.ICLR.
[7]Pennington,Socher,D. Manning.(2014).GloVe: Global Vectors for Word Representation.EMNLP,1532-1543.
[8]Iyyer M, Manjunatha V, Boyd-Graber J, et al. Deep unordered composition rivals syntactic methods for text classification[C]. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL-15). 2015, 1: 1681-1691.
[9]Kim Y. Convolutional neural networks for sentence classification[C]. In Proceedings of the Conference on Empirical Methods on Natural Language Processing (EMNLP-14), 2014:1746-1751.
[10]Kalchbrenner N, Grefenstette E, and Blunsom P. A convolutional neural network for modelling sentences[C]. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL-14), 2014:655-665.
[11]Graves A. Supervised sequence labelling[M]. Springer Berlin Heidelberg, 2012.
[12]Irsoy O, and Cardie C. Opinion mining with deep recurrent neural networks[C]. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP-14), 2014: 720-728.
[13]Tang D, Wei F, Qin B, Zhou M, and Liu T. Building large-scale twitter-specific sentiment lexicon: A representation learning approach[C]. In Proceedings of the 25th International Conference on Computational Linguistics: Technical Papers (COLING-14), 2014: 172-182.
[14]Wang X, Liu Y, Sun C, Wang B, and Wang X. Predicting polarities of tweets by composing word embeddings with long short-term memory[C]. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL-15), 2015: 1343-1353.
[15]K. Hornik,M. Stinchcombe,and H. White.(1989). Multilayer feedforward networks are universal approximators.Neural Networks,2(5):359-366.
[16] Huang, Y., et al. "Multi-task deep neural network for multi-label learning." IEEE International Conference on Image Processing IEEE, 2013:2897-2900.
[17]Yu, Liang-Chih, and Chun-Yuan Ho. "Identifying Emotion Labels from Psychiatric Social Texts Using Independent Component Analysis." COLING. 2014.
[18] https://www.csie.ntu.edu.tw/~yvchen/f105-adl/doc/161006_WordVector1.pdf,18
[19]http://speech.ee.ntu.edu.tw/~tlkagk/courses/MLDS_2015_2/Lecture/DNN%20(v4).pdf,25
[20] https://www.csie.ntu.edu.tw/~yvchen/f105-adl/doc/161013_WordVector2.pdf,32
[21] Yu, Liang Chih, C. H. Wu, and F. L. Jang. "Psychiatric document retrieval using a discourse-aware model." Artificial Intelligence 173.7-8(2009):817-829.
[22] www.psychpark.org/
[23]Hochreiter, S, and J. Schmidhuber. "Long Short-Term Memory." Neural Computation 9.8(1997):1735-1780.
[24]Lecun, By Y. "et al, “Gradient-based learning applied to document recognition." Proceedings of the IEEE 2010.
[25]http://speech.ee.ntu.edu.tw/~tlkagk/courses/MLDS_2015_2/Lecture/RNN%20(v4).pdf,31
[26]Gustavsson, Anders, et al. "On the difficulty of training Recurrent Neural Networks." Computer Science 52.3(2012):337-345.
[27]Zhang, Min Ling, and Z. H. Zhou. "A Review on Multi-Label Learning Algorithms." IEEE Transactions on Knowledge & Data Engineering26.8(2014):1819-1837.
[28] Sareewan Dendamrongvit and Miroslav Kubat. “Undersampling Approach for Imbalanced Training Sets and Induction from Multi-label Text-Categorization Domains”. Lecture Notes in Computer Science(including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 5669 LNAI, pp. 40-52, 2010.
[29]董氏基金会断词
[30] Bo-Hsiang Tseng, Sheng-syun Shen, Hung-Yi Lee, Lin-Shan Lee. "Towards Machine Comprehension of Spoken Content: Initial TOEFL Listening Comprehension Test by Machine".the 17th Annual Conference of the International Speech Communication Association (INTERSPEECH'16), San Francisco, Sept. 2016