臺灣博碩士論文加值系統

由於關鍵字能為一篇文章提供精簡、扼要的內容整理，自動化關鍵字擷取的方法在近二十年來已經被廣泛的研究。傳統的關鍵字擷取方法非常倚賴人工定義的特徵去優化效能，而抽取出有效的特徵是一件非常耗時的事情。
一篇最新的論文提出了一個利用深度循環神經網路模型來抽取關鍵詞片語的方法，省去了人工抽取特徵的過程。然而，該篇論文所提出來的模型有三個缺點：首先，他們沒有考慮到每個字的重要性會受到同句子內其他字的影響。一篇文章中的每個句子會呈現出不一樣的重要性，而這樣的重要性差異通常來自於句子中有某些具有重要意涵的字，因此句子的語意應該在擷取關鍵字的時候一起被考慮。接著，他們的模型沒有針對那些不曾在訓練資料中出現的單字做處理。測試文章中大約有百分之十五的單字是在訓練資料中沒有的，而這些未知字詞會降低模型的準確度，因此字符資訊應該要被考慮進來。最後，他們使用了詞向量作為輸入，因而導致一定程度上句法資訊的遺失。然而句法資訊已經被許多研究證實能夠有效的幫助抽取關鍵字詞，應該要被更有效的利用以解決這類的問題。
在這篇論文中，我們使用了兩組卷積神經網路所組成的區域語意資訊，以及搭配線性轉換的詞性向量所組成的句法資訊，來強化類神經網路關鍵字擷取模型。最後，我們使用了兩個公開資料集去驗證我們的方法，實驗結果顯示我們的方法能夠顯著的超越非監督式和監督式方法中的頂尖技術。

Keywords can provide condensed information about a document and hence automatic keyword extraction has attracted the interest of researchers in recent decades. Traditional methods largely rely on handcrafted features to optimize the performance, which is usually a time-consuming procedure.
A recent work proposes a novel deep recurrent neural network (RNN) model to extract keyphrases without manual feature engineering. However, there are three drawbacks of the previous work. To begin with, the fact that importance of a word may be influenced by other words in the same sentence is not considered in their work. It is known that each sentence possesses distinct influence on the document and such discrepancy is usually caused by some sentences contain meaningful words. Hence sentence semantics should be taken into consideration. Then, their work does not deal with these words not appearing in training data explicitly. About 15 percent of words in testing documents are not seen in training corpus, and these out-of-vocabulary (OOV) words lower the performance. For this reason, character-level information should be regarded. Finally, syntactic information is lost to some extent when they choose to use word embeddings as input. Nevertheless, syntactic information has been proved to be effective in extracting key terms, and ought to be fully utilized when it comes to such problems.
In this work, we enhance neural keyword extraction with local semantic information and syntactic information, which are composed by two convolutional neural networks and POS embeddings with linear transformation, respectively.
The experimental results show that our proposed model outperforms both unsupervised and supervised state-of-the-art baselines on two datasets significantly.

口試委員會審定書.................................. i
誌謝.......................................... ii
摘要.......................................... iii
Abstract........................................ iv
Contents........................................ vi
List of Figures..................................... viii
List of Tables ..................................... ix

1. Introduction.................................... 1
2. Related Work ................................... 4
2.1 Unsupervised Methods............................ 4
2.2 SupervisedMethods ............................. 5
2.3 CurrentState-of-the-art............................ 6
3. Problem Formulation ............................... 8
3.1 ProblemFormulation............................. 8
3.2 Loss Function................................. 9
3.3 Optimization ................................. 10
4. Methodology ................................... 11
4.1 Global Semantic Information......................... 11
4.2 Local Semantic Information ......................... 13
4.2.1 Sentence Semantics .......................... 13
4.2.2 Character-levelInformation...................... 14
4.3 Local Syntactic Information ......................... 16
4.4 Our Proposed Hybrid Model ......................... 17
5. Experimental Setup................................ 19
5.1 Datasets.................................... 19
5.2 Baseline Methods............................... 20
5.3 ImplementationDetails............................ 21
5.4 Pre-trained Word Vectors........................... 21
5.5 Evaluation Method .............................. 22
6 Experimental Results ............................... 23
6.1 Baselines Comparison ............................ 23
6.2 Training Convergence ............................ 24
6.3 Effectiveness of Local Semantic Information ............................ 26
6.4 Effectiveness of Local Syntactic Information ............................ 28
6.5 Impact of Document Length ......................... 30
7 Conclusions and Future Work........................... 31
7.1 Conclusions.................................. 31
7.2 Future Work.................................. 32
Bibliography ..................................... 33

[1] Abilhoa,W.D.anddeCastro,L.N.(2014).Akeywordextractionmethodfromtwitter messages represented as graphs. Applied Mathematics and Computation, 240:308–325.
[2] Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. CoRR, abs/1409.0473.
[3] Barskar, R., Ahmed, G. F., and Barskar, N. (2012). An approach for extracting exact answers to question answering (qa) system for english sentences. Procedia Engineer- ing, 30:1187 – 1194.
[4] Bracewell, D. B., Ren, F., and Kuriowa, S. (2005). Multilingual single document keyword extraction for information retrieval. In 2005 International Conference on Natural Language Processing and Knowledge Engineering, pages 517–522.
[5] Cheng, J. and Lapata, M. (2016). Neural summarization by extracting sentences and words. CoRR, abs/1603.07252.
[6] Dostál, M. and Jezek, K. (2011). Automatic keyphrase extraction based on NLP and statistical methods. In Proceedings of the Dateso 2011: Annual International Work- shop on DAtabases, TExts, Specifications and Objects, Pisek, Czech Republic, April 20, 2011, pages 140–145.
[7] Erkan, G. and Radev, D. R. (2004). Lexrank: Graph-based lexical centrality as salience in text summarization. J. Artif. Intell. Res. (JAIR), 22:457–479.
[8] Hochreiter, S. and Schmidhuber, J. (1997). Long short-term memory. Neural Com- putation, 9(8):1735–1780.
[9] Hulth, A. (2003). Improved automatic keyword extraction given more linguistic knowledge. In Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing, EMNLP ’03, pages 216–223, Stroudsburg, PA, USA. Associa- tion for Computational Linguistics.
[10] Kim, S. N., Medelyan, O., Kan, M., and Baldwin, T. (2010). Semeval-2010 task 5 : Automatic keyphrase extraction from scientific articles. In Proceedings of the 5th International Workshop on Semantic Evaluation, SemEval@ACL 2010, Uppsala Uni- versity, Uppsala, Sweden, July 15-16, 2010, pages 21–26.
[11] Kim, Y. (2014). Convolutional neural networks for sentence classification. In Pro- ceedings of the 2014 Conference on Empirical Methods in Natural Language Process- ing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pages 1746–1751.
[12] Lahiri, S., Choudhury, S. R., and Caragea, C. (2014). Keyword and keyphrase ex- traction using centrality measures on collocation networks. CoRR, abs/1401.6571.
[13] Liu, F., Pennell, D., Liu, F., and Liu, Y. (2009). Unsupervised approaches for au- tomatic keyword extraction using meeting transcripts. In Human Language Technolo- gies: Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, May 31 - June 5, 2009, Boulder, Colorado, USA, pages 620– 628.
[14] Mao, J., Xu, W., Yang, Y., Wang, J., and Yuille, A. L. (2014). Deep captioning with multimodal recurrent neural networks (m-rnn). CoRR, abs/1412.6632.
[15] Mihalcea, R. and Tarau, P. (2004). Textrank: Bringing order into text. In Proceed- ings of the 2004 Conference on Empirical Methods in Natural Language Processing , EMNLP 2004, A meeting of SIGDAT, a Special Interest Group of the ACL, held in conjunction with ACL 2004, 25-26 July 2004, Barcelona, Spain, pages 404–411.
[16] Mikolov, T., Karafiát, M., Burget, L., Cernocký, J., and Khudanpur, S. (2010). Re- current neural network based language model. In INTERSPEECH 2010, 11th Annual Conference of the International Speech Communication Association, Makuhari, Chiba, Japan, September 26-30, 2010, pages 1045–1048.
[17] Nguyen, T. D. and Kan, M. (2007). Keyphrase extraction in scientific publications. In Asian Digital Libraries. Looking Back 10 Years and Forging New Frontiers, 10th International Conference on Asian Digital Libraries, ICADL 2007, Hanoi, Vietnam, December 10-13, 2007, Proceedings, pages 317–326.
[18] Ohsawa, Y., Benson, N. E., and Yachida, M. (1998). Keygraph: Automatic indexing by co-occurrence graph based on building construction metaphor. In Proceedings of the IEEE Forum on Reasearch and Technology Advances in Digital Libraries, IEEE ADL ’98, Santa Barbara, California, USA, April 22-24, 1998, pages 12–18.
[19] Pay, T. (2016). Totally automated keyword extraction. In 2016 IEEE International Conference on Big Data, BigData 2016, Washington DC, USA, December 5-8, 2016, pages 3859–3863.
[20] Pennington, J., Socher, R., and Manning, C. D. (2014). Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pages 1532–1543.
[21] Salton, G. and Buckley, C. (1988). Term-weighting approaches in automatic text retrieval. Inf. Process. Manage., 24(5):513–523.
[22] Tixier, A. J., Malliaros, F. D., and Vazirgiannis, M. (2016). A graph degeneracy- based approach to keyword extraction. In Proceedings of the 2016 Conference on Em- pirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pages 1860–1870.
[23] Tsatsaronis, G., Varlamis, I., and Nørvåg, K. (2010). Semanticrank: Ranking key- words and sentences using semantic graphs. In COLING 2010, 23rd International Conference on Computational Linguistics, Proceedings of the Conference, 23-27 Au- gust 2010, Beijing, China, pages 1074–1082.
[24] Turney, P. D. (2000). Learning algorithms for keyphrase extraction. Inf. Retr., 2(4):303–336.
[25] Venugopalan, S., Xu, H., Donahue, J., Rohrbach, M., Mooney, R. J., and Saenko, K. (2014). Translating videos to natural language using deep recurrent neural networks. CoRR, abs/1412.4729.
[26] Wan, X. and Xiao, J. (2008). Single document keyphrase extraction using neighbor- hood knowledge. In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence, AAAI 2008, Chicago, Illinois, USA, July 13-17, 2008, pages 855–860.
[27] Witten, I. H., Paynter, G. W., Frank, E., Gutwin, C., and Nevill-Manning, C. G. (1999). KEA: practical automatic keyphrase extraction. CoRR, cs.DL/9902007.
[28] Zhang, C., Wang, H., Liu, Y., Wu, D., Liao, Y., and Wang, B. (2008). Automatic keyword extraction from documents using conditional random fields. Journal of Com- putational Information Systems, 4(3):1169–1180.
[29] Zhang, K., Xu, H., Tang, J., and Li, J. (2006). Keyword extraction using support vector machine. In Advances in Web-Age Information Management, 7th International Conference,WAIM 2006, Hong Kong, China, June 17-19, 2006, Proceedings, pages 85–96.
[30] Zhang, Q., Wang, Y., Gong, Y., and Huang, X. (2016). Keyphrase extraction us- ing deep recurrent neural networks on twitter. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pages 836–845.