臺灣博碩士論文加值系統

在現今資訊爆炸的時代中，人們會想要在最短的時間從大量的文本中擷取重點資料,而如何快速的篩選出需要的資訊就是一門重要的議題，自動文本摘要(Automatic Text Summarization)是其中一種合適的選擇，而在生活中常用的中文卻在此技術上發展緩慢，本研究針對中文的自然語言處理(Natural Language Processing, NLP)開始研究並使用現有的雙向(Bidirectional)與單向(Unidirectional)循環神經網路(Recurrent Neural Network)組成序列到序列(Sequence to Sequence)架構，並搭配注意力(Attention)機制與詞向量(Word Vectors)訓練多個不同參數的抽象式自動文本摘要模型，並且搭配集束搜索(Beam Search)與貪婪搜索(Greedy Search)在選字階段做比較，再使用召回率導向的摘要評估(Recall-Oriented Understudy for Gisting Evaluation, ROUGE)來比對自動文本摘要與人工摘要的分數，而實驗中比較出分數最高，摘要能力最好的是向量維度為500維，搭配512層雙向循環神經網路並使用集束寬度為2的集束搜索法所組合之模型,而在實驗中也發現新聞格式的中文文本,在選用貪婪搜索與集束寬度為2的集束搜索會有較好的平均分數與摘要能力。

In the age of information communication technology, finding out the required information is a significant research topic and Automatic Text Summarization is one of the choices. Currently Chinese is one of the most speaking languages in the world, thus it is required to study this language on this technology. The purpose of the study is to explore Natural Language Processing (NLP) regarding Chinese language employing the Bidirectional and Unidirectional Recurrent Neural Network to form a Sequence to Sequence structure, also with the Attention mechanism and Word2Vec to train multiple abstract automatic text summary models. Additionally, we compared the word selection stage between Beam Search and Greedy Search, and Recall-Oriented Understudy was used for Gisting Evaluation (ROUGE). Finally, we compared the scores of the Automatic Text Summarization and the manual summary to find out the most suitable model or combination of the models.
Word2Vec 500 dimensions and 512-layer Bidirectional Recurrent Neural Network and Beam Search with the beam width of 2 scored the highest and the most suitable to explain. Additionally, Greedy Search and Beam Search with the beam width of 2 were found to have a better average score in news format of the Chinese text.

第一章、緒論 1
1.1 研究背景 1
1.2 研究動機與目的 3
1.3 研究架構 4
第二章、文獻探討 5
2.1 自動文本摘要: 5
2.2 中文自然語言處理 6
2.2.1分詞方式: 7
2.2.2詞性標註: 7
2.2.3標點符號: 8
2.2.4詞彙粒度: 8
2.2.5句法結構: 10
2.2.6指代消解處理: 10
2.2.7詞彙間關聯關係 11
2.3 深度學習 11
2.3.1激勵函數 11
2.3.2損失函數 13
2.3.3反向傳播法 14
2.4 循環神經網路 19
2.5 雙向循環神經網路 22
2.6 長短期記憶模型 23
2.7 序列到序列 28
2.8 注意力機制 29
2.9 搜索法 31
第三章、研究方法 33
3.1資料集介紹: 33
3.2 研究環境: 34
3.3 研究方法架構: 34
3.4 資料前處理: 36
3.5 評估工具: 36
3.6 實驗設計: 38
第四章、資料分析與結果 41
4.1 實驗一:要找出適合的epoch。 41
4.2 實驗二:找出適合的詞向量維度: 42
4.3 實驗三:找出最合適的序列到序列模型組合 44
4.4 實驗四:找出最合適的搜索方式 45

第五章、結論與未來工作 47
5.1 研究結論 47
5.2 未來研究方向 48
第六章、參考文獻 49

Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078.
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555.
Elman, J. L. (1990). Finding structure in time. Cognitive science, 14(2), 179-211.
Gong, Y., & Liu, X. (2001). Generic text summarization using relevance measure and latent semantic analysis. Paper presented at the Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval.
Greff, K., Srivastava, R. K., Koutník, J., Steunebrink, B. R., & Schmidhuber, J. (2016). LSTM: A search space odyssey. IEEE transactions on neural networks and learning systems, 28(10), 2222-2232.
Hinton, G. E., Sejnowski, T. J., & Poggio, T. A. (1999). Unsupervised learning: foundations of neural computation: MIT press.
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735-1780.
Hu, B., Chen, Q., & Zhu, F. (2015). Lcsts: A large scale chinese short text summarization dataset. arXiv preprint arXiv:1506.05865.
Jozefowicz, R., Zaremba, W., & Sutskever, I. (2015). An empirical exploration of recurrent network architectures. Paper presented at the International Conference on Machine Learning.
Junyi, S. (Retrieved 2019). Jieba.[online]. Retrieved from https://github.com/fxsjy/jieba.
Kupiec, J., Pedersen, J., & Chen, F. (1999). A trainable document summarizer. Advances in Automatic Summarization, 55-60.
LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. nature, 521(7553), 436.
Li, P., Lam, W., Bing, L., & Wang, Z. (2017). Deep recurrent generative decoder for abstractive text summarization. arXiv preprint arXiv:1708.00625.
Lin, C.-Y. (2004). Rouge: A package for automatic evaluation of summaries. Paper presented at the Text summarization branches out.
Lin, S.-H., & Chen, B. (2009). Improved speech summarization with multiple-hypothesis representations and Kullback-Leibler divergence measures. Paper presented at the Tenth Annual Conference of the International Speech Communication Association.
Luong, M.-T., Pham, H., & Manning, C. D. (2015). Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025.
Mani, I. (1999). Advances in automatic text summarization: MIT press.
Mazur, M. (2015). A Step by Step Backpropagation Example. Retrieved from https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/.
Mihalcea, R., & Tarau, P. (2004). Textrank: Bringing order into text. Paper presented at the Proceedings of the 2004 conference on empirical methods in natural language processing.
Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. Paper presented at the Advances in neural information processing systems.
Olah, C. (2015). Understanding LSTM Networks. Retrieved from http://colah.github.io/posts/2015-08-Understanding-LSTMs/.
Paulus, R. (2018). Deep Reinforced Model for Abstractive Summarization. In: Google Patents.
Paulus, R., Xiong, C., & Socher, R. (2017). A deep reinforced model for abstractive summarization. arXiv preprint arXiv:1705.04304.
Ranzato, M. A., Chopra, S., Auli, M., & Zaremba, W. (2015). Sequence level training with recurrent neural networks. arXiv preprint arXiv:1511.06732.
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1988). Learning representations by back-propagating errors. Cognitive modeling, 5(3), 1.
Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural networks, 61, 85-117.
Schuster, M., & Paliwal, K. K. (1997). Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11), 2673-2681.
See, A., Liu, P. J., & Manning, C. D. (2017). Get to the point: Summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368.
Shen, D., Sun, J.-T., Li, H., Yang, Q., & Chen, Z. (2007). Document summarization using conditional random fields. Paper presented at the IJCAI.
Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. Paper presented at the Advances in neural information processing systems.
Wang, H., & Yeung, D.-Y. (2016). Towards bayesian deep learning: A survey. arXiv preprint arXiv:1604.01662.
Weng, C., Cui, J., Wang, G., Wang, J., Yu, C., Su, D., & Yu, D. (2018). Improving Attention Based Sequence-to-Sequence Models for End-to-End English Conversational Speech Recognition. Paper presented at the Interspeech.
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2018). Recent trends in deep learning based natural language processing. ieee Computational intelligenCe magazine, 13(3), 55-75.
林婷嫻 , 張. (2018). 中研院-斷開中文的鎖鍊！自然語言處,馬偉雲專訪. Retrieved from http://research.sinica.edu.tw/nlp-natural-language-processing-chinese-knowledge-information/.
陳運文. (2019). 在NLP領域中文對比英文的難點分析(達觀數據陳運文）. Retrieved from http://www.52nlp.cn/11458-2.