臺灣博碩士論文加值系統

在人工智慧領域發展快速的時代中，我們一直在尋找機器可以幫助人類的方式。在自然語言領域中，智慧對話機器人是近年來備受重視的項目。然而教導機器要如何與人類溝通，以完成一項具體任務是相當有挑戰性的。其中一個需要克服的問題在於學術研究上常預先定義出特定領域中對話的明確特性，再依照這些明確的定義去設計對話情境並蒐集對話資料集，蒐集的過程中會針對句中的意圖及目標詞彙與其類別進行標注，也會額外標出該次對話中的一些細節(如:限制條件、最終目標、是否完成對話任務…等)。這些額外標注的資訊是現實對話資料中不會出現的。使用這種方式蒐集的資料，有許多跟實際狀況不同的地方。
本研究主要是探討中文客服資料集要如何在對話系統中訓練各個模組，並舉出理想資料集與實際資料集的差異。在中文對話系統裡的自然語言理解模組，需要對自然語句做斷詞及命名實體辨識。我們改進了斷詞模組及命名實體辨識模組，使其效能提升並增加可擴充性。使用改進後的中文對話理解模組進行訓練深度學習模型，相比一般的自然語言理解模組，在中文客服對話系統的效果有所提升。

In the era of artificial intelligence field rapidly develop, we are always searching for ways in which machines can help the human.
And in the field of natural language processing, the intelligent conversational bot is the project that has been highly regarded in recent years.
However, it is full of challenges to teach machines how to communicate with humans in order to accomplish a specific task. One of the problems we need to overcome is that during academic research, we often define the explicit characteristic in dialogues of specific field beforehand, then design the situation of the dialogue and collect the dialogue data set. While collecting the data set, intentions lie in the sentences, target vocabulary and its category would be marked. Furthermore, details in the certain conversation such as condition restriction, the ultimate goal, and whether it accomplishes the conversation task or not, would also be marked. This additional information we mark would not appear in real conversation data, hence the data collected this way would be different from actual circumstances.
This study is mainly to explore how the Chinese customer service dialogue dataset trains each module in the dialogue system and list out the differences between ideal and actual datasets.
In the Chinese dialogue system, the natural language understanding module needs to do the word segment and the Named-Entity Recognition. We improve the Chinese Word Segment module and the Named-Entity Recognition so as to upgrade the efficiency and increase the expansibility. The improved Chinese language understanding module is used to enhance the training deep learning model. Compare to the ordinary natural language understanding module, the effectiveness has been improved.

摘要 I
ABSTRACT II
誌謝 III
目錄 IV
圖目錄 VI
表目錄 VII
第一章 緒論 1
1.1 研究背景 1
1.2 研究動機 1
1.3 章節概要 3
第二章 文獻探討 4
2.1 對話系統 4
2.1.1自然語言理解 5
2.1.2對話管理 7
2.1.3自然語言生成 7
2.2 深度學習 8
2.2.1 卷積神經網路(Convolutional Neural Network) 8
2.2.2 循環神經網路(Recurrent Neural Network) 9
2.2.3 長短期記憶網路(Long short-term memory) 10
2.2.4 雙向循環神經網路(Bi-directional Recurrent Neural Network) 12
第三章 實驗資料分析 14
3.1 實驗資料分析 14
第四章 研究方法 19
4.1自然語言理解 19
4.1.1資料清理模組 19
4.1.2命名實體辨識模組 20
4.1.3中文斷詞模組 21
4.2 深度神經網路模型框架 (DEEP NEURAL NETWORK, DNN) 22
4.2.1句子編碼器 (Sentence Encoder) 23
第五章 實驗方法 25
5.1參數說明 25
5.2實驗結果 26
5.2.1 命名實體辨識模型 26
5.2.2 斷詞模型 27
5.2.3 對話系統模型結果 28
5.3錯誤分析 29
第六章 結論與未來研究 31
6.1結論 31
6.2未來方向 31
附錄 33
參考文獻 35

Bordes, A., Boureau, Y.-L., & Weston, J. (2016). Learning end-to-end goal-oriented dialog. arXiv preprint arXiv:1605.07683.
Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078.
Clancey, W. J. (1979). Dialogue management for rulebased tutorials. Paper presented at the Proceedings of the 6th international joint conference on Artificial intelligence-Volume 1.
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. (2011). Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12(Aug), 2493-2537.
Dhingra, B., Li, L., Li, X., Gao, J., Chen, Y.-N., Ahmed, F., & Deng, L. (2016). Towards end-to-end reinforcement learning of dialogue agents for information access. arXiv preprint arXiv:1609.00777.
Elman, J. L. (1990). Finding structure in time. Cognitive science, 14(2), 179-211.
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735-1780.
Hu, B., Lu, Z., Li, H., & Chen, Q. (2014). Convolutional neural network architectures for matching natural language sentences. Paper presented at the Advances in neural information processing systems.
Huang, C., & Zhao, H. (2007). Chinese word segmentation: A decade review. Journal of Chinese Information Processing, 21(3), 8-20.
Jordan, M. I. (1997). Serial order: A parallel distributed processing approach. In Advances in psychology (Vol. 121, pp. 471-495): Elsevier.
Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014). A convolutional neural network for modelling sentences. arXiv preprint arXiv:1404.2188.
Kim, Y. (2014). Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882.
Kingma, D., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
LeCun, Y., & Bengio, Y. (1995). Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks, 3361(10), 1995.
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989). Backpropagation applied to handwritten zip code recognition. Neural computation, 1(4), 541-551.
Li, X., Chen, Y.-N., Li, L., Gao, J., & Celikyilmaz, A. (2017). End-to-end task-completion neural dialogue systems. arXiv preprint arXiv:1703.01008.
Ma, W.-Y., & Chen, K.-J. (2003). Introduction to CKIP Chinese word segmentation system for the first international Chinese Word Segmentation Bakeoff. Paper presented at the Proceedings of the second SIGHAN workshop on Chinese language processing-Volume 17.
Mann, W. C., & Thompson, S. A. (1987). Rhetorical structure theory: Description and construction of text structures. In Natural language generation (pp. 85-95): Springer.
McCallum, A., & Li, W. (2003). Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons. Paper presented at the Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003-Volume 4.
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010). Recurrent neural network based language model. Paper presented at the Eleventh Annual Conference of the International Speech Communication Association.
Mrkšić, N., Séaghdha, D. O., Wen, T.-H., Thomson, B., & Young, S. (2016). Neural belief tracker: Data-driven dialogue state tracking. arXiv preprint arXiv:1606.03777.
Nadeau, D., & Sekine, S. (2007). A survey of named entity recognition and classification. Lingvisticae Investigationes, 30(1), 3-26.
Ramshaw, L. A., & Marcus, M. P. (1999). Text chunking using transformation-based learning. In Natural language processing using very large corpora (pp. 157-176): Springer.
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985). Learning internal representations by error propagation. Retrieved from
Schuster, M., & Paliwal, K. K. (1997). Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11), 2673-2681.
Serban, I. V., Sordoni, A., Bengio, Y., Courville, A. C., & Pineau, J. (2016a). Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models. Paper presented at the AAAI.
Serban, I. V., Sordoni, A., Bengio, Y., Courville, A. C., & Pineau, J. (2016b). Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models. Paper presented at the AAAI.
Shi, H., Ushio, T., Endo, M., Yamagami, K., & Horii, N. (2016). A multichannel convolutional neural network for cross-language dialog state tracking. Paper presented at the Spoken Language Technology Workshop (SLT), 2016 IEEE.
Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. Paper presented at the Advances in neural information processing systems.
Traum, D. R. (1999). Speech acts for dialogue agents. In Foundations of rational agency (pp. 169-201): Springer.
Tseng, H., Chang, P., Andrew, G., Jurafsky, D., & Manning, C. (2005). A conditional random field word segmenter for sighan bakeoff 2005. Paper presented at the Proceedings of the fourth SIGHAN workshop on Chinese language Processing.
Vinyals, O., & Le, Q. (2015). A neural conversational model. arXiv preprint arXiv:1506.05869.
Wen, T.-H., Gasic, M., Mrksic, N., Su, P.-H., Vandyke, D., & Young, S. (2015). Semantically conditioned lstm-based natural language generation for spoken dialogue systems. arXiv preprint arXiv:1508.01745.
Wen, T.-H., Vandyke, D., Mrksic, N., Gasic, M., Rojas-Barahona, L. M., Su, P.-H., . . . Young, S. (2016). A network-based end-to-end trainable task-oriented dialogue system. arXiv preprint arXiv:1604.04562.
Wieger, L., & Davrout, L. (1965). Chinese characters: Their origin, etymology, history, classification and signification: A thorough study from Chinese documents: Paragon Book Reprint Corporation.
Wu, Y., Wu, W., Xing, C., Zhou, M., & Li, Z. (2016). Sequential matching network: A new architecture for multi-turn response selection in retrieval-based chatbots. arXiv preprint arXiv:1612.01627.
Xue, N. (2003). Chinese word segmentation as character tagging. International Journal of Computational Linguistics & Chinese Language Processing, Volume 8, Number 1, February 2003: Special Issue on Word Formation and Chinese Language Processing, 8(1), 29-48.
Young, S., Gašić, M., Keizer, S., Mairesse, F., Schatzmann, J., Thomson, B., & Yu, K. (2010). The hidden information state model: A practical framework for POMDP-based spoken dialogue management. Computer Speech & Language, 24(2), 150-174.
Young, S. J. (2000). Probabilistic methods in spoken–dialogue systems. Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, 358(1769), 1389-1402.
Zhang, Y., & Wallace, B. (2015). A sensitivity analysis of (and practitioners' guide to) convolutional neural networks for sentence classification. arXiv preprint arXiv:1510.03820.