臺灣博碩士論文加值系統

近年來深度學習蓬勃發展，讓機器學習方式學習到人類的行為模式，而在深度學習薰陶下，機器人對話又慢慢的浮出水面，在傳統機器人對話中以自己建立的資料庫來打造機器人，必須依賴於強大問答資料庫一個問題一個答案的方式進行，往往產生出資料所花費時間與理想超出許多，在面對客戶會想知道客戶問什麼樣的問題，產生出問題成為了重要關鍵，本研究利用seq2seq模型進行訓練打造一個產生問句模型，使機器人可以學習人造句模式產生問句，以利於發現客戶所會提的問題，本研究使用八種模式進行模型訓練來產生問句，研究測試在多領域與單領域中字元級別seq2seq達到90%及85%準確率，與上述條件相同下seq2seq加入注意力機，單領域seq2seq則為80%準確率多領域為90%準確率，在多領域詞級別seq2seq加入注意力機制可達86%準確率，單領域下則80%準確率。

In recent years, deep learning has flourished, allowing machine learning to learn human behavior patterns. Under the deep learning, robot dialogue has slowly surfaced, and in the traditional robot dialogue, the robot is built with its own database. It must rely on the powerful question and answer database to answer one question and one answer. It often takes a lot of time and ideals to generate the information. In the face of the customer, they will want to know what kind of problem the customer asks, and the problem becomes an important key. This study uses the seq2seq model to train to create a question-making model, so that the robot can learn the artificial sentence pattern to generate questions, in order to find out the problems that the customer will ask. This study uses eight models to carry out model training to generate questions. The research test achieved 90% and 85% accuracy in the character level seq2seq in multi-domain and single-field. Under the above conditions, seq2seq was added to the attention machine, and the single-field seq2seq was 80% accurate. The multi-field was 90% accurate. The multi-domain word level seq2seq adds 86% accuracy to the attention mechanism and 80% accuracy in the single field.

摘要 i
Abstract ii
目錄 iii
表目錄 v
圖目錄 vi
壹、緒論 1
一、 研究動機 1
二、 研究目的 2
貳、文獻探討 3
一、 Word Embedding 3
二、Word2Vec 3
三、遞歸神經網路（Recurrent neural network, RNN） 5
四、 長短期記憶網路（Long Short Term Memory Network, LSTM） 7
五、seq2seq模型 9
六、注意力機制（Attention） 10
柒、批量標準化（Batch Normalization） 11
參、實驗方法 12
一、 單領域與多領域 13
二、 Char-level字元級and word-level詞級 13
三、 word2vec 13
四、 Bi-LSTM 雙向長短期記憶, Attention注意力機制, BN批量標準化 14
肆、實驗結果 15
一、 character-level字元級+BI-LSTM雙向長短期記憶 15
二、 word2vec+BI-LSTM雙向長短期記憶 16
三、 character-level字元級+BI-LSTM雙向長短記憶+Attention注意力機制 16
四、 word-level詞級+ BI-LSTM雙向長短期記憶+ Attention注意力機制 18
五、 輸出問句結果 19
5.1 character-level字元級+ BI-LSTM雙向長短期記憶+單領域 19
5.2 character-level詞級+ BI-LSTM雙向長短期記憶+多領域 21
5.3 word2vec +BI-LSTM雙向長短期記憶+單領域 22
5.4 word2vec +BI-LSTM雙向長短期記憶+多領域 23
5.5 character-level字元級+BI-LSTM雙向長短期記憶+Attention注意力機制+單領域 25
5.6 character-level字元級+BI-LSTM雙向長短期記憶 + Attention注意力機制+多領域 26
5.7 word-level詞級+ BI-LSTM雙向長短期記憶+ Attention注意力機制+單領域 27
5.8 word-level詞級+BI-LSTM雙向長短期記憶+Attention注意力機制+多領域 29
伍、討論與未來展望 31
參考文獻 33

Bellman, R. (2013). Dynamic programming: Courier Corporation.
Donahue, J., Anne Hendricks, L., Guadarrama, S., Rohrbach, M., Venugopalan, S., Saenko, K., & Darrell, T. (2015). Long-term recurrent convolutional networks for visual recognition and description. Paper presented at the Proceedings of the IEEE conference on computer vision and pattern recognition.
Elman, J. L. (1990). Finding structure in time. 14(2), 179-211.
Graves, A., Mohamed, A.-r., & Hinton, G. (2013). Speech recognition with deep recurrent neural networks. Paper presented at the 2013 IEEE international conference on acoustics, speech and signal processing.
Hinton, G. E., McClelland, J. L., & Rumelhart, D. E. (1984). Distributed representations: Carnegie-Mellon University Pittsburgh, PA.
Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. 79(8), 2554-2558.
Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift.
Jordan, M. (1986). Attractor dynamics and parallelism in a connectionist sequential machine. Paper presented at the Proc. of the Eighth Annual Conference of the Cognitive Science Society (Erlbaum, Hillsdale, NJ), 1986.
Lilleberg, J., Zhu, Y., & Zhang, Y. (2015). Support vector machines and word2vec for text classification with semantic features. Paper presented at the 2015 IEEE 14th International Conference on Cognitive Informatics & Cognitive Computing (ICCI* CC).
Liu, X., Xia, T., Wang, J., Yang, Y., Zhou, F., & Lin, Y. (2016). Fully convolutional attention networks for fine-grained recognition.
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space.
Mikolov, T., Yih, W.-t., & Zweig, G. (2013). Linguistic regularities in continuous space word representations. Paper presented at the Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.
Mnih, V., Heess, N., & Graves, A. (2014). Recurrent models of visual attention. Paper presented at the Advances in neural information processing systems.
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. Paper presented at the Proceedings of the IEEE conference on computer vision and pattern recognition.
Schuster, M., & Paliwal, K. (1997). Bidirectional recurrent neural networks. 45(11), 2673-2681.
Stroh, E., Student, S., & Mathur, P. (2016). Question answering using deep learning. In.
Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. Paper presented at the Advances in neural information processing systems.
Venugopalan, S., Rohrbach, M., Donahue, J., Mooney, R., Darrell, T., & Saenko, K. (2015). Sequence to sequence-video to text. Paper presented at the Proceedings of the IEEE international conference on computer vision.
Colah(2015) Understanding LSTM Networks .Available from http://colah.github.io/posts/2015-08-Understanding-LSTMs/
Googleblog(2015) Computer, respond to this email. Available from https://ai.googleblog.com/2015/11/computer-respond-to-this-email.html
YJango(民107) YJango的循环神经网络——介绍.民國107年12月20號 取自: https://zhuanlan.zhihu.com/p/24720659