為建立Retrieval-based聊天機器人，我們從聊天紀錄中來產生訓練所需的問答配對(Question-Answer Pair)，然而問答配對並非完全依序地呈現在聊天紀錄中，不同內容的問答配對可能互相穿插，而從互相穿插的訊息中分離出內容不同的會話的任務即為對話解構(conversation disentanglement)。
現有的對話解構研究大多透過計算兩個訊息的相似度來解決問題，在此論文中，我們得出透過計算訊息相似度判斷訊息是否屬於相同會話是非常困難的，但若我們透過計算相似度來預測訊息的回覆關係則可以解決此問題。此外我們指出過去研究中的模型無法處理未經訓練的訊息，而無法在實務上運用的缺陷。
此論文中，我們使用IRC與Reddit資料集進行實驗，並使用QNAP聊天記錄進行對話解構。其中人工合成的Reddit資料集提供額外的大量訓練資料，且BERT模型在此資料集上的回覆關係預測獲得良好的效能。

In order to build a retrieval-based chatbot, we generate the Question-Answer Pairs from the chat log. However, Question-Answer Pairs don’t present in order in the chat log. Question-Answer Pairs of different content may interleave with each other. The task of separating mixed messages into detached conversation are called conversation disentanglement.
Most of the existing research deal with this task by calculating the similarity of two messages. In this paper, we find that it is very difficult to predict whether two messages belong to the same conversation by calculating the similarity of the message, but if we predict the reply relation of the message by calculating the similarity, this problem can be solved. In addition, we point out that the models in the past research are unable to deal with untrained messages, and cannot be used in real world.
In this paper, we used IRC and Reddit datasets for experiments and QNAP chat log for conversation disentanglement. The synthetic Reddit dataset provides an additional amount of training data, and the BERT model gets good performance on predicting reply relationship on this dataset.

摘要........................i
Abstract........................ii
目錄........................iii
圖目錄........................iv
表目錄........................v
1.緒論........................1
2.相關研究........................5
2.1對話解構(Conversation Disentanglement)........................5
2.2句表示(Sentence Representation)........................6
3.模型架構........................8
4.實驗與分析........................11
4.1資料準備與評估........................11
4.1.1資料集........................12
4.1.2資料與訊息配對的選擇........................13
4.1.3訓練、驗證與測試資料的分割........................14
4.1.4評估方法........................15
4.1.5資料的數量........................15
4.2結果與分析........................16
4.2.1相同會話任務........................17
4.2.2回覆預測任務........................19
4.2.3切除實驗........................21
5.QNAP聊天紀錄........................23
6.結論與未來工作........................25
參考文獻........................26

[1] James Allan. 2002. Introduction to topic detection and tracking. Topic detection and tracking pages 1–16.
[2] Paul M Aoki, Margaret H Szymanski, Luke Plurkowski, James D Thornton, Allison Woodruff, and Weilie Yi. 2006. Where’s the party in multiparty?: Analyzing the structure of small-group sociable talk. In CSCW’06. ACM, pages 393–402.
[3] Yoshua Bengio, Rejean Ducharme, Pascal Vincent, and Christian Janvin. 2003. A neural probabilistic language model. JMLR, 3:1137–1155.
[4] Yann N. Dauphin, Angela Fan, Michael Auli, and David Grangier. 2016. Language modeling with gated convolutional networks. arXiv Preprint. arXiv: 1612.08083.
[5] Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv preprint arXiv:1810.04805.
[6] Micha Elsner and Eugene Charniak. 2008. You talking to me? a corpus and algorithm for conversation disentanglement. In ACL’08. ACL, pages 834–842.
[7] Micha Elsner and Eugene Charniak. 2010. Disentangling chat. Computational Linguistics 36(3):389– 409.
[8] Micha Elsner and Eugene Charniak. 2011. Disentangling chat with local coherence models. In ACLHLT’11. ACL, pages 1179–1189.
[9] Jyun-Yu Jiang, Francine Chen, Yang-Ying Chen, and Wei Wang. 2018. Learning to Disentangle Interleaved Conversational Threads with a Siamese Hierarchical Network and Similarity Ranking. In Proceedings of NAACL.
[10] Ryan Kiros, Yukun Zhu, Ruslan Salakhutdinov, Richard S. Zemel, Antonio Torralba, Raquel Urtasun, and Sanja Fidler. 2015. Skip-Thought Vectors. In Proceedings of NIPS.
[11] Lajanugen Logeswaran and Honglak Lee. 2018. An efficient framework for learning sentence representations. In ICLR.
[12] Ryan Lowe, Nissan Pow, Iulian Serban, and Joelle Pineau. 2015. The ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems. arXiv preprint arXiv:1506.08909.
[13] Elijah Mayfield, David Adamson, and Carolyn Penstein Ros´e. 2012. Hierarchical conversation structure prediction in multi-party chat. In SIGDIAL’12. ACL, pages 60–69.
[14] Shikib Mehri and Giuseppe Carenini. 2017. Chat disentanglement: Identifying semantic reply relationships with random forests and recurrent neural networks. In IJCNLP’17. volume 1, pages 615–623.
[15] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a. Efficient Estimation of Word Representations in Vector Space. In ICLR Workshop Papers.
[16] Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In EMNLP.
[17] Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In NAACL.
[18] Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding with unsupervised learning. Technical report, OpenAI.
[19] Dou Shen, Qiang Yang, Jian-Tao Sun, and Zheng Chen. 2006. Thread detection in dynamic text message streams. In SIGIR’06. ACM, pages 35–42.
[20] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, pages 6000–6010.
[21] Lidan Wang and Douglas W Oard. 2009. Contextbased message expansion for disentanglement of interleaved text conversations. In NAACL’09. ACL, pages 200–208.
[22] TensorFlow, https://www.tensorflow.org/
[23] freenode, https://freenode.net/