臺灣博碩士論文加值系統

本論文主軸為改進任務導向(Task-Oriented)對話系統的評分機制，其目標為提升對話系統的使用情境，包括自然語言處理、拆解意圖、對話狀態追蹤等模組，使對話系統能更精準的決策最佳的回應句子，在評分機制上能更趨近於教師的人工評分。對話系統是以口語的方式與機器溝通，分為任務導向及非任務導向兩種，而本論文針對「任務導向」進行探討。
本研究以任務導向型的對話系統作為電腦輔助教學的媒介，輔助教師讓學習者增強他們的英語口說能力，模擬具有目標導向特性的對話情境，學習者反覆的練習以提升口說能力及語言流暢度來建立自信心。
基於樣板式模型建置任務導向對話系統來輔助語言學習，提供學習者以遊戲的方式進行，教材由專業的英語教師建構涵蓋日常生活用語的對話劇本，此方向之特點為將語言模型導入對話管理者模組，使系統能依據學習者輸入的語句給予不同的回應及對話評分，同時提出三種不同的模型來預測分數，並驗證是否能校準於教師所產生的人工評分。
研究對象為大學部應用外國語文學系的學生，採用Willis提出的三段式教學步驟，於教學實驗前讓學習者熟悉系統以及任務描述的事項，同時教師在此階段進行任務解說，實驗過程中由學習者自行發揮口說能力，此階段教師則扮演監督者的角色而不干預測驗，待任務結束後由系統根據對答狀況給予評分，讓教師日後能針對不同學習者的狀況給予適當的教材，達到語言加強階段的效益。
本研究提出三種不同的評分機制，以均方根誤差及平均絕對誤差來衡量，平均絕對誤差最少的為系統評分，換言之，系統評分最趨近於人工評分，但均方根誤差最少的為機器學習預測評分。本研究貢獻為建置任務導向對話系統輔助語言學習，擺脫受時間及地點的限制，提出一個系統框架，依照此框架能發展其他語言學習的系統。

This study improves the scoring mechanism of the Task-Oriented dialogue system, the goal is to improve the context of the dialogue system, including natural language processing, natural language understanding, and dialog state tracker module. Make the dialogue system more accurate find best response sentences, experiment with a mechanism similar to the teacher scoring mechanism. dialogue system can communicate with the machine and can be divided into Task-Oriented and Non-Task-Oriented, this study explores task orientation.
Previous research has shown that Computer Assisted Instruction is effective for language learning, can assist teachers help learners enhance English speaking ability, simulate conversation situation, after the learners continuous to practice, improve conversational fluency and confidence.
This study developed a system of Task-Oriented dialogue assisted language learning, Rule-based model design, provides learners with a game-based learning experience. Dialogue tree is designed by a professional English teacher, contain food, transportation, etc., scripts. Learners can easily talk to the system, because this study, imported a language model into dialogue management module. When the learner say different sentences will get different responses and scoring. This study proposes three different ways to predict scores, and evaluation is similar to teacher scoring.
The research target is a college student in Chung Yuan Christian University, Department of Applied Linguistics and Language Studies, using the three-stage teaching steps proposed by Willis in 1996. Make the learner familiar with the system and task description before the teaching experiment, the teacher is the observer during the experiment, after the task is over, the learner will get a rating from the system and the teacher.
This study proposes three different ways to scoring mechanism, evaluation method includes Root Mean Squared Error and Mean Absolute Error. minimum Mean Absolute Error is system scoring, but minimum Root Mean Square Error is the machine learning prediction scores. The contribution of this research is to developed a Task-Oriented dialogue assisted language learning system, it is not limited by time and place, according to this framework, can develop systems for other language learning.

目 錄
摘 要 ..................................................................................................................................................... I
Abstract................................................................................................................................................. II
目 錄 ..................................................................................................................................................III
圖目錄 .............................................................................................................................................. V
表目錄 ............................................................................................................................................ VI
第一章 緒論 ....................................................................................................................................... 1
1.1 研究背景與動機.......................................................................................................................... 1
1.2 研究問題 ..................................................................................................................................... 3
1.3 研究目的 ...................................................................................................................................... 3
1.4 論文架構 ..................................................................................................................................... 4
第二章 文獻探討 ............................................................................................................................... 5
2.1 任務導向教學法(Task-Based Learning)..................................................................................... 5
2.2 電腦輔助教學(Computer-Aided Instruction).............................................................................11
2.3 自然語言處理(Natural Language Processing)...........................................................................13
2.4 對話模型評估(Evaluation Metrics for Dialogue Response)......................................................13
2.5 類神經網路(Neural Network)....................................................................................................14
第三章 研究設計與系統架構 ..........................................................................................................18
3.1 簡介 .............................................................................................................................................18
3.2 研究設計 .....................................................................................................................................19
3.3 研究流程 .....................................................................................................................................21
3.4 系統架構與設計.........................................................................................................................22
3.5 教學活動設計 .............................................................................................................................23
3.6 系統模組概觀 .............................................................................................................................28
3.7 訓練語料庫 ................................................................................................................................32

第四章 實驗設計與結果 ..................................................................................................................33
4.1 實驗流程 ....................................................................................................................................33
4.2 實驗工具 ....................................................................................................................................38
4.3 系統成果與展示..........................................................................................................................39
4.4 系統驗證 ....................................................................................................................................48
第五章 結論與建議 ..........................................................................................................................53
5.1 研究貢獻 .....................................................................................................................................53
5.2 研究限制與建議.........................................................................................................................54
5.3 未來研究 ....................................................................................................................................55
參考文獻................................................................................................................................................56

圖目錄
圖 2-1 三段式教學步驟 ........................................................................................................... 6
圖 2-2 任務導向對話系統流程，以點餐為例 ..................................................................... 10
圖 2-3 類神經網路示意圖 ..................................................................................................... 14
圖 2-4 遞迴式類神經網路架構圖 .......................................................................................... 15
圖 2-5 Word2vec 詞向量示意圖 ............................................................................................. 16
圖 2-6 強化學習架構圖 ......................................................................................................... 17
圖 3-1 研究流程圖 ................................................................................................................. 21
圖 3-2 系統架構圖 ................................................................................................................. 22
圖 3-3 樹狀對話劇本集範例，以餐廳點餐為例 ................................................................. 25
圖 3-4 系統頁面 ..................................................................................................................... 27
圖 3-5 任務導向對話系統流程 ............................................................................................. 28
圖 3-6 樣板式模型關鍵字提取 ............................................................................................. 29
圖 3-7 例外處理示意圖 ......................................................................................................... 30
圖 4-1 任務導向對話系統實驗設計 ..................................................................................... 33
圖 4-2 任務主題 Transportation - Buy Train Tickets ............................................................ 35
圖 4-3 任務主題 Accommodation - Hotel Reservation ......................................................... 36
圖 4-4 任務主題 Restaurant - Ordering in Restaurants ......................................................... 37
圖 4-5 註冊頁面 ..................................................................................................................... 39
圖 4-6 任務主題列表 ............................................................................................................. 40
圖 4-7 任務對話場景 ............................................................................................................. 41
圖 4-8 任務執行紀錄回放 ..................................................................................................... 42
圖 4-9 任務主題管理列表 ..................................................................................................... 43
圖 4-10 新增或編輯對話樹 ................................................................................................... 44
圖 4-11 綁定對話樹與 NPC .................................................................................................. 45
圖 4-12 瀏覽學習狀況 ........................................................................................................... 46
圖 4-13 人工評分 ................................................................................................................... 47
圖 4-14 系統驗證流程 ........................................................................................................... 48
圖 4-15 系統評分與機器學習評分相較於人工評分 ........................................................... 50


表目錄
表 2-1 過往的任務導向教學法與對話系統(備註: 技術方面) ............................................. 7
表 2-2 任務與練習區別表 ....................................................................................................... 8
表 2-3 任務導向教學法之優缺點 ........................................................................................... 9
表 2-4 傳統教學與電腦輔助教學之比較 ............................................................................. 11
表 3-1 評分指標 ..................................................................................................................... 24
表 3-2 對話樹與學習者語句相似度對照表 ......................................................................... 32
表 4-1 任務主題規劃表 ......................................................................................................... 34
表 4-2 實驗設定參數表 ......................................................................................................... 49
表 4-3 實驗使用指標比較表 ................................................................................................. 49
表 4-4 模型評估結果統計表 ................................................................................................. 50
表 4-5 實驗分析結果表 ......................................................................................................... 52

中文參考文獻
朱育民(2005)。應用語音技術建置一個國小英語教學之學習護照系統。國立中央大學網路學習科技研究所，桃園市。
吳彥儂(2016)。運用任務型語言教學法提升國民小學四年級學生英語口說能力之行動究。國立臺中教育大學教師專業碩士學位學程，台中市。
李致緯(2018)。任務導向及非任務導向對話系統之改進：以華語教學系統與聊天機器為例。國立臺灣大學電信工程學研究所，台北市。
許聞廉(2007)。行動學習環境下任務導向式語言教學之成效。國立清華大學資訊系統應用研究所，新竹市。
陳文典(2003)。自然與生活科技學習領域主題式教學活動設計。國立教育研究院籌備處，台北市。
陳冠瑜(2015)。利用語意分析模型分析谷歌部落格搜尋引擎效能。國立東華大學資訊理碩士學位學程，花蓮縣。
黃翊慈(2015)。語音辨識之電腦輔助教學軟體－以任務遊戲為例。國立臺北教育大學位科技設計學系，台北市。
葉蘭思(2016)。以任務導向教學提升國小五年級學生英語口語能力之行動研究。國立屏東大學教育學系碩士班，屏東縣。
劉秉鈞(2016)。人工智慧於模擬人類對話系統設計。逢甲大學自動控制工程學系，台中市。
簡晨卉(2013)。數位遊戲式學習在城鄉國小數學加減法學習成效之研究。國立臺中教 大學，台中市。

網頁參考文獻
廖曉青(2002)。任務與練習的區別。取自https://reurl.cc/32myl
 
英文參考文獻
Basquille, A. (2004). Task Based Learning. Retrieved July, 30, 2012.
Bohus, D., & Rudnicky, A. I. (2003). RavenClaw: Dialog management using hierarchical task decomposition and an expectation agenda. In Eighth European Conference on Speech Communication and Technology.
Chen, C. T. (2008). THE EFFECTIVENESS OF INCORPORATING THE INTERNET TO IMPROVE (Doctoral dissertation, University of Louisville).
Chen, C. T., Chang, M., Wu, K. H., & Yu, P. S. Web-based Conversation Quest for Enhancing English Speaking Skills. In Workshop Proceedings of the 26th International Conference on Computers in Education (pp. 652-660).
Ehsani, F., & Knodt, E. (1998). Speech technology in computer-aided language learning: Strengths and limitations of a new CALL paradigm. Language Learning & Technology, 2(1), 54-73.
Gee, J. P. (2003). What video games have to teach us about learning and literacy. Computers in Entertainment (CIE), 1(1), 20-20.
Hauglie, J. W. (2000). Language at work: computer-mediated communication and the adult second language learner. Dissertation Abstracts International, A: The Humanities and Social Sciences, 60(9), 3295-A.
Henderson, M., Thomson, B., & Young, S. (2014). Word-based dialog state tracking with recurrent neural networks. In Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL) (pp. 292-299).
Lowe, R., Noseworthy, M., Serban, I. V., Angelard-Gontier, N., Bengio, Y., & Pineau, J. (2017). Towards an automatic turing test: Learning to evaluate dialogue responses. arXiv preprint arXiv:1708.07149.
Morton, H., & Jack, M. A. (2005). Scenario-based spoken interaction with virtual agents. Computer Assisted Language Learning, 18(3), 171-191.
Peng Zhang, Qingyun Liu, Wei Yang. (2016). Smart Similarity Search Based on Simhash over Compressed Data in Cloud Computing, ACSAC2016.
Raux, A., & Eskenazi, M. (2004). Using task-oriented spoken dialogue systems for language learning: potential, practical applications and challenges. In InSTIL/ICALL Symposium 2004.
 
Raux, A., Langner, B., Bohus, D., Black, A. W., & Eskenazi, M. (2005). Let''s Go Public! Taking a spoken dialog system to the real world. In Ninth European conference on speech communication and technology.
Richards, J. C., & Rodgers, T. S. (2014). Approaches and methods in language teaching. Cambridge university press.
Sepp Hochreiter and Jurgen Schmidhuber. (1997). Long short-term memory. Neural computation, vol. 9, no. 8, pp. 1735–1780.
Sugathadasa, K., Ayesha, B., de Silva, N., Perera, A. S., Jayawardana, V., Lakmal, D., & Perera, M. (2017, December). Synergistic union of word2vec and lexicon for domain specific semantic similarity. In 2017 IEEE International Conference on Industrial and Information Systems (ICIIS) (pp. 1-6). IEEE.
Sugathadasa, K., Ayesha, B., de Silva, N., Perera, A. S., Jayawardana, V., Lakmal, D., & Perera, M. (2017, December). Synergistic union of word2vec and lexicon for domain specific semantic similarity. In 2017 IEEE International Conference on Industrial and Information Systems (ICIIS) (pp. 1-6). IEEE.
Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Advances in neural information processing systems (pp. 3104-3112).
Teed, R. (2004). http://serc.carleton.edu/introgeo/games/index.html. SERC, Carleton College. University Press.
Vygotsky, L. (1962). Thought and language. Cambridge, MA: MIT Press.
W. JI, Z. LI, W. CHAO and X. CHEN. (2009). A New Method for Calculating Similarity between Sentences and Application on Automatic Abstracting. Intelligent Information Management, Vol. 1 No. 1, pp. 36-42.
Wen, T. H., Vandyke, D., Mrksic, N., Gasic, M., Rojas-Barahona, L. M., Su, P. H., ... & Young, S. (2016). A network-based end-to-end trainable task-oriented dialogue system. arXiv preprint arXiv:1604.04562.
Wenqian, J., Zhoujun, L., Wenhan, C., & Xiaoming, C. H. E. N. (2009). A new method for calculating similarity between sentences and application on automatic abstracting. Intelligent Information Management, 1(01), 36.
Willis D., & Willis J. (2009). Doing task-based teaching. Oxford, England: Oxford
Willis, J. (1996). A framework for task-based learning. Pearson PTR.
Willis, J., & Willis, D. (2013). Doing Task-Based Teaching-Oxford Handbooks for Language Teachers. Oxford University Press.
Wu, H. Y. (2006). A Task-oriented Approach to the Pre-service Teacher Education in the Classroom of English Teaching Methodology (ETM). Sino-US English Teaching, 3(7), 18-22.
Young, S. (2006, December). Using POMDPs for dialog management. In 2006 IEEE Spoken Language Technology Workshop (pp. 8-13). IEEE.