臺灣博碩士論文加值系統

不同程度的聽語障者，在社會化過程中，常常遭受到成長、就學、就業等層面極大的阻礙，而難於融入社會主流。由於許多語法、語用結構上的差異，使得聽人與聾人在互動溝通模式上，易產生意圖理解或文句翻譯上的問題。因此，本論文研究目的即期以簡易、人性化及快速輸入為目標，運用手語構成所需之基本手勢碼為基本輸入/檢索單元，透過具容錯的檢索機制及中文文句生成處理，研發出符合本土需求之台灣手語溝通輔助系統，改善其日常生活中的溝通表達。
本研究之研發架構，主要著重於：1).擬人化手語手勢碼檢索介面設計：根據手勢碼構詞基本精神，透過人因工程之設計與手形相似度、手形分類適用性之實驗評估，提出新型手勢碼檢索與構句之人機介面；2).具容錯之手勢碼檢索機制：考量複雜的手語操作模式及使用者的用法習慣，提出具手形/方向省略及替代之容錯式檢索機制；3).具強健性文句轉譯生成機制：承續句型樣版樹之技術，進一步改善句型樣版樹建構、句型延伸及句型不足之缺點，並結合語意關聯性資訊、類別化語言模型平滑化及動詞片語之同義詞轉移等技術，建立更具調適性之預測與自動化文句生成機制。
實驗中，選取1881常用手語詞彙及2000句對話語語句（平均語句長度為6.24字）為訓練及測試語料，所建構之句型樣版語言模型含括461條路徑（平均4.65節點）。系統效能評估部分，取手勢碼資料庫中1/5為測試語料（TOP 30為限制條件），單一手語詞彙檢索正確率為99.73%；手形省略實驗的檢索正確率為77.9%；方向/動作省略實驗的檢索正確率為97.07%；個案之語句生成實驗部分，分訓練、調適、評估三個時期的教育訓練，平均按鍵次數及平均操作時間皆呈顯著的改善成效；構句成功率隨著語句長度增加，皆可達八成以上的正確率。本研究所提之新型手勢碼預測檢索介面及具容錯檢索機制確實可增進手語使用者與聽人溝通的效能，且易於移植本系統於可攜式個人化行動裝置，具體達成手語轉語音溝通的目標。

People with hearing/speech impairments usually have communication problems in daily activities, education and vocation leading to incapable of getting into the mainstream of society. These dysfunctions often affect and limit the language learning and expression seriously. Presently, there are few of Augmentative and Alternative Communication (AAC) technology and devices as well as the associated education-training system available in Taiwan. Besides, Taiwanese Sign Language (TSL) and written Chinese have several structural differences in linguistics. Deaf students usually make ill-formed sentences from the viewpoint of written Chinese.
The purpose of this thesis is to develop an innovative TSL AAC system to provide communication aids in daily activities and language learning. More specifically, the study focuses on: 1) developing an effective TSL virtual keyboard for more intuitional selection input, 2) developing an error tolerant sign cue retrieval mechanism for word prediction and 3) integrating the predictive sentence template language model (PST) with path branching between equivalence classes for robust sentence generation.
In order to evaluate the performance of our approach, 1881 frequently used signs and 2000 Chinese sentences, in which the mean length of utterance is 6.24 words, were selected as the training and testing database. The trained PST language model includes 461 sentence templates. The retrieval enhancement using word prediction, hand-shape deletion and movement deletion achieved 99.73%, 77.9% and 97.07%, respectively. For the assessment of practical communication aid, 8 profoundly deaf students were asked to conduct the experiments. After training, adaptation and evaluation phases, the accuracy of sentence generation achieved 80%. This proposed system aims to improve speech communication ability and activities of daily life for communication-impaired people.

中文摘要 Ⅳ
英文摘要 Ⅵ
目錄 Ⅶ
圖表目錄 Ⅰ

第一章 序論 1
1.1 研究背景 1
1.2 文獻回顧 3
1.3 研究動機與目的 7
1.4 研究方法 8
1.5 章節概要 9
第二章 手語手勢碼編碼與介面設計 10
2.1 手語構成要素 10
2.2 手語手勢碼標註 12
2.3 介面分析與設計 14
2.3.1 手形(DEZ)分類 14
2.3.2 位置(TAB)及虛擬人像介面 19
第三章 手語手勢碼檢索 22
3.1 手勢碼檢索概念 22
3.2 手勢碼輸入問題之探討 23
3.3 手勢碼檢索架構 24
3.4 具容錯檢索機制之建立 26
3.4.1 手勢碼序列轉換 26
3.4.2 手勢碼序列相似度計算 26
3.4.3 手形錯誤修復 30
第四章 自動化文句生成 33
4.1 手語轉譯及文句生成的困難點 34
4.2 自動化文句生成處理 34
4.2.1 句型樣版樹(Predictive Sentence Template Language Model) 34
4.2.2 語言模型 36
4.2.3 語意合法性考量 38
4.2.4 文句生成架構 39
4.3 句型樣版樹改良 40
4.3.1 最佳句型樣版篩選 40
4.3.2 句型樣版延伸 41
4.3.3 句型樣版延伸整體架構 42
第五章 實驗結果與討論 46
5.1 手勢碼之涵蓋率分析 46
5.2 手勢碼檢索組合之混淆度分析 49
5.3 對話語料分析 51
5.4 系統效能評估實驗 53
5.5 個案實測評量 55
5.5.1 個案選取 55
5.5.2 階段性評量 55
5.5.3 評量結果與討論 56
第六章 結論與未來研究方向 59
6.1 結論 59
6.2 未來研究方向 59

參考文獻 61

[1].J.G. Webster, et. al., “Electronic Devices for Rehabilitation”, John Wiley & Sons, Inc., 1985.
[2].AOTA: “Assistive technology for Persons with Disabilities: the role of occupational therapy, pp. 127-132, 1991.
[3].林寶貴，語言障礙者就學、就業之科技支援，台灣師範大學特殊教育學系，1997。
[4].Joe Reichle: “Implementing Augmentative and Alternative Communication: strategies for learners with severe disabilities. pp 1-2, 1991. Paul H.Bookes Publishing Co.
[5].陳勝良，語言溝通障礙者數位溝通輔具系統之研發，國立成功大學醫學工程研究所碩士論文，民90年。
[6].David R. Be and Pat Mirendan: “Augmentative and Alternative Communication”.pp 159-172, 1992. Paul H.Bookes Publishing Co.
[7].Hunnicut, S., ”Word Prediction: Exploring the Use of Semantic information”, Augmentative and Alternative Communication, Vol. 6, No. 2., 1990.
[8].Demasco, P. and McCoy, K. F.: Generating Text from Compressed Input: An Intelligent Interface for People with Severe Motor Impairments. Communication of the ACM, Vol. 35. pp. 68-78, 1992.
[9].Simpson, R. C. and Koester, H. H.: Adaptive One-Switch Row-Column Scanning. IEEE Transaction on Rehabilitation Engineering, Vol. 7, pp. 464-473, 1999.
[10].郭啟祥，應用自然語言處理於台灣手語轉語音系統之研究，民國88年。
[11].Chung-Hsien Wu, Yu-Hsien Chiu, and Kung-Wei Cheng, ”Multi-Modal Sign Icon Retrieval for Augmentative Communication,” to appear in Lecture Notes in Computer Science Series, Springer-Verlag, 2001.
[12].史文漢，台灣手語語言學概論，台北市政府勞工局手語翻譯員培訓班教材，民國89年。
[13].史文漢、丁立芬，手能生橋，中華民國聾人協會，第十一版，民88年。
[14].交通大學手語社，http://www.cc.nctu.edu.tw/~signlang/.
[15].Valli, Clayton and Lucas, Ceil.: Linguistics of American Sign Language: An Introduction. Gallaudet University Press, Washington, D.C., 1996.
[16].C. D. Manning and H. Schutze, “Foundations of statistical natural language processing”, The MIT Press, 1999.
[17].Xuedong Huang, Alex Acero, and H. W. Hon, “Spoken Language Processing: A Guide to Theory, Algorithm, and System Development,” Prentice Hall Press, New Jersey, 2001.
[18].B. Y. Ricardo and R. N. Berthier, “Modern Information Retrieval,” ACM Press, 1999.
[19].Jiawei Han and Micheline Kamber, “Data Mining: Concept and Techniques,” Morgan Kaufmann Publishers, 2001.
[20].G. D. Zhou and K. T. Lua, “Interpolation of n-gram and mutual information based trigger pair language models for Mandarin speech recognition”, Computer Speech and Language, vol. 13, pp.125-141, 1999.
[21].GuoDong ZHOU, KimTeng LUA, “Word Association and MI-Trigger-based Language Modeling”.
[22].M. Siu and M. Ostendorf, “Variable n-grams and extensions for conversational speech language modeling”, IEEE Trans. on Speech and Audio Processing, vol. 8, no. 1, Jan. 2000。
[23].Colombo, C.; Del Bimbo, A.; Pala, P.: Semantics in Visual Multimedia Information Retrieval. IEEE Multimedia, Vol. 63, pp.38-53, 1999.
[24].吳宗憲、陳昭宏、林超群，中文文句翻語音系統中連音處理之研究，第九屆計算語言學研討會，pp.85-104，民國85年。
[25].J. R. Bellegarda, “Exploiting Latent Semantic Information in Statistical Language Modeling,” Proceedings of the IEEE, Vol. 88, No. 8, 2000.
[26]. Berlin Chen, H. M. Wang, and L. S. Lee, “An HMM/N-gram-based Linguistic Processing Approach for Mandarin Spoken Document Retrieval,” the 7th European Conference on Speech Communication and Technology (EuroSpeech), Demark, 2001.
[27].Ronald Rosenfeld, “Adaptive Statistical Language Modeling: A Maximum Entropy Approach,” the Ph.D. Thesis in School of Computer Science, CMU, 1994.
[28].Wolfgang Wahlster, “Vermobil: Foundations of Speech-to-Speech Translation,” Springer-Verlag Press, 2000.
[29].教育部特殊教育工作小組，2001年手語教學與應用研討會論文集，民90年。
[30].Ann, Jean, A Linguistic Investigation of the Relationship Between Physiology and HandShape, Ph.D. Thesis, Dept. of Linguistics in the University of Arizona, 1993.
[31].Agrawal, A. and Srikant, R., “Fast Algorithms for Mining Association Rules”, Proc. Of the 20th VLDB Conf., Santiago-Chile, 487-499, 1994.
[32].Tung-Hui Chiang, Yi-Chung Lin, and Keh-Yih Su, “On Jointly Learning the Parameters in a Character-Synchronous Integrated Speech and Language Model”, IEEE Trans. on Speech and Audio Processing, vol. 4, no. 3, 1996.
[33].Keh-Yih Su, Tung-Hui Chiang, and Yi-Chung Lin, “A Unified Framework to Incorporate Speech and Language Information in Spoken Language Processing”, 1992.
[34].Tung-Hui Chiang and Yi-Chung Lin, “An Integrated Scoring Function for A Spoken Dialogue System”, Proceedings of ICSP ’98.
[35].L. L. Chang, “The Modality words in modern Mandarin, “CKIP Technical Report, no. 93-106, Jun. 1993.
[36].CKIP, http://godel.iis.sinica.edu.tw, 中央研究院資訊科學研究所詞庫小組。
[37].Q. Zhou and S. Feng, “Building a relation network representation for How-Net”, Proceedings of International Conference on Multilingual Information Processing, pp. 139-145, 2000.
[38].I. H. Witten and T. C. Bell, “The zero-frequency problem: Estimating the probabilities of novel events in adaptive text compression.”, IEEE Transactions on Information Theory, vol.37, pp.1085-1094, 1991.
[39].Kazuaki Ando, Tae-hun Lee, Masami Shishibori, and Jun-ichi Aoe, “A method of Pre-computing Connectivity Relations for Japanese/Korean POS Tagging”, CICLing 2001, LNCS 2004, pp.363-374, 2001.
[40].Hsin-Hsien Lee, Jane Tsay and James Myers, “Handshape Articulation in Taiwan Sign Language and Signed Chinese,” the manuscript, Institute of Linguistics, National Chung Cheng University, 2002.