臺灣博碩士論文加值系統

自發性語音(Spontaneous speech)是最接近人們日常生活的對話，因此也顯得非常重要。本研究主要分為兩個部分，分別對於自發性語音之聲學模型(Acoustic Model, AM)與韻律模型(Prosodic Model, PM)進行改善。在聲學模型方面，本研究利用朗讀式語音(Read speech)來協助訓練，我們使用模型調適(Model Adaptation)的方法，將朗讀式語音的聲學模型調適至自發性語音的聲學模型，再進一步利用Skip state HMM來改善刪除型錯誤過多的情形。
而在自發性語音韻律模型方面，本研究沿用過去所提出的階層式韻律模型(Hierarchical Prosodic Model, HPM)為基礎，來建構適合自發性語音的韻律模式，本研究對於音節韻律模型(Syllable prosodic model)進行修改，考慮其他可能的影響因子(Affecting Factor, AF)，最後，語料庫經過自動標記後，探討自發性語音中特有現象的韻律變化，並期望這些發現以及改善可幫助未來進行自發性語音相關的研究。

The spontaneous speech is the closest talking way to people’s daily life, therefore it appears to be very important. This thesis has two parts, one is about improving acoustic model and the other is about improving prosody model. In acoustic modeling, we use the read speeh data to assist the training and use the method of model adaptation to adapting acoustic model of read speech to spontaneous speech. Furthermore, we use the technology of the skip state HMM to fix deletion error problem.
In prosodic modeling, we construct prododic model which is adapted to spontaneous speech based on the Hierarchical Prosodic Model (HPM). We modify syllable prosodic model and consider other possible affecting factors. Lastly, an analysis of disfluencies related to the labeling results is also discussed and we expect those results would be able to improve the research on spontaneous speech.

中文摘要 I
ABSTRACT II
致謝 III
目錄 IV
表目錄 VII
圖目錄 IX
第1章 緒論 1
1.1 研究動機 1
1.2 文獻回顧 1
1.2.1 聲學模型之相關研究 1
1.2.2 韻律模型之相關研究 1
1.3 研究方向 2
1.4 章節概要說明 3
第2章 語料庫之簡介 4
2.1 MCDC之簡介 4
2.2 音檔格式說明 5
2.3 語料標記格式說明 6
2.4 自發性語音之特性 7
2.4.1 特殊音韻現象(Pronunciation variation) 7
2.4.2 無法或難以辨認的語音(Unintelligible speech sound) 7
2.4.3 不流暢的語流(Disfluency) 7
2.4.4 受外語或方言影響(Socio-linguistic phenomena) 8
2.4.5 語助詞(Marker)與感嘆詞(Particle) 8
2.4.6 非語音(Non-Speech Sounds) 8
2.5 MCDC語料庫之後處理 9
2.5.1 斷詞與詞性標記 9
2.5.2 修正音節切割位置 9
2.5.3 修正聲調標記 9
2.5.4 修正轉寫錯誤 9
2.6 TCC300之簡介 10
第3章 聲學模型之建立與改善 11
3.1 訓練語料與測試語料 11
3.2 聲學模型之建立 12
3.2.1 特徵參數抽取 12
3.2.2 實驗一 13
3.2.2.1 實驗一之辨識結果 14
3.2.3 實驗二 15
3.2.3.1 實驗二之辨識結果 17
3.2.3.2 Skip State HMM 18
第4章 韻律模型之建立與改善 21
4.1 漢語語音韻律階層式架構 21
4.2 階層式韻律模型設計 23
4.2.1 音節韻律模型 25
4.2.2 停頓標記聲學模型 27
4.2.3 韻律狀態轉移模型 28
4.2.4 停頓標記語法模型 29
4.3 韻律相關聲學參數之萃取 29
4.3.1 音節韻律參數之萃取 29
4.3.2 音節間韻律參數之萃取 33
4.3.3 相鄰兩音節間差異韻律參數之萃取 34
4.4 韻律模型之訓練 34
4.4.1 初始化 35
4.4.2 重覆疊代 37
第5章 韻律模型之實驗結果與分析 38
5.1 音節韻律模型 39
5.1.1 音節層次之影響樣型 40
5.1.2 上層韻律狀態層次之影響樣型 55
5.2 停頓標記聲學模型 57
5.3 韻律狀態轉移模型 58
5.4 停頓標記語法模型 60
5.5 韻律標記結果與自發性語音特殊現象之討論 62
5.5.1 停頓標記結果分析 62
5.5.2 特殊現象之分析 66
5.6 詞語修補(REPAIR)現象 70
5.6.1 重覆(Repetition) 70
5.6.2 部分重覆(Restart)與詞語更正(Repair) 75
第6章 結論與未來展望 80
參考文獻 82
附錄一：中研院詞類標記集 84
附錄二 86
附錄三：問題集 89

[1] M. Riley, W. Byrne, M. Finke, S. Khudanpur, and A. Ljolje, “Stochastic pronunciation modeling from hand-labelled phonetic corpora,” Speech Communication, Vol. 29, No. 2-4, pp. 209-224, 1999
[2] Y. Liu, and P. Fung, “State-Dependent Phonetic Tied Mixtures with Pronunciation Modeling for Spontaneous Speech Recognition,” IEEE Transactions on Speech and Audio Processing, Vol. 12, No.4, pp. 351-364, 2004
[3] Nanjo, H.; Kawahara, T., "Language model and speaking rate adaptation for spontaneous presentation speech recognition," Speech and Audio Processing, IEEE Transactions on , vol.12, no.4, pp. 391- 400, July 2004
[4] Måhl, Lena, “Speech recognition and adaptation experiments on children’s speech”, Master of Science thesis at the Department of Speech, Music and Hearing, KTH (The Royal Institute of Technology), 2004.
[5] K. Silverman, M. Beckman, J. Pitrelli, M. Ostendorf, C. Wightman, P. Price, J.
Pierrehumbert, and J. Hirschberg, “TOBI: A standard for Labeling English Prosody,” In Proc. of ICSLP, pp. 864-870, 1992.
[6] Aijun Li, “Chinese prosody and prosodic labeling of spontaneous speech,” in Proc. of Speech Prosody, pp. 39-46, 2002.
[7] Maekawa, K., H. Kikuchi, Y. Igarashi and J. Venditti. “X-JToBI: An extended J_ToBI for spontaneous speech,” in Proc. of ICSLP, pp. 1545-1548, 2002.
[8] M. Ostendorf, I. Shafran, S. Shattuck-Hufnagel, L. Carmichael, and W. Byrne, “A prosodically labeled database of spontaneous speech,” in Proc. of the ISCA Workshop on Prosody in Speech Recognition and Understanding , pp. 119-121, 2001.
[9] 江振宇，“非監督式中文語音韻律標記及韻律模式”，國立交通大學博士論文，民國九十八年三月。
[10] 周裕倫，“中文自發性語音之韻律標記及韻律模式”，國立交通大學碩士論文，民國九十八年七月。
[11] 曾淑娟, 劉怡芬, “現代漢語口語對話語料庫標註系統說明,” 中央研究院語言學研究所籌備處, September. 2002
[12] 李柏蒼，“自發性國語語音辨識”，國立交通大學碩士論文，民國九十六年八月。
[13] The HTK Book (for HTK version 3.4)
[14] WaveSurfer Homepage：www.speech.kth.se/wavesurfer/
[15] 吳聲鋒，“使用於中文自發性語音辨認之聲學模式及韻律模式”，國立交通大學碩士論文，民國一零三年八月。
[16] Z. Sheng, J.-H. Tao, and D.-L. Jiang, “Chinese prosodic phrasing with extended features,” Proceedings of the IEEE ICASSP , Vol. 1, pp. 492–495, 2003
[17] C.-Y. Tseng, S.-H. Pin, Y.-L. Lee, H.-M. Wang, and Y.-C. Chen, “Fluent speech prosody: Framework and modeling,” Speech Commun. special issue on quantitative prosody modeling for natural speech description and generation, 46, 284–309, 2005.
[18] C.Y. Tseng and Z.Y. Su, “Corpus approach to phonetic investigation - methods, quanitative evidence and findings of Mandarin speech prosody,” in Proc. of Oriental COCOSDA Workshop, pp. 123-138, 2006.
[19] S.H. Chen and Y.R. Wang, “Vector Quantization of Pitch Information in Mandarin Speech”, IEEE Transactions on Communications, Vol. 38, No. 9, pp. 1317-1320, 1990.
[20] D. Povey, A. Ghoshal, et al., "The Kaldi Speech Recognition Toolkit," in Proc. ASRU, 2011.
[21] Ghahremani, P., BabaAli, B., Provey, D., Riedhammer, K., Trmal, J. &;Khudanpur, S., “A Pitch Extraction Algorithm Tuned for Automatic Speech recognition”, in Proc ICASSP, Florence, 2014.
[22] S.C. Tseng, “Repairs in Mandarin Conversation,” Journal of Chinese Linguistics, Vol. 34, No.1, pp. 80-120, 2006.