(3.236.231.61) 您好!臺灣時間:2021/05/11 22:36
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果

詳目顯示:::

我願授權國圖
: 
twitterline
研究生:蔡緯鴻
研究生(外文):Wei-Hong Tsai
論文名稱:基於遷移學習之低資源語音辨識
論文名稱(外文):Low-Resource Speech Recognition Based on Transfer Learning
指導教授:王家慶
指導教授(外文):Jia-Ching Wang
學位類別:碩士
校院名稱:國立中央大學
系所名稱:資訊工程學系
學門:工程學門
學類:電資工程學類
論文出版年:2020
畢業學年度:108
語文別:中文
論文頁數:62
中文關鍵詞:語音辨識低資源端到端
外文關鍵詞:speech recognitionlow-resourceend-to-end
相關次數:
  • 被引用被引用:0
  • 點閱點閱:69
  • 評分評分:系統版面圖檔系統版面圖檔系統版面圖檔系統版面圖檔系統版面圖檔
  • 下載下載:0
  • 收藏至我的研究室書目清單書目收藏:0
近年端到端語音辨識(End-to-End Speech Recognition)成為語音辨識的研究趨勢,許多研究致力於探索語音辨識更高的準確性,並且在各個著名的語料庫上達到更高的準確性。然而,這些高度的準確性建立在龐大的語料上,而世界上有許多少數語言,沒有充足的語料建立該種語言的語音辨識,所建構出的語音辨識往往準確性過低,因此,如何以少量的語料建立語音辨識系統一直是語音辨識上的一項議題。
本論文使用ESPnet toolkit實現序列對序列的(Sequence to Sequence, Seq2Seq)端到端語音辨識模型,以及Fairseq toolkit實現輔助語音辨識的無監督預訓練模型,利用無標籤的(Unlabeled)單一語音資料協助擷取語音特徵,並透過遷移學習(Transfer Learning),將建立於語料較充足的語音辨識模型遷移至語料較缺乏的客語語音辨識,以此建立一個較強健的低資源(Low Resource)客語語音辨識。
Recent years, end-to-end speech recognition become a popular architecture. Many research aim to improve accuracy in end-to-end speech recognition, and they achieve higher accuracy on various famous corpora indeed. However, there are many language which do not have enough data to build their speech recognition system in the world. The system often can not get a reliable result and can not be used in real-world. Therefore, how to build a reboust low-resource speech recognition is an important issue in speech recognition.
This paper uses ESPnet toolkit to implement an end-to-end speech recognition model based on sequence-to-sequence architecture, and also uses Fairseq toolkit to implement an unsupervised pre-training model for assisted speech recognition. Using unlabeled speech data to help extract speech features, and transfer a speech recognition model based on sufficient corpus to Haaka speech recognition with less corpus through transfer learning. Establish a more robust low-resource Hakka speech recognition.
摘要 i
Abstract ii
章節目次 iii
圖目次 v
表目次 vi
第一章 緒論 1
1.1 研究動機 1
1.2 研究方向 2
1.3 章節概要 2
第二章 語音辨識簡介及文獻探討 3
2.1 特徵向量 3
2.1.1 頻譜圖(Spectrogram) 3
2.1.2 梅爾頻譜(Mel-Spectrum) 4
2.1.3 梅爾頻率倒譜係數(Mel-Frequency Cepstral Coefficients, MFCCs) 5
2.2 傳統語音辨識架構 6
2.3 相關文獻 8
第三章 模型架構及預訓練方法 10
3.1 端到端語音辨識 10
3.1.1 連結時序分類(Connectionist Temporal Classification, CTC) 11
3.1.2 Speech-Transformer 15
3.1.3 CTC/Attention混合模型 21
3.1.4 標籤平滑化(Label Smoothing) 22
3.1.5 聯合解碼(Joint Decoding) 22
3.2 wav2vec模型 23
3.2.1 因果卷積神經網路(Causal Convolution Neural Network) 24
3.2.2 Encoder Network 25
3.2.3 Context Network 25
3.2.4 wav2vec目標函數 26
3.2.5 基於wav2vec之半監督語音辨識 27
3.3 Encoder模型遷移 28
3.4 基於wav2vec及模型遷移之半監督語音辨識 29
第四章 實驗 31
4.1 實驗流程與環境 31
4.2 語料介紹 32
4.2.1 Aishell語料庫 32
4.2.2 Aishell2語料庫 32
4.2.3 Aidatatang_200zh語料庫 33
4.2.4 MAGICDATA語料庫 33
4.2.5 Primewords語料庫 34
4.2.6 ST-CMDS-20170001_1語料庫 34
4.2.7 THCHS-30語料庫 35
4.2.8 客語語料庫 35
4.2.9 Librispeech語料庫 36
4.4 端到端語音辨識模型配置 37
4.5 Encoder預訓練配置 38
4.6.1 使用wav2vec於語音辨識 39
4.6.2 使用Encoder模型遷移 43
4.6.3 基於wav2vec及Encoder模型遷移之半監督語音辨識 44
第五章 結論與未來展望 46
第六章 參考文獻 48
[1] Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
[2] Graves, A., Fernández, S., Gomez, F., & Schmidhuber, J. (2006, June). Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference on Machine learning (pp. 369-376).
[3] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. In Advances in neural information processing systems (pp. 5998-6008).
[4] Mohamed, A., Okhonko, D., & Zettlemoyer, L. (2019). Transformers with convolutional context for ASR. arXiv preprint arXiv:1904.11660.
[5] Dong, L., Xu, S., & Xu, B. (2018, April). Speech-transformer: a no-recurrence sequence-to-sequence model for speech recognition. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 5884-5888). IEEE.
[6] He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).
[7] Kim, S., Hori, T., & Watanabe, S. (2017, March). Joint CTC-attention based end-to-end speech recognition using multi-task learning. In 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP) (pp. 4835-4839). IEEE.
[8] Watanabe, S., Hori, T., Kim, S., Hershey, J. R., & Hayashi, T. (2017). Hybrid CTC/attention architecture for end-to-end speech recognition. IEEE Journal of Selected Topics in Signal Processing, 11(8), 1240-1253.
[9] Nakatani, T. (2019). Improving transformer-based end-to-end speech recognition with connectionist temporal classification and language model integration.
[10] Schneider, S., Baevski, A., Collobert, R., & Auli, M. (2019). wav2vec: Unsupervised pre-training for speech recognition. arXiv preprint arXiv:1904.05862.
[11] Watanabe, S., Hori, T., Karita, S., Hayashi, T., Nishitoba, J., Unno, Y., ... & Renduchintala, A. (2018). Espnet: End-to-end speech processing toolkit. arXiv preprint arXiv:1804.00015.
[12] Park, D. S., Chan, W., Zhang, Y., Chiu, C. C., Zoph, B., Cubuk, E. D., & Le, Q. V. (2019). Specaugment: A simple data augmentation method for automatic speech recognition. arXiv preprint arXiv:1904.08779.
[13] Ko, T., Peddinti, V., Povey, D., & Khudanpur, S. (2015). Audio augmentation for speech recognition. In Sixteenth Annual Conference of the International Speech Communication Association.
[14] Adams, O., Wiesner, M., Watanabe, S., & Yarowsky, D. (2019). Massively multilingual adversarial speech recognition. arXiv preprint arXiv:1904.02210.
[15] Stoian, M. C., Bansal, S., & Goldwater, S. (2020, May). Analyzing ASR pretraining for low-resource speech-to-text translation. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 7909-7913). IEEE.
[16] Bansal, S., Kamper, H., Livescu, K., Lopez, A., & Goldwater, S. (2018). Pre-training on high-resource speech recognition improves low-resource speech-to-text translation. arXiv preprint arXiv:1809.01431.
[17] Ott, M., Edunov, S., Baevski, A., Fan, A., Gross, S., Ng, N., ... & Auli, M. (2019). fairseq: A fast, extensible toolkit for sequence modeling. arXiv preprint arXiv:1904.01038.
[18] Bu, H., Du, J., Na, X., Wu, B., & Zheng, H. (2017, November). Aishell-1: An open-source mandarin speech corpus and a speech recognition baseline. In 2017 20th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment (O-COCOSDA) (pp. 1-5). IEEE.
[19] Du, J., Na, X., Liu, X., & Bu, H. (2018). AISHELL-2: transforming mandarin ASR research into industrial scale. arXiv preprint arXiv:1808.10583.
[20] Beijing DataTang Technology Co., Ltd , “aidatatang 200zh, a free Chinese Mandarin speech corpus,” .
[21] Magic Data Technology Co., Ltd, “MAGICDATA Mandarin Chinese Read Speech Corpus,” http: //www.imagicdatatech.com/index.php/home/ dataopensource/data_info/id/101, 2019.
[22] Primewords Information Technology Co., Ltd., “Primewords Chinese Corpus Set 1,” 2018, https://www. primewords.cn.
[23] Surfingtech, “ST-CMDS-20170001 1 Free ST Chinese Mandarin Corpus,” .
[24] Wang, D., & Zhang, X. (2015). Thchs-30: A free chinese speech corpus. arXiv preprint arXiv:1512.01882.
[25] Ba, J. L., Kiros, J. R., & Hinton, G. E. (2016). Layer normalization. arXiv preprint arXiv:1607.06450.
[26] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016). Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2818-2826).
[27] Panayotov, V., Chen, G., Povey, D., & Khudanpur, S. (2015, April). Librispeech: an asr corpus based on public domain audio books. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 5206-5210). IEEE.
[28] Zeghidour, N., Xu, Q., Liptchinsky, V., Usunier, N., Synnaeve, G., & Collobert, R. (2018). Fully convolutional speech recognition. arXiv preprint arXiv:1812.06864.
[29] Hannun, A., Lee, A., Xu, Q., & Collobert, R. (2019). Sequence-to-sequence speech recognition with time-depth separable convolutions. arXiv preprint arXiv:1904.02619.
[30] Chan, W., Jaitly, N., Le, Q. V., & Vinyals, O. (2015). Listen, attend and spell. arXiv preprint arXiv:1508.01211.
[31] Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.
[32] Graves, A. (2012). Sequence transduction with recurrent neural networks. arXiv preprint arXiv:1211.3711.
[33] Chiu, C. C., Sainath, T. N., Wu, Y., Prabhavalkar, R., Nguyen, P., Chen, Z., ... & Jaitly, N. (2018, April). State-of-the-art speech recognition with sequence-to-sequence models. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 4774-4778). IEEE.
[34] Zeghidour, N., Usunier, N., Synnaeve, G., Collobert, R., & Dupoux, E. (2018). End-to-end speech recognition from the raw waveform. arXiv preprint arXiv:1806.07098.
[35] Chen, Y., Wang, W., & Wang, C. (2020). Semi-supervised ASR by End-to-end Self-training. arXiv preprint arXiv:2001.09128.
[36] Baskar, M. K., Watanabe, S., Astudillo, R., Hori, T., Burget, L., & Černocký, J. (2019). Self-supervised Sequence-to-sequence ASR using Unpaired Speech and Text. arXiv preprint arXiv:1905.01152.
[37] Peddinti, V., Povey, D., & Khudanpur, S. (2015). A time delay neural network architecture for efficient modeling of long temporal contexts. In Sixteenth Annual Conference of the International Speech Communication Association.
[38] Bai, S., Kolter, J. Z., & Koltun, V. (2018). An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271.
[39] Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., ... & Kavukcuoglu, K. (2016). Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499.
[40] Mohamed, A. R., Dahl, G. E., & Hinton, G. (2011). Acoustic modeling using deep belief networks. IEEE transactions on audio, speech, and language processing, 20(1), 14-22.
[41] Sak, H., Senior, A., & Beaufays, F. (2014). Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition. arXiv preprint arXiv:1402.1128.
[42] Povey, D., Ghoshal, A., Boulianne, G., Burget, L., Glembek, O., Goel, N., ... & Silovsky, J. (2011). The Kaldi speech recognition toolkit. In IEEE 2011 workshop on automatic speech recognition and understanding (No. CONF). IEEE Signal Processing Society.
[43] Povey, D., Peddinti, V., Galvez, D., Ghahremani, P., Manohar, V., Na, X., ... & Khudanpur, S. (2016, September). Purely sequence-trained neural networks for ASR based on lattice-free MMI. In Interspeech (pp. 2751-2755).
[44] Sriram, A., Jun, H., Satheesh, S., & Coates, A. (2017). Cold fusion: Training seq2seq models together with language models. arXiv preprint arXiv:1708.06426.
連結至畢業學校之論文網頁點我開啟連結
註: 此連結為研究生畢業學校所提供,不一定有電子全文可供下載,若連結有誤,請點選上方之〝勘誤回報〞功能,我們會盡快修正,謝謝!
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top
無相關論文
 
無相關期刊
 
無相關點閱論文
 
系統版面圖檔 系統版面圖檔