臺灣博碩士論文加值系統

本研究旨在探討學生口譯員英進中同步口譯時，線上免費語音辨識字幕對數字與專有名詞處理之影響，透過實驗比較有語音辨識字幕與沒有語音辨識字幕情況下，口譯學生表現之差異。實驗素材為兩篇長度九分半鐘的英文演講，以錄音檔的形式播放演講音檔，一篇音檔單獨搭配投影片播放，一篇音檔則於投影片旁播放語音辨識字幕。由於學生口譯員普遍較容易使用免付費的語音辨識服務，故選用免費的Otter.ai軟體產出英文語音辨識字幕，並在參與者口譯有語音辨識字幕的音檔時，以眼動軟體追蹤參與者目光。本實驗邀請13位臺灣的口譯碩士生參與研究，參與者完成同步口譯實驗後，進行刺激回想訪談，回想口譯時的挑戰與採用之策略，以此探究語音辨識字幕是否影響訊息處理以及口譯產出。
本實驗將參與者口譯內容進行分析，首先替譯文中的數字評分，發現在有語音辨識字幕的情況下，參與者獲得的平均分數較高，而且成對樣本t檢定結果顯示達顯著水準。研究結果也發現數字的錯誤率在有語音辨識字幕的情況下較低。本研究亦針對含數字與專有名詞的句子以及相鄰的句子進行準確度評分，結果顯示在有語音辨識字幕的情況下，句子準確度的平均分數較無字幕的平均分數高，然而成對樣本t檢定結果顯示此平均值無顯著差異。本實驗透過刺激回想訪談的結果，探究參與者對於口譯搭配語音辨識字幕應用之想法。結果顯示參與者對同步口譯搭配語音辨識字幕持保留態度，部分人認為語音辨識字幕在特定情況下有助於口譯表現。本研究最終參考氣力模型，探討字幕對同步口譯氣力分配之影響。

This empirical study investigated the effects of unedited automatic speech recognition (ASR) transcript on student interpreters’ simultaneous interpretation (SI) performance from English into Chinese in terms of numbers and proper names. The experiment recruited thirteen student interpreters who took part in an SI experiment and a following retrospective interview. The SI experiment required the participants to interpret two speeches, one with the presence of unedited ASR transcript and one without. During the tasks with ASR, a laptop displayed slides accompanied by ASR transcripts generated by Otter.ai, and an eye-tracking software was adopted to capture the participants’ eye movements during the task. In the tasks without the ASR, only slides were provided. Each of the two speeches in the SI experiments contained 20 numbers and 10 proper names. After the interpreting tasks and the retrospective interviews, the participants were asked to fill in a post-experiment survey and rate the perceived effectiveness of the ASR. Quantitative analysis was adopted to analyze the number renditions and the accuracy of sentences containing problem triggers and the neighboring sentences. A paired sample t-test showed that the participants scored significantly higher in number renditions when ASR was present. The accuracy of sentences in the tasks with ASR received higher scores, but a paired sample t-test showed there was no significance difference between the scores of the tasks with ASR and without ASR. In terms of proper names, results showed there were fewer omissions in the tasks with ASR. Finally, the retrospective interviews revealed that the participants adopted a more neutral stance towards the application of unedited ASR service, and they maintained that the ASR support might be useful under certain circumstances. Gile’s Effort Models were discussed at the end of this paper with an aim to explain the impact of unedited ASR on an SI task.

Table of Contents
Acknowledgement i
Abstract (English) ii
Abstract (Chinese) iii
Chapter 1 Introduction 1
Chapter 2 Literature Review 8
2.1 Effort Models and Competition Hypothesis of Simultaneous Interpretation 8
2.2 Remote Simultaneous Interpreting and Computer-assisted-interpreting 9
2.3 Automatic Speech Recognition and its Integration with CAI Tools 11
2.4 SI with Text vs. SI with ASR support 15
2.5 Numbers as a Problem Trigger in Simultaneous Interpreting 19
2.6 Proper Names as a Problem Trigger in Simultaneous Interpreting 23
Chapter 3 Methods 26
3.1 Participants 26
3.2 Materials 28
3.3 Automatic Speech Recognition Tool 30
3.4 Experiment Setup and Pilots 30
3.5 Procedure and Data Collection 31
3.6 Data Analysis 35
Chapter 4 Results and Discussion 45
4.1 Output and Accuracy of ASR 45
4.2 Number Renditions 50
4.3 Analysis of Proper Names Renditions 70
4.4 Accuracy of Critical Sentences 83
4.5 Attitudes towards ASR Support 85
4.6 Incorporating ASR into Gile’s Effort Models and Implication for Training 92
Chapter 5 Conclusion 97
5.1 Findings 97
5.2 Limitations 101
5.3 Future Research and Contribution of the Present Study 102
References 105
Appendix A. Post-Experiment Survey 111
Appendix B. Materials 115
Appendix C. Instructions for Stimulated Retrospective Interview 129
Appendix D. Consent Form 130

Alessandrini, M. S.(1990).Translating numbers in consecutive interpretation: An experimental study.The Interpreters' Newsletter, 3, 77-80.
Braun S., Clarici A. (1996). Inaccuracy for numerals in simultaneous interpretation: neurolinguistic and neuropsychlogical perspectives. Edizioni LINT Trieste.
Defrancq, B., & Fantinuoli, C. (2020). Automatic speech recognition in the booth: Assessment of system performance, interpreters’ performances and interactions in the context of numbers. Target. International Journal of Translation Studies, 33(1), 73-102.
Ericsson, K. A., & Simon, H. A. (1996). Verbal reports as data. Psychological Review, Cambridge, MA: MIT Press. (revised edition).
Fantinuoli, C. (2018). Interpreting and technology: The upcoming technological turn. Interpreting and technology, 1-12.
Fantinuoli, C., Marchesini, G., Landan, D., & Horak, L. (2022). KUDO Interpreter Assist: Automated Real-time Support for Remote Interpretation. Proceedings of Translator and Computer 53 Conference (2022). https://doi.org/10.48550/arXiv.2201.01800

Fantinuoli, C., & Montecchio, M. (2022). Defining maximum acceptable latency of AI-enhanced CAI tools. Proceedings of LingTech21 (2022). https://doi.org/10.48550/arXiv.2201.02792

Fantinuoli, C., & Prandi, B. (2018). Teaching information and communication technologies: a proposal for the interpreting classroom. Transkom, 11(2), 162-182.
Gile, D. (1984). Des difficultés de la transmission informationnelle en interprétation simultanée. Babel, 30(1), 18-25.
Gile, D. (1995). Basic Concepts and Models for Interpreter and Translator Training. Amsterdam: Benjamins.
Gile, D. (1999). Testing the Effort Models' tightrope hypothesis in simultaneous interpreting-A contribution. HERMES-Journal of Language and Communication in Business, (23), 153-172.
Gile, D. (2009). Basic concepts and models for interpreter and translator training (Vol. 8). John Benjamins Publishing.
Gile, D. (2020, August 4). Forty years of Effort Models of Interpreting: looking back, looking ahead [Paper presentation]. Japan Translation and Interpretation Forum 2020, Japan
Hemakumar, G., & Punitha, P. (2013). Speech recognition technology: a survey on Indian languages. International Journal of Information Science and Intelligent System, 2(4), 1-38.
Hinton, G., Deng, L., Yu, D., Dahl, G. E., Mohamed, A. R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen P., Sainath T. N., & Kingsbury, B. (2012). Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal processing magazine, 29(6), 82-97.
Huang. (2013). An Analysis on Performances between English-Chinese Simultaneous Interpretation with Text and without Text [Masters thesis, The Graduate Institute of Translation and Interpretation National Changhua University of Education]. National Digital Library of Theses and Dissertations in Taiwan. https://hdl.handle.net/11296/2a68b5
Ivanova, A. (2000). The use of retrospection in research on simultaneous interpreting. Benjamins Translation Library, 37, 27-52.
Kohn, K., & Kalina, S. (1996). The strategic dimension of interpreting. Meta: Journal des traducteurs/Meta: Translators' Journal, 41(1), 118-138.
Korpal, P., & Stachowiak-Szymczak, K. (2018). The whole picture: Processing of numbers and their context in simultaneous interpreting. Poznan Studies in Contemporary Linguistics, 54(3), 335-354.
Lambert, S. (2004). Shared attention during sight translation, sight interpretation and simultaneous interpretation. Meta: Journal des traducteurs/Meta: Translators' Journal, 49(2), 294-306.
Liao, Zhao, Yang & Yang. (2020). AI yǔ yīn biàn rèn jì shù yīng yòng yú zì dòng méi tǐ jié mù zì mù chǎn zhì [AI Automatic Speech Recognition application on automated subtitles for media]. Diàn gōng tōng xùn jì kān, 2020(4), 1-15.
Malik, M., Malik, M. K., Mehmood, K., & Makhdoom, I. (2021). Automatic speech recognition: A survey. Multimedia Tools and Applications, 80(6), 9411-9457.
Mazza, C. (2001). Numbers in simultaneous interpretation. The Interpreters’ Newsletter, 11, 87– 104.
Meyer, B. (2008). Interpreting proper names: Different interventions in simultaneous and consecutive interpreting. Transkom, 1(1), 105-122.
Moser-Mercer, B. (2003). Remote interpreting: assessment of human factors and performance parameters. Joint project International.
Nguyen, T. S., Niehues, J., Cho, E., Ha, T.-L., Kilgour, K., Muller, M., Sperber, M., Stueker, S., & Waibel, A. (2020). Low latency asr for simultaneous speech translation. Institute for Anthropomatics and Robotics Karlsruhe Institute of Technology. https://doi.org/10.48550/arXiv.2003.09891
Orlando, M. (2014). A study on the amenability of digital pen technology in a hybrid mode of interpreting: Consec-simul with notes. Translation & Interpreting, The, 6(2), 39-54.
Pellatt, V. (2005). The trouble with numbers: how linguistic, arithmetical and contextual complexity affect the interrpetation of numbers. Professionalisation in Interpreting: International experience and developments in China.
Pisani, E., & Fantinuoli, C. (2021). Measuring the Impact of Automatic Speech Recognition on Number Rendition in Simultaneous Interpreting. In Empirical Studies of Translation and Interpreting (pp. 181-197). Routledge. Chicago.
Prandi, B. (2017). Designing a multimethod study on the use of CAI tools during simultaneous interpreting. Translating and the Computer, 39, 76-113.
Roziner, I., & Shlesinger, M. (2010). Much ado about something remote: Stress and performance in remote interpreting. Interpreting, 12(2), 214-247.
Spelke. E. S. & Tsivkin, S. (2001). Language and number: a bilingual training study. Cognition, 78, 45-88.
techforword (2020) Automatic live captions for any meeting or webinar with Otter.ai (Mac and Windows) [Video]. Youtube. https://www.youtube.com/watch?v=stzApmOS74k&ab_channel=techforword
Vianna, B. (2005). Simultaneous interpreting: A relevance-theoretic approach. Intercultural Pragmatics, 2(2). https://doi.org/10.1515/IPRG.2005.2.2.169
Vogler, N., Stewart, C., & Neubig, G. (2019). Lost in interpretation: Predicting untranslated terminology in simultaneous interpretation. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Association for Computational Linguistics, 109–118, https://doi.org/10.18653/v1/N19-1010
Wang, H. (2006). Numbers as a quality variable in simultaneous interpreting: A case study of English into Chinese SI. [Dissertation: Graduate Institute of Translation and Interpretation, National Taiwan Normal University] https://hdl.handle.net/11296/s3743h
Zapata, J., & Kirkedal, A. S. (2015, May). Assessing the performance of automatic speech recognition systems when used by native and non-native speakers of three major languages in dictation workflows. In Proceedings of the 20th Nordic Conference of Computational Linguistics (NODALIDA 2015) (pp. 201-210).