臺灣博碩士論文加值系統

本研究提出一自動歌唱情感表現力評分系統，歌唱情感表現力在此定義為聽者感受到的歌者投入程度。本系統依據音準、抖音、尾音漸弱、音色粗糙程度、音量與音高的相對關係等五項特徵進行評分，再以支持向量回歸（support vector regression）進行得分估計，本系統可在不具任何已知資料（如：原曲音高、斷句位置）的情況下對任意樂句數的歌曲進行評分。網上所搜尋到的唯一最相似文獻，則僅能處理單一五秒左右長度的樂句。在本研究中，為評估系統準確度，蒐集建構了一個由九位參與者演唱，包含342段清唱音檔的歌唱資料庫，並以線上問卷取得聽者對於資料庫內音檔的評分。實驗結果顯示（leave-one-singer-out test），由本系統給出的評分與線上問卷取得的人類評分具有0.51的夠顯著正相關係數。

A system for automatically evaluating singing enthusiasm is proposed in this study. The definition of singing enthusiasm is how much enthusiasm is perceived in a song being evaluated. This system evaluates the singing enthusiasm on the basis of pitch accuracy, vibrato, diminuendo, roughness, and the correlation between pitch and loudness. A support vector regression (SVR) machine is used for the evaluation. This system can deal with songs having multiple phrases without any reference information such as the pitch ground truth or phrase location. To the authors’ knowledge, only one such system has previously been proposed which could only handle a single phrase of about 5-second long. To evaluate this system, a singing corpus with 342 song clips sung by nine participants was recorded and ground-truth enthusiasm evaluation scores were obtained by an online questionnaire. The experimental results obtained from a leave-one-singer-out test revealed that the enthusiasm scores evaluated by the proposed system had a significant positive correlation coefficient of 0.51 with the human-labeled ground truth.

口試委員審定書 i
致謝 ii
摘要 iii
Abstract iv
Content v
List of Figures vii
List of Tables viii
1 Introduction 1
1.1 Motivation 1
1.2 Objective: Singing Enthusiasm Evaluation 1
1.3 Contributions 2
1.4 Chapter Outline 2
2 Related Works of Singing Evaluation 3
3 Singing Corpus 5
3.1 Selection of Songs 5
3.2 Recording 6
4 Subject Enthusiasm Evaluation 8
4.1 Questionnaire Design 8
4.2 Source of Subjects 9
4.3 Rewards 9
4.4 Results 9
4.4.1 Information of Subjects 9
4.4.2 Evaluation Results 11
5 Singing Enthusiasm Evaluation System 14
5.1 System Overview 14
5.2 End-point Detection 14
5.3 Feature Selection 16
5.4 Low Level Features Extraction 17
5.4.1 Pitch 17
5.4.2 Loudness 19
5.4.3 Roughness 20
5.5 Features Used for Singing Enthusiasm Evaluation 21
5.5.1 Pitch Accuracy 21
5.5.2 Vibrato 22
5.5.3 Diminuendo at the End of Segments 24
5.5.4 Correlation between Loudness and Pitch 24
5.5.5 Average Roughness 25
5.6 Classifier: Support Vector Regression Machine 26
6 System Evaluation Experiment 28
7 Results and Discussion 29
7.1 System Evaluation Results 29
7.2 Comparison between Singer-intended, Human-labeled, and System-evaluated Enthusiasm 30
7.3 Emotion Expression in Different Song Types 32
8 Conclusions 35
Reference 36
Appendix 41

[1]H. J. Chong. 2010. Do we all enjoy singing? A content analysis of non-vocalists’ attitudes toward singing. The Arts in Psychotherapy, vol. 37, no. 2, pp. 120-124.
[2]T. Jeong. 1995. Score evaluation display device for an electornic song accompaniment apparatus. U.S. Patent No. 5434949.
[3]C. S. Park. 1996. Karaoke system capable of scoring singing of a singer on accompaniment thereof. U.S. Patent No. 5567162.
[4]T. Sone, K. Tsurumi, H. Kato, and T. Tanaka. 1998. Karaoke apparataus with individual scoring of duet singers,” U.S. Patent No. 5804752.
[5]W. H. Tsai and H. C. Lee. 2011. An automated singing evaluation method for karaoke systems. Proceedings of the IEEE International Conference Acoustic Speech Signal Process (ICASSP), pp. 2428–2431.
[6]W. H. Tsai and H. C. Lee. 2012. Automatic evaluation of Karaoke singing based on pitch, volume, and rhythm features. IEEE Transactions on Audio, Speech, and Language Processing, vol. 20, no. 4, pp. 1233–1243.
[7]T. Nakano, M. Goto, and Y. Hiraga. 2006. An automatic singing skill evaluation method for unknown melodies using pitch interval accuracy and vibrato features. International Conference on Spoken Language Processing, pp. 1706–1709.
[8]H. Takeuchi, M. Hoguro, and T. Umezaki. 2010. A KARAOKE system singing evaluation method that more closely matches human evaluation. IEEJ Transactions on Electrical and Electronic Engineering, vol. 7, pp. 6–7.
[9]R. Daido, M. Ito, S. Makino, and A. Ito. 2014. Automatic evaluation of singing enthusiasm for karaoke. Computer Speech and Language, vol. 28, no. 2, pp. 501–517.
[10]O. Mayor, J. Bonada, and A. Loscos. 2006. The singing tutor: expression categorization and segmentation of the singing voice. Proceedings of the AES 121st Convention.
[11]R. E. Thayer. 1989. The biopsychology of mood and arousal. Oxford University Press.
[12]J. C. Chen and J. S. R. Jang. 2008. TRUES: Tone recognition using extended segments. ACM Transactions on Asian Language Information Processing (TALIP), vol. 7, no. 3, pp. 1–23.
[13]Z. C. Fan, J. Jang, and C. L. Lu. 2016. Singing voice separation and pitch extraction from monaural polyphonic audio music via DNN and adaptive pitch tracking. IEEE International Conference on Multimedia Big Data (BigMM), vol. 1.
[14]E. Pampalk. 2004. A Matlab toolbox to compute music similarity from audio. Proceedings of the 2004 International Conference on Music Information Retrieval (ISMIR ’04), pp. 254–257.
[15]E. Pampalk, A. Rauber, and D. Merkl. 2002. Content-based organization and visualization of music archives. Proceedings of the tenth ACM international conference on Multimedia - MULTIMEDIA ’02, p. 570.
[16]H. Fastl and E. Zwicker. 1999. Psychoacoustics, facts and models. Springer-Verlag Berlin Heidelberg.
[17]M. R. Schroeder, B. S. Atal, and J. L. Hall. 1979. Optimizing digital speech coders by exploiting masking propoerties of the human ear. The Journal of the Acoustical Society of America, vol. 66, no. 6, pp. 1647–1652.
[18]R. Bladon. 1981. Modeling the judgment of vowel quality differences. The Journal of the Acoustical Society of America, vol.69, no. 5, pp. 1414-1422.
[19]W. M. Hartmann. 1998. Signal sound and sensation. American Institute of Physics.
[20]O. Lartillot, O. Lartillot, P. Toiviainen, and P. Toiviainen. 2007. A Matlab toolbox for musical feature extraction from audio. International Conference on Digital Audio Effects, pp. 237-244.
[21]W. A. Sethares. 2005. Tuning, Timbre, Spectrum, Scale. Springer Science & Business Media.
[22]R. Plomp and W. J. M. Levelt. 1965. Tonal consonance and critical bandwidth. The journal of the Acoustical Society of America, vol. 38, no. 4, pp. 548–560.
[23]F. Eyben, G. L. Salomão, J. Sundberg, K. R. Scherer, and B. W. Schuller. 2015. Emotion in the singing voice — a deeper look at acoustic features in the light of automatic classification. EURASIP Journal on Audio, Speech, and Music Processing, vol. 2015, no. 1, pp. 1-9.
[24]M. Kob, N. Henrich, H. Herzel, D. Howard, I. Tokuda, and J. Wolfe. 2011. Analysing and understanding the singing voice: recent progress and open questions. Current Bioinformatics, vol. 5, no. 3, pp. 362–374.
[25]J. Sundberg. 1995. Acoustic and psychoacoustic aspects of vocal vibrato. Vibrato, pp. 35-62.
[26]C. Chang and C. Lin. 2011. LIBSVM : A Library for Support Vector Machines. ACM Transactions on Intelligent Systems and Technology (TIST), vol. 2, pp. 1-39.
[27]H. Drucker, C. J. C. Burges, L. Kaufman, A. Smola, and V. Vapnik. 1996. Support vector regression machines. Advances in neural information processing systems, vol. 9, pp. 155-161.