在電話客服系統中導入語音情緒辨識系統，能夠掌握客服人員通話時的情緒狀態，得以在當客服人員與顧客發生爭執時及時做出妥善的處理。但客服系統每天有著龐大的通話量，對於情緒辨識系統是龐大的負擔，因此發展一個機制使得情緒辨識系統在最有效率的情況下運作相當重要。

本研究在已結合語音情緒辨識功能的VoIP客服系統中，將欲進行辨識的語音資料以負載平衡方式分配到後端辨識引擎，除可提升效率外，在分配語音資料的過程中能依據辨識系統間效能的變化具有自我調適的能力。

為進一步加速辨識速度，我們將中性情緒基線模型與語音情緒辨識系統結合，用來初步過濾中性情緒的語音資料，以降低辨識系統的負荷。實驗結果顯示，針對已建立模型之語者在保留至少87%生氣情緒語音片段的前提下，最高可過濾84%的中性情緒語音片段，由於客服系統中的對話絕大部分是中性情緒，這種作法可以大幅提升辦識引擎效率。

By Introducing the speech emotion recognition system into a call center, the emotion of a customer service representative can be monitored in real-time during the conversation with customers. Hence, a quick reaction procedure can be developed to resolve the emotional problem of the customer service representative when a conflict between the service representative and the customer. However, the amount of call data of a call center is enormous which becomes a huge burden to the speech emotion recognition system. Therefore, a mechanism to improve the efficiency of a speech emotion recognition system is one of the key issues in the successfulness of the integration of an emotion recognition into the call center system.

In this research, we proposed a mechanism to integrate the neutral emotion baseline model and a load balanced voice data distribution method to a VoIP call center with speech emotion recognition capability. With the neutral emotion baseline model, we can filter out most of the voice segments with neutral emotion. With this mechanism, we can improve the performance of the speech emotion recognition system drastically. On the other hand, the proposed load balance mechanism has the self adjust capability in facing various load conditions as well as one or more recognition engines failure.

From the experimental result, with the introducing of neutral emotion baseline model, we can filter out at least 84% neutral emotion voice segment while keeping at least 87% of anger emotion voice segment. Considering the fact that most of the dialogs in the call center are with neutral emotion in nature, the performance of the recognition engine can be greatly improved.

致謝 i
摘要 ii
ABSTRACT iii
目錄 iv
圖目錄 vi
表目錄 vii
第一章 緒論 1
1.1 前言 1
1.2 動機和目標 1
1.3 論文架構 3
第二章 相關研究 4
2.1 情緒分類 4
2.2 語音特徵 6
2.2.1 Mel-Frequency Cepstral coefficients(MFCC) 7
2.3 分類器 11
2.3.1 Gaussian Mixture Model(GMM) 12
2.3.2 Weighted Discrete-KNN (WD-KNN) 13
2.4 語料庫 14
2.4.1 Mandarin Chinese Emotion Corpus 2010(MCEC2010) 14
2.5 語音資料分配機制 15
2.5.1 固定分配 (Fix Assignment) 15
2.5.2 循環分配 (Round-Robin) 16
2.5.3 負載平衡分配 (Load Balance) 17
第三章 系統設計及架構 19
3.1 系統架構 19
3.2 情緒辨識引擎 (Emotion Recognition Engine) 21
3.2.1 特徵擷取 (Feature Extraction) 22
3.2.2 基線模型 (Baseline Model)和情緒分類 (Classification) 23
3.3 負面情緒偵測 (Negative Emotion Detection) 24
第四章 實驗及結果 25
4.1 VoIP通話平台 25
4.2 建立語音情緒基線模型 27
4.3 語音情緒辨識引擎 27
4.4 辨識結果儲存、呈現 30
第五章 結論與未來工作 33
參考文獻 35
附錄 40

[1] Chung-Hsien Wu and Wei-Bin Liang, "Emotion Recognition of Affective Speech Based on Multiple Classifiers Using Acoustic-Prosodic Information and Semantic Labels," IEEE Transactions on Affective Computing, vol. 2, no. 1, pp. 10-21, 2011.
[2] Abhinav Dhall, Akshay Asthana, Roland Goecke, and Tom Gedeon, "Emotion Recognition Using PHOG and LPQ features," in 2011 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011), Santa Barbara, CA, 2011.
[3] Miraj Shah, David G. Cooper, Houwei Cao, Ruben C. Gur, Ani Nenkova, and Ragini Verma, "Action Unit Models of Facial Expression of Emotion in the Presence of Speech," in 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction (ACII), Geneva, Switzerland, 2013.
[4] Chendi Wang and Feng Wang, "An emotional analysis method based on heart rate variability," in 2012 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), Hong Kong , 2012.
[5] ChungK Lee, SK Yoo, YoonJ Park, NamHyun Kim, KeeSam Jeong, and ByungChae Lee, "Using Neural Network to Recognize Human Emotions from Heart Rate Variability and Skin Resistance," in 27th Annual Conference on Engineering in Medicine and Biology, Shanghai, China, 2005.
[6] "Basis emotions," [Online]. Available: http://changingminds.org/explanations/emotions/basic%20emotions.htm. [Accessed Nov. 14, 2013].
[7] Eun Ho Kim, Kyung Hak Hyun, Soo Hyun Kim, and Yoon Keun Kwak, "Improved Emotion Recognition With a Novel Speaker-Independent Feature," IEEE/ASME Transactions on Mechatronics, vol. 14, no. 3, pp. 317-325, 2009.
[8] Bjorn Schuller, Gerhard Rigoll, and Manfred Lang, "Hidden Markov Model-Based Speech Emotion Recognition," in 2003 IEEE International on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP), 2003.
[9] Bjorn Schuller, Gerhard Rigoll, and Manfred Lang, "Speech Emotion Recognition Combining Acoustic Features and Linguistic Information in a Hybrid Support Vector Machine - Belief Network Architecture," in IEEE International Conference on Acoustics, Speech, and Signal Processing, 2004. Proceedings. (ICASSP '04). pp. I - 577-580 vol.1, 2004.
[10] Ronald B‥ock, David H‥ubner, Andreas Wendemuth, "Determining Optimal Signal Features and Parameters for HMM-Based Emotion Classification," in MELECON 2010, 15th IEEE Mediterranean Electrotechnical Conference, Valletta, 2010.
[11] Liqin Fu, Changjiang Wang, and Yongmei Zhang, "A Study on Influence of Gender on Speech Emotion Classification," 2010 2nd International Conference on Signal Processing Systems (ICSPS) IEEE, pp. Vl-534-Vl-537, 2010 .
[12] Qingli Zhang, Ning An, Kunxia Wang, Fuji Ren, and Lian Li, "Speech Emotion Recognition using Combination of Features," in 2013 Fourth International Conference on Intelligent Control and Information Processing (ICICIP), Beijing, China, Jun. 2013.
[13] Dan-Ning Jiang and Lian-Hong Cai, "Speech Emotion Classification with the Combination of Statistic Features and Temporal Features," in 2004 IEEE International Conference on Multimedia and Expo, 2004. ICME '04. vol. 3, pp. 1967-1970, 2004.
[14] Anurag Jain, Nupur Prakash , and S.S.agrawa, "Evaluation of MFCC for Emotion Identification in Hindi Speech," in Communication Software and Networks (ICCSN),2011 IEEE 3rd International Conference, Xi'an, 2011.
[15] Nobuo Sato and Yasunari Obuchi, "Emotion Recognition using Mel-Frequency Cepstral Coefficients," Natural Language Processing, vol. 14, no. 4, pp. 83-96, 2007.
[16] Douglas A. Reynolds and Richard C. Rose, "Robust Text-Independent Speaker Identification using Gaussian Mixture Speaker Models,", IEEE Transactions on Speech and Audio Processing, vol. 3, no. 1, pp. 72-83, Jan 1995.
[17] S. Ramamohan and S. Dandapat, "Sinusoidal Model-Based Analysis and Classification of Stressed Speech," IEEE Transactions on Audio, Speech, and Language Processing, vol. 14, no. 3, pp. 737-746, May 2006.
[18] Hao Hu, Ming-XingXu, and Wei Wu, "GMM Supervector Based SVM with Spectral Features for Speech Emotion Recognition," in IEEE International Conference on Acoustics, Speech and Signal rocessing, 2007. ICASSP 2007. Honolulu, HI, 2007.
[19] Tsang-Long Pao, Yun-Maw Cheng,Jun-Heng Yeh, Yu-Te Chen, Chen-Yu Pai and Yao-Wei Tsai, "Comparison between Weighted D-KNN and Other Classifiers for Music Emotion Recognition," in 3rd International Conference on Innovative Computing Information and Control, 2008. ICICIC '08. Dalian, Liaoning, 2008.
[20] Abdulbasit Al-Talabani, Harin Sellahewa, and Sabah Jassim, "Excitation Source and Low Level Descriptor Features Fusion for Emotion Recognition using SVM and ANN," in 2013 5th Computer Science and Eletronic Engineering Conference(CEEC), 2013.
[21] Ling He, Margaret Lech, Namunu Maddage, Sheeraz Memon, and Nicholas Allen, "Emotion Recognition in Spontaneous Speech within Work and Family Environments," in 3rd International Conference on Bioinformatics and Biomedical Engineering , 2009. ICBBE 2009., Beijing, 2009.
[22] Vipul Garg, Harsh Kumar, and Rohit Sinha, "Speech Based Emotion Recognition Based on Hierarchical Decision Tree with SVM, BLG and SVR Classifiers," in 2013 National Conference on Communications (NCC), New Delhi, India, 2013.
[23] Y. Chang, "Emotion Recognition and Evaluation of Mandarin Speech Using Weighted D-KNN Classification," Master Thesis, Tatung University, 2005.
[24] M. Murugappan, "Human Emotion Classification using Wavelet Transform and KNN," in 2011 International Conference on Pattern Analysis and Intelligent Robotics, Putrajaya, Malaysia, Jun. 2011.
[25] J.-H. Yeh, “The Study on Emotion Recognition in Continuous Mandarin Chinese Speech,” Dissertation, Tatung University, 2010.