臺灣博碩士論文加值系統

本篇論文提出一個基於生理訊號的情緒辨識系統，藉由分析腦波圖(EEG)與心電圖(ECG)來辨識情緒。要辨識的情緒有七種：平常狀態(一般不受刺激的情緒狀態)、快樂、壓力、悲傷、噁心、生氣、驚訝。
本情緒辨識系統的類別為使用者獨立，系統可分為生理訊號擷取、訊號切割、特徵擷取、特徵選取與分類五個部分。在生理訊號擷取上，採用十個人當受測者擷取資料，實驗環境安靜不受外界打擾，使用的情緒刺激源為觀賞一段二至四分鐘的影片。區段切割是使用離散小波轉換與共同資訊理論來尋找腦波圖在長時間刺激下的有效情緒誘發區間。在特徵擷取上，腦波圖的特徵包含時域特徵、頻域特徵、情緒區間與基線區間差異特徵、不同channel的差異特徵、高階統計特徵與非線性特徵，共六類特徵；心電圖的特徵包含時域特徵、HRV序列特徵、Poincare plot特徵、基線區間特徵、非線性特徵、頻域特徵、波形特徵，共七類特徵。在特徵選取上，利用費雪鑑別分析、逐次反向式搜尋法、逐次正向式搜尋法與基因演算法來保留有效的特徵且移除多餘的特徵，提升正確率。最後採用K-最鄰近分類法與支持向量機為分類器，再用留一交叉驗證方式一次分出七種情緒。
結果顯示，使用基因演算法特徵選取搭配支持向量機分類器的形式，腦波圖辨識情緒最高可達87.14%，心電圖辨識情緒最高可達92.86%，腦波圖結合心電圖辨識情緒最高可達97.14%。

In this thesis, we proposed an emotion recognition system based on physiological signals. Electroencephalogram (EEG) and electrocardiogram (ECG) were used to recognize seven kinds of emotions, including normal (non-stimulated state), happy, stress, sad, disgust, anger, and surprise.
The emotion recognition system was user-independent, and was divided into five parts, including physiological signal acquisition, signal segmentation, feature extraction, feature selection, and classification. In the physiological signal acquisition part, we acquired data from 10 participants who watched video programs of two to four minutes in length to stimulate distinct emotions in a quiet room. We then used technologies based on discrete wavelet transform and mutual information to find emotion-related EEG segments. Six categories of features were extracted from the EEG segment, including time-domain features, frequency-domain features, features of difference between response and baseline, differential channel features, high-order-statistic features, and nonlinear features. Seven categories of features were extracted from the ECG segment, including time-domain features, HRV features, Poincare plot features, baseline features, nonlinear features, frequency-domain features, and waveform features. Four feature selectors, including fisher discriminant analysis (FDA), sequential backward selection (SBS), sequential forward selection (SFS), and genetic algorithms (GA), proceeded to select useful features and reduce feature dimensions. Finally, leave-one-out cross-validation was performed in combination with either the k-nearest neighbor (KNN) classifier or the support vector machine (SVM) classifier to recognize seven kinds of emotions.
The results demonstrated that the average classification accuracies base on EEG and ECG were 87.14% and 92.86% respectively when using the GA feature selector and the SVM classifier. The average classification accuracy increased to 97.14% when the features calculated from EEG and ECG signals were combined to work with the GA feature selector and the SVM classifier.

感謝辭 I
摘要 II
Abstract III
目錄 V
圖目錄 IX
表目錄 XI
第一章 緒論 1
1.1 研究動機 1
1.2 相關文獻回顧 2
1.3 論文架構 3
第二章 情緒辨識系統 4
2.1 情緒介紹 4
2.1.1 情緒的定義 4
2.1.2 情緒的分類 5
2.2 情緒辨識系統簡介 6
2.3 情緒辨識系統的類別 7
2.3.1 使用者相依的情緒辨識系統 7
2.3.2 使用者獨立的情緒辨識系統 8
2.4 生理訊號情緒辨識系統 8
2.4.1 生理訊號情緒辨識系統簡介 8
2.4.2 生理訊號介紹 9
2.4.2.1 腦波圖 9
2.4.2.2 心電圖 12
第三章 研究方法 15
3.1 生理訊號擷取 16
3.1.1 量測生理訊號的方法 16
3.1.2 訊號擷取的裝置 17
3.1.3 實驗流程 18
3.2 有效區段切割 20
3.2.1 離散小波轉換 21
3.2.2 共同資訊 24
3.3 特徵擷取 27
3.3.1 EEG特徵 27
3.3.1.1 頻域特徵 27
3.3.1.2 時域特徵 28
3.3.1.3 情緒區間與基線區間差異特徵 30
3.3.1.4 不同channel的差異特徵 30
3.3.1.5 高階統計特徵 30
3.3.1.6 非線性特徵 31
3.3.2 ECG特徵 36
3.3.2.1 時域特徵 37
3.3.2.2 HRV序列特徵 37
3.3.2.3 Poincare plot特徵 38
3.3.2.4 基線區間特徵 39
3.3.2.5 非線性特徵 39
3.3.2.6 頻域特徵 41
3.3.2.7 波形特徵 42
3.3.3 特徵正規化 45
3.4 特徵選取 45
3.4.1 費雪鑑別分析 45
3.4.2 逐次反向式搜尋法 46
3.4.3 逐次正向式搜尋法 47
3.4.4 基因演算法 48
3.5 分類 50
3.5.1 k-最近鄰居分類法 50
3.5.2 支持向量機 51
3.5.3 交叉驗證 53
第四章 實驗結果與討論 54
4.1 使用腦波圖的結果與討論 54
4.2 使用心電圖的結果與討論 56
4.3 結合腦波圖與心電圖的結果與討論 58
4.3.1 結合選取後的腦波圖與心電圖特徵 59
4.3.2 結合腦波圖與心電圖特徵再選取 61
4.4 針對實驗結果的進一步分析 63
4.4.1 以生理訊號做比較 63
4.4.2 以切割的區段做比較 63
4.4.3 以特徵選取及分類器的效能來比較 63
4.4.4 以特徵優劣做探討 65
4.5 情緒量表的結果 67
4.6 相關文獻比較 68
第五章 結論與未來發展 69
5.1 結論 69
5.2 未來發展 69
參考文獻 71
附錄 76

[1]Takahashi, K., “Remarks on emotion recognition from bio-potential signals”, 2nd International Conference on Autonomous Robots and Agents, 2004, Page(s): 186-191.
[2]Murugappan, M., Nagarajan, R. and Yaacob, Sazali , “Comparison of Different Wavelet Features from EEG Signals for Classifying Human Emotions”, IEEE Symposium on Industrial Electronics and Applications, 2009, Page(s): 836-841.
[3]Murugappan, M., Rizon, M., Nagarajan, R., Yaacob, S., Zunaidi, I. and Hazry, D., “EEG feature extraction for classifying emotions using FCM and FKM,” International Journal of Computer Networks & Communications, 2007, Vol. 1, Page(s): 21-25.
[4]Murugappan, M., Rizon, M., Nagarajan, R., Yaacob, S., Zunaidi, I. and Hazry, D., “Lifting scheme for human emotion recognition using EEG”, Information Technology, 2008, Vol. 2, Page(s): 1-7.
[5]Lin, Y. P., Wang, C.H., Wu, T.L., Jeng, S.K., Chen, J.H., “EEG-based emotion recognition in music listening: A comparison of schemes for multiclass support vector machine”, Acoustics, Speech and Signal Processing, 2009 , Page(s): 489-492.
[6]Kim, K. H., Bang, S. W., Kim, S. R., “Emotion Recognition System Using Short-term Monitoring of Physiological Signals”, Medical & Biological Engineering & Computing, 2004, Vol. 42, Page(s): 419-427.
[7]Rosalind, W. P., “Toward Machine Emotional Intelligence: Analysis of Affective Physiological State”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 23, No. 10, 2001, Page(s): 1175-1191.
[8]Rigas, G., Katsis, C.D., Ganiatsas, G., Fotiadis, D.I., “A User Independent, Biosignal Based, Emotion Recognition Method”, Lecture Notes In Artificial Intelligence, 2007, Vol. 4511, Page(s): 341-318.
[9]Arnold, M.B., “Emotion and Personality”, New York: Columbia Press, 1960.
[10]Mehrabian, A., Russell, J. A., “An Approach to Environmental Psychology”, MIT Press, Cambridge, MA, 1974.
[11]Schwartz, J., Shaver, P. R., “Emotions and emotion knowledge in interpersonal relations. In W. Jones & D. Perlman (Eds.), Advances in personal relationships”, Greenwich, CT: JAI Press, 1987, Vol. 1, Page(s): 197-241.
[12] Ekman, P. and Friesen, W. V., “Constants across cultures in the face and emotion,” Journal of Personality and Social Psychology, 1971, Vol. 17, Page(s): 124-129.
[13]Osgood, C. E., Suci, J. G., Tannenbaum, P.H., “The Measurement of Meaning”, The University of Illinois Press, Urbana, 1957.
[14]Russell, J.A., Pratt, G., “A Description of The Affective Quality Attributed to Environments”, Journal of Personality and Psychology, 1980, Vol. 38, No. 2, Page(s): 311-322.
[15]Russell, J.A., “Emotion Development Lab”, http://www2.bc.edu/~russeljm/
[16]Velten, E., “A laboratory task for induction of mood states”, Behavior, 1968.
[17]Lisetti, C., Nasozin, F., “Affective intelligent car interfaces with emotion recognition”, Proceedings of 11th International Conference on Human Computer Interaction, 2005, Vol. 2.
[18]Lang, P. J., “Behavioral Treatment and Bio-Behavioral Assessment: Computer Appliances”, In J. H. Johnson J. B. Sidowski, & T. A. Williams (Eds.), Technology in Mental Health Care Delivery Systems., 1980, Page(s): 119-137.
[19]Molina, G.G., Tsoneva, T., Nijholt, A., “Emotional brain-computer interfaces”, Affective Computing and Intelligent Interaction and Workshops, 2009.
[20]Berger, H., “Archiv für Psychiatrie und Nervenkrankheiten”, 1929, Page(s): 527-570
[21]周嘉陞, “心電圖學必備”, 合記圖書出版社, 2001.
[22]Petrantonakis, P.C. and Hadjileontiadis, L.J., ”Adaptive Extraction of Emotion-Related EEG Segments Using Multidimensional Directed Information in Time-Frequency Domain”, Engineering in Medicine and Biology 32th Annual Conference, 2010, Page(s): 1-4
[23]Davidson, R. J., Schwartz, G. E., Saron, C., Bennett, J. and Goleman, D. J., “Frontal versus parietal EEG asymmetry during positive and negative affect,” Psychophysiology, 1979, vol. 16, Page(s): 202-203.
[24]John Holland, “Adaptation in Natural an Artificial System,” University of Michigan Press, 1975.
[25]Goupillaud, P., Grossman, A. and Morlet, J., “Cycle-Octave and Related
Transforms in Seismic Signal Analysis”, Geoexploration, 1984, Page(s)::85-102.
[26]Adeli, H., Zhou, Z., Dadmehr, N., “Analysis of EEG records in an epileptic patient using wavelet transform”, Journal of Neuroscience Methods, 2003, Vol. 123, no. 1, Page(s): 69–87
[27]Mallat, S. G., “A Theory for Multiresolution Signal Decomposition: The Wavelet Representation”, Pattern Analysis and Machine Intelligence, 1989, Vol. 11, No. 7, Page(s): 674-693.
[28]Robert Horlings, Dragos Datcu, Leon J. M. Rothkrantz, “Emotion Recognition using Brain Activity”, International Conference on Computer Systems and Technologies, 2008, Page(s): 1-6.
[29]Andrew J. Niemiec, Brian J. Lithgow, “alpha-band characteristics in EEG spectrum indicate reliability of frontal brain asymmetry measures in diagnosis of depression”, Engineering in Medicine and Biology 27th Annual Conference, 2005, Page(s): 7517-7520.
[30]Chang, C. C., Lin, C. J., “a Library for Support Vector Machines”, 2001.
[31]Petrantonakis, P.C. and Hadjileontiadis, L.J., “Emotion Recognition From EEG Using Higher Order Crossings”, Journal: IEEE Transactions on Information Technology in Biomedicine, 2010, Vol. 14, no. 2, Page(s): 186-197.
[32]Hsu, K. C. and Yu , S. N., “Classification of Seizures in EEG Using
Wavelet-Chaos Methodology and Genetic Algorithm”, World Congress on Medical Physics and Biomedical Engineering, 2009, Vol. 25/IV, Page(s): 564–567.
[33]Fraser, A. M., Swinney, H. L., “Independent coordinates for strange attractors from mutual information”, Physical Review A, 1986, Vol. 33, No. 2, Page(s): 1134-1140.
[34]Cao, L., “Practical method for determining the minimum embedding dimension of a scalar time series”, Physica D: Nonlinear Phenomena, 1997, Vol. 110, No. 1-2, Page(s): 43-50.
[35]Kantz, H. and Schreiber, T., “Nonlinear time series analysis, 2nd ed. ”, Cambridge University Press, New York, 2004.
[36]Borovkova, S., Burton, R. and Dehling, H., “Consistency of the Takens estimator for the correlation dimension”, Annals of Applied Probability, 1999, Vol. 9, Page(s): 376-390.
[37]Wolf, A., Swift, J. B., Swinney, H. L., et al. “Determining Lyapunov exponents from a time series”, Physica D: Nonlinear Phenomena, 1985, Vol. 16, No.3, Page(s): 285-317.
[38]Rosso, O. A., Blanco, S., Yordanova, J., Kolev, V., Figliola, A., Schurmann, M., Basar, E., "Wavelet entropy: a new tool for analysis of short duration brain electrical signals", Journal of Neuroscience Methods, 2001, Vol.105 (1), Page(s): 65-75.
[39]Brennan, M., Palaniswami, M., Kamen, P., “Do Existing Measures of Poincare Plot Geometry Reflect Nonlinear Features of Heart Rate Variability? “, IEEE Transactions on Biomedical Engineering, 2001, Vol. 48, No. 11, Page(s): 1342-1346.
[40]Pincus, S. M., “Heart rate control in normal and aborted-SIDS infants”, American Journal of Physiology, Vol. 33, 1991, Page(s): 638-646.
[41]Metin, A., “Approximate Entropy and Its Application in Biosignal Analysis “, Nonlinear Biomedical Signal Processing：Dynamic Analysis and Modeling, IEEE Press, 2001, Vol. 2, Page(s): 72-91.
[42]Katz, M. J, “Fractals and Analysis of Waveforms”, computers in biology and medicine 18, 1988, Page(s): 145–156.
[43]Balli, T. and Palaniappan, R., “A Combined Linear & Nonlinear Approach for Classification of Epileptic EEG Signals”, IEEE Engineering in Medicine and Biology Conference on Neural Engineering, 2009, Page(s): 714–717.
[44]Yalcin, I.; Mehmet, K.; “Combining classical HRV indices with wavelet entropy measures improves to performance in diagnosing congestive heart failure”, Computers in Biology and Medicine 37, 2007, Page(s): 1502-1510.
[45]Shawe-Taylor, N.C.a.J., “An Introduction to Support Vector Machines and
Other Kernel-based Learning Methods”, Cambridge University Press, 2000.
[46]“LIBSVM – A Library for Support Vector Machines”, http://www.csie.ntu.edu.tw/~cjlin/libsvm/index.html
[47]Sanei, S. and Chambers, J.A., “EEG SIGNAL PROCESSING”, Centre of Digital Signal Processing Cardiff University, UK, 2007.
[48]李穎潔, 邱意弘, 朱貽盛, ”腦電信號分析方法及其應用”, 科學出版社, 2009.