臺灣博碩士論文加值系統

本研究提出一個基於三種短時間生理訊號的情緒辨識系統，其中包含了心電圖(Electrocardiogram, ECG)、光體積變化描記圖(Photoplethysmprgraphy, PPG)及皮膚阻抗(Skin Impedance, SI)，分辨五種負面情緒，平常狀態(未受刺激的狀態)、傷心(Sadness)、壓力(Stress)、生氣(Anger)及噁心(Disgust)。
本研究所發展的情緒辨識系統為使用者獨立，系統流程依序為生理訊號擷取、特徵擷取、特徵選取或特徵萃取與分類五個部分。在生理訊號擷取上，共有50名受試者，共有22名男性與28名女性，並使用影像或視覺刺激使用者的情緒。在特徵擷取方面，從生理訊號之刺激區間取出其中20秒做擷取，從心電圖中擷取7大類的特徵；光體積變化描記圖中擷取10大類的特徵；皮膚阻抗中擷取3大類的特徵，使用上述三類生理訊號擷取出140個特徵，目的為找出能夠代表情緒，並排除個體差異的特徵。計算出特徵後將其正規化，使特徵規範在同一動態範圍內。在特徵選取上，利用基因演算法(GA)來進行特徵的選取，目的為找出能夠使準確率最高的特徵子集。而在特徵萃取的部分，本研究比較了主成份分析(PCA)、獨立成份分析(ICA)、線性鑑別分析(LDA)以及三種改良式LDA (OLDA、SLDA、RLDA)的效能，目的為將特徵映射到更具代表性或更有分辨性的維度上。最後分類時採用支持向量機(LIBSVM)，並使用Leave-one-out的方式進行交叉驗證。
在結果方面，特徵選取可以找到一組特徵使得所有資料組合平均準確率達到70.4%；特徵萃取方面將每組資料準確率最佳化後可以得到的平均最佳準確率為67.6%；若將特徵萃取的映射向量以特徵選取的方式挑選，再將每組資料準確率最佳化後之平均最佳準確率為95.2%。

In this paper, we proposed an emotion recognition system based on three short-time physiological signals. Electrocardiogram (ECG), Photoplethysmorgraphy (PPG) and Skin Impedance (SI) were used to recognize five kinds of negative emotions, including neutral (non-stimulated state), sad, stress, anger and disgust.
In our study, we aimed to develop a user-independent system. This emotion recognition system was composed of data acquisition (physiological signals), feature calculation, normalization, feature selection or feature extraction, and classification. First, in the data acquisition part, 50 subjects were recruited to participate in this study, including 22 males and 28 females. By employing visual and audio stimulation, the subject emotions were induced and the signals were recorded. Second, in the feature calculation part, we calculated 7 types ECG features from wave-form and HRV sequence, 10 types PPG features from wave-form and HRV sequence and 3 types SI features from wave-form and SCR sequence. Totally, 140 features were calculated. Third, we normalized our feature set to the same level. Fourth, in the feature selection part, we performed Genetic Algorithm (GA) to select the most effective feature set to enhance accuracy. On the other hand, the feature extraction part, we compared the performance of the Principal Component Analysis (PCA), Independent Component Analysis (ICA), Linear Discriminant Analysis (LDA) and 3 modified LDA (OLDA, SLDA and RLDA) methods in reducing the feature dimensions by mapping the original data to the better subspace. Finally, we used SVM to classify emotions. And we performed leave-one-out scheme for cross validation.
According to the result, the accuracy were 70.4% when using GA feature selector, 67.6% when using OLDA feature extractor, 95.2% when using OLDA feature extractor in combination with the GA feature selector.

誌謝 I
摘要 II
Abstract III
目錄 V
圖目錄 VIII
表目錄 X
第一章 緒論 1
1.1 前言 1
1.2 研究動機 1
1.3 研究目標 2
1.4 研究架構 2
第二章 研究背景 3
2.1 情緒 3
2.1.1 什麼是情緒 3
2.1.2 情緒的分類 4
2.2 情緒辨識系統 5
2.2.1 User-dependent的情緒辨識系統 5
2.2.2 User-Independent的情緒辨識系統 5
2.3 心電圖 6
2.3.1 心臟的傳導系統 6
2.3.2 心電圖量測 7
2.3.3 心率變異分析(Heart Rate Variability) 8
2.4 光體積變化描記圖 9
2.4.1 光體積變化描記圖介紹 9
2.5皮膚阻抗 11
2.5.1 皮膚阻抗介紹 11
第三章 研究方法 13
3.1 生理訊號擷取 14
3.1.1 量測生理訊號的方法 14
3.1.2 訊號擷取 15
3.1.3 實驗介紹 16
3.2 心電圖(ECG)特徵擷取 19
3.2.1 時域特徵 19
3.2.2 波型特徵 20
3.2.3 HRV序列特徵 23
3.3 光體積變化描記圖(PPG)特徵擷取 30
3.3.1 時域特徵與PPI序列特徵 31
3.3.2 呼吸頻率特徵 31
3.3.3 脈博速率特徵 32
3.3.4 灌流指標特徵 32
3.3.5 波谷到波峰距離佔整段波長的比例特徵 33
3.4 皮膚阻抗(SI)特徵擷取 34
3.4.1 時域特徵 34
3.4.2 非線性特徵 35
3.4.3 皮膚導電反應特徵 35
3.5 特徵正規化 38
3.6 特徵選取方法 39
3.6.1 基因演算法 39
3.7 特徵萃取方法 41
3.7.1 主成份分析 41
3.7.2 獨立成份分析 42
3.7.3 線性鑑別分析 43
3.8 分類 48
3.8.1 支持向量機(Support Vector Machine) 48
3.8.2 交叉驗證 51
第四章 實驗結果與討論 52
4.1 特徵選取的結果與討論 53
4.2 特徵萃取的結果與討論 57
4.2.1 對訓練集特徵萃取得到映射矩陣 57
4.2.2 對全部資料特徵萃取得到映射矩陣 60
4.3 特徵萃取搭配特徵選取的結果與討論 64
4.4 情緒量表的結果 68
4.5 相關文獻比較 69
第五章 結論與未來展望 70
5.1 結論 70
5.2未來展望 72
參考文獻 73
附錄 78

[1] C. K. Lee, S. K. Yoo, Y. J. Park, "Nam-Hyun Kim: Using Neural Network to Recognize Human Emotions from Heart Rate Variability and Skin Resistance", Engineering in Medicine and Biology Society, 2005. IEEE-EMBS 2005. 27th Annual International Conference of the 2005, Page(s): 5523–5525.
[2] K. H. Kim, S. W. Bang, S. R. Kim, "Emotion Recognition System Using Short-term Monitoring of Physiological Signals", Med. Biol. Eng. Compute. Vol. 42, 2004, Page(s): 419-427.
[3] J. Wagner, J. Kim, E. Andre, "From Physiological Signals to Emotions: Implementing and Comparing Selected Methods for Feature Extraction and Classification", Multimedia and Expo, 2005. ICME 2005. IEEE International Conference on 6-6 July 2005, Page(s): 940–943.
[4] M. B. Arnold, "Emotion and Personality", New York: Columbia Press, 1960.
[5] A. Mehrabian, J. A. Russell, "An Approach to Environmental Psychology", MIT Press, Cambridge, MA, 1974.
[6] C. E. Osgood, J. G. Suci, P. H. Tannenbaum, "The Measurement of Meaning", The University of Illinois Press, Urbana, 1957.
[7] J. A. Russell, G. Pratt, "A Description of The Affective Quality Attributed to Environments", Journal of Personality and Psychology, Vol.38, No.2, 1980, Page(s): 311-322.
[8] J. A. Russell, "Emotion Development Lab", http://www2.bc.edu/~russeljm/
[9] R. W. Picard, E. Vyzas, J. Healey, "Toward Machine Emotional Intelligence: Analysis of Affective Physiological State", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 23, No. 10, October 2001.
[10] P. J. Lang, "Behavioral Treatment and Bio-Behavioral Assessment: Computer Appliances", In J. H. Johnson J. B. Sidowski, & T. A. Williams (Eds.), Technology in Mental Health Care Delivery Systems, 1980, Page(s): 119-137.
[11] “Wikipedia”, http://simple.wikipedia.org/wiki/Main_Page
[12] 周嘉陞, “心電圖學必備”, 合記圖書出版社, 2001.
[13] J. Pan, W. J. Tompkins, "A Real-Time QRS Detection Algorithm", IEEE Transactions on Biomedical Engineering, Vol. Bme-32, No. 3, March, 1985, Page(s): 230-236.
[14] A. B. Hertzman, C. R. Spielman, "Observations on the finger volume pulse
recorded photoelectrically", Am. J. Physiol., 1937, Vol.119, Page(s): 334-335.
[15] “交感神經”, https://zh.wikipedia.org/wiki/%E4%BA%A4%E6%84%9F%E7%A5%9E%E7%BB%8F
[16] “副交感神經”, https://zh.wikipedia.org/wiki/%E5%89%AF%E4%BA%A4%E6%84%9F%E7%A5%9E%E7%BB%8F
[17] “Autonomic nervous system”, http://en.wikipedia.org/wiki/Autonomic_Nervous_System
[18] M. Brennan, M. Palaniswami, P. Kamen, “Do Existing Measures of Poincare Plot Geometry Reflect Nonlinear Features of Heart Rate Variability? “, IEEE Transactions on Biomedical Engineering, 2001, Vol. 48, No. 11, Page(s): 1342-1346.
[19] I. Yalcin, K. Mehmet, “Combining classical HRV indices with wavelet entropy measures improves to performance in diagnosing congestive heart failure”, Computers in Biology and Medicine 37, 2007, Page(s): 1502-1510.
[20] S. M. Pincus, “Heart rate control in normal and aborted-SIDS infants”, American Journal of Physiology, Vol. 33, 1991, Page(s): 638-646.
[21] A. Metin, “Approximate Entropy and Its Application in Biosignal Analysis “, Nonlinear Biomedical Signal Processing：Dynamic Analysis and Modeling, IEEE Press, 2001, Vol. 2, Page(s): 72-91.
[22] M. J. Katz, “Fractals and Analysis of Waveforms”, computers in biology and medicine 18, 1988, Page(s): 145-156.
[23] T. Balli, R. Palaniappan, “A Combined Linear & Nonlinear Approach for Classification of Epileptic EEG Signals”, IEEE Engineering in Medicine and Biology Conference on Neural Engineering, 2009, Page(s): 714-717.
[24] K. Nakajima, T. Tamura, H. Miike, "Monitoring of heart and respiratory
rates by photoplethysmography using a digital filtering technique", Med. Eng. Phys.,1996, Vol.18, No.5, pp.365-372.
[25] Task Force of The European Society of Cardiology and The North American
Society of Pacing and Electrophysiology, "Heart rate variability. Standards of
measurement, physiological interpretation, and clinical use", Circulation, 1996,
Vol. 93, Page(s): 1043-1065.
[26] A. Granelli, I. Östman-Smith, "Noninvasive peripheral perfusion index as a possible tool for screening for critical left heart obstruction." Acta Paediatrica, 2007, Page(s): 1455-1459.
[27] C. D. Katsis, N. Katertsidis, G. Ganiatsas, D. I. Fotiadis, “Toward Emotion Recognition in Car-Racing Drivers: A Biosignal Processing Approach”, IEEE Transactions on Systems, Vol. 38, No. 3, May 2008, Page(s): 502-512.
[28] J. Holland, “Adaptation in Natural an Artificial System”, University of Michigan Press, 1975.
[29] 陳順宇, “多變量分析(第二版)”, 華泰書局, 2000.
[30] Alvin Lu, “獨立成份分析”, https://www.youtube.com/watch?v=3-mhd5C-cpU
[31] Z. Koldovský, P. Tichavský, E. Oja, “Efficient Variant of Algorithm FastICA for Independent Component Analysis Attaining the Cramér-Rao Lower Bound”, IEEE Transactions on Neural Networks, Vol. 17, No. 5, September 2006, Page(s): 1265-1277.
[32] D. Cai, X. He, J. Han, “SRDA: An Efficient Algorithm for Large-Scale Discriminant Analysis”, IEEE Transactions on Knowledge and Data Engineering, Vol. 20, No. 1, January 2008, Page(s): 1-12.
[33] X. He, S. Yan, Y. Hu, P. Niyogi, H.-J. Zhang, “Face Recognition Using Laplacianfaces”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, No. 3, March 2005, Page(s): 328-340.
[34] J. Ye, “Characterization of a Family of Algorithms for Generalized Discriminant Analysis on Undersampled Problems”, J. Machine Learning Research, Vol. 6, 2005, Page(s): 483-502.
[35] J. H. Friedman, “Regularized Discriminant Analysis”, J. Am. Statistical Assoc., Vol. 84, No. 405, 1989, Page(s): 165-175.
[36] Roger Jang, “Linear Disriminant Analysis”, http://mirlab.org/jang/books/dcpr/doc/lda.pdf
[37] J. Duchene, S. Leclercq, “An Optimal Transformation for Discriminant and Principal Component Analysis”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 10, No. 6, November 1988, Page(s): 978-983.
[38] Vapnik, V., “Statistical Learning Theory”, New York: Wiley, 1998.
[39] Shawe-Taylor, N.C.a.J., “An Introduction to Support Vector Machines and Other Kernel-based Learning Methods”, Cambridge University Press, 2000.
[40] C.-C. Chang, C.-J. Lin, “LIBSVM -- A Library for Support Vector Machines”, http://www.csie.ntu.edu.tw/~cjlin/libsvm/
[41] C.-W. Hsu, C.-C. Chang, C.-J. Lin, “A Practical Guide to Support Vector Classification”, Department of Computer Science, National Taiwan University, April 2010.
[42] “心率變異分析”, https://zh.wikipedia.org/wiki/%E5%BF%83%E7%8E%87%E8%AE%8A%E7%95%B0%E5%88%86%E6%9E%90
[43] C. K. Lee, S. K. Yoo, Y. J. Park, N. H. Kim, K. S. Jeong, B. C. Lee, “Using Neural Network to Recognize Human Emotions from Heart Rate Variability and Skin Resistance”, IEEE-EMBS, 2005, Page(s): 5523-5525.
[44]B.-J. Park, E.-H. Jang, S.-H. Kim, C. Huh, J.-H. Sohn, “Seven Emotion Recognition by Means of Particle Swarm Optimization on Physiological Signals”, 2012 9th IEEE International Conference on ICNSC, 2012, Page(s): 277-282.
[45] J. Cai, G. Liu, “The Research on Emotion Recognition from ECG Signal”, 2009 International Conference on ITCS, 2009, Page(s): 497-500.
[46] “Pulse Trace”, http://www.micromedical.co.uk/downloads/pdf/pulse2.pdf
[47] E.-H. Jang, B.-J Park, S.-H. Kim, “Identification of the Optimal Emotion Recognition Algorithm using Physiological Signals”, IEEE International Conference on ICEI, 2011, Page(s): 1-6.