臺灣博碩士論文加值系統

隨著日益增加的應用，自動化的臉部表情辨識已成為相當熱門的研究主題。臉部表情辨識主要透過人臉影像進行臉部特徵擷取，再將取出的特徵經由分類器進行表情的分類。本論文提出一個以改良式區域方向圖樣與澤尼克動差為基準之特徵擷取方法，對臉部表情影像擷取具有辨識能力的特徵，再將取得的臉部特徵以支持向量機進行分類。實驗結果證實，我們提出的方法在著名的Cohn-Kanade、JAFFE以及TFEID表情資料庫中，有更好的辨識效能。

Automatic facial expression recognition has been a subject of investigation in the last years due to the great number of potential applications. In this paper, we proposed a facial expression representation approach which combines improved Local Directional Pattern (iLDP) and Zernike moments. To extract the discriminative facial features from facial images, and then these features are classified by the support vector machine. Experimental results are provided to illustrate the proposed approach is an effective method on the Cohn-Kanade, JAFFE, and TEFID databases.

摘要 i
ABSTRACT ii
致謝 iii
目錄 iv
圖目錄 vi
表目錄 vii
第一章 導論 1
1.1 研究動機與目的 1
1.2 研究方法與架構 2
第二章 表情辨識相關研究 3
2.1 區域二元圖樣及其變形 4
2.2 動差運算子 5
第三章 研究方法 7
3.1 結合動差運算子與區域紋理特徵之表情辨識系統 7
3.2 改良式區域方向圖樣 9
3.2.1區域方向圖樣 10
3.2.2直方圖等化 12
3.2.3改良式區域方向圖樣 12
3.2.4局部紋理特徵擷取 14
3.3 澤尼克動差 15
3.4 支持向量機 16
3.5 結合方法 17
第四章 實驗結果 19
4.1 實驗環境 19
4.2 實驗結果與分析 21
4.2.1 實驗一 23
4.2.2 實驗二 24
4.2.3 實驗三 25
4.2.4 實驗四 28
4.2.5 實驗五 34
4.2.6 實驗六 39
4.2.7 實驗七 43
4.2.8 實驗八 45
4.2.9 實驗九 47
第五章 結論與未來工作 48
參考文獻 50

[1]A. Mehrabian, Silent Messages, Belmont , CA : Wadsworth, pp. 44-45, 1971.
[2]Y. Tian, T. Kanade, and J. Cohn, “Recognizing action units for facial expression analysis,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 97-115, Feb. 2001.
[3]R. E. Kaliouby and P. Robinson, “Real-time inference of complex mental states from facial expressions and head gestures,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, 2005, pp. 181-200.
[4]M. Pantic and L. J. M. Rothkrantz, “Facial action recognition for facial expression analysis from static face images,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 34, no. 3, pp. 1449-1461, 2004.
[5]M. Pantic and I. Patras, “Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 36, no. 2, pp. 433-449, 2006.
[6]M. Pantic and L. Rothkrantz, “Expert system for automatic analysis of facial expression,” Image and Vision Computing, vol. 18, no. 11, pp. 881-905, 2000.
[7]Y. Tian, “Evaluation of face resolution for expression analysis,” in Proc. Computer Vision and Pattern Recognition Workshop on Face Processing in Video, 2004, pp. 82-88.
[8]M. S. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I. Fasel, and J. Movellan, “Recognizing facial expression: machine learning and application to spontaneous behavior,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, 2005, vol. 2, pp. 568-573.
[9]S. Bashyal and G. K. Venayagamoorthy, “Recognition of facial expressions using Gabor wavelets and learning vector quantization,” Engineering Applications of Artificial Intelligence, vol. 21, no. 7, pp. 1056-1064, 2008.
[10]M. Yeasin, B. Bullot, and R. Sharma, “From facial expression to level of interests: a spatio-temporal approach,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, 2004, vol. 2, pp. 922-927.
[11]J. Hoey and J. J. Little, “Value directed learning of gestures and facial displays,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, 2004, vol. 2, pp. 1026-1033.
[12]P. Ekman and W. V. Friesen, “Facial Action Coding System,” Consulting Psychologists Press, College Avenue, Palo Alto, CA, 1978.
[13]M. Valstar, I. Patras, and M. Pantic, “Facial action unit detection using probabilistic actively learned support vector machines on tracked facial point data,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, 2005, vol. 3, pp. 76-84.
[14]M. Valstar and M. Pantic, “Fully automatic facial action unit detection and temporal analysis,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, 2006, pp. 149-156.
[15]J. Hamm, C. G. kohler, R. C. Gur, and R. Verma, “Automated facial action coding system for dynamic analysis of facial expressions in neuropsychiatric disorders,” Journal of Neuroscience Methods, vol. 200, no. 2, pp. 237-256, Jun. 2011.
[16]Y. L. Xue, X. Mao, and F. Zhang, “Beihang university facial expression
database and multiple facial expression recognition,” in Proc. International Conference on Machine Learning and Cybernetics, 2006, pp. 3282-3287.
[17]M. Turk and A. P. Pentland, “Face recognition using eigenfaces,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, 1991, pp. 586-591.
[18]M. S. Bartlett, J. R. Movellan, and T. J. Sejnowski, “Face recognition by independent component analysis,” IEEE Trans. Neural Network, vol. 6, no. 13, pp. 1450-1464, 2002.
[19]G. Donato, M. Bartlett, J. Hager, P. Ekman, and T. Sejnowski, “Classifying facial actions,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 10, pp. 974-989, 1999.
[20]X. Feng, M. Pietikäinen, and A. Hadid, “Facial expression recognition with local binary patterns and linear programming,” Pattern Recognition and Image Analysis, vol. 15, no. 2, pp. 546-548, 2005.
[21]X. Feng, A. Hadid, and M. Pietikäinen, “A coarse-to-fine classification scheme for facial expression recognition,” in Proc. International Conference on Image Analysis and Recognition, 2004, vol. 3212, pp. 668-675.
[22]S. M. Lajevardi and Z. M. Hussain, “Facial expression recognition using Log-Gabor filters and local binary pattern operators,” in Proc. International Conference on Communication, Computer and Power, Feb. 2009, pp. 15-18.
[23]C. Shan, S. Gong, and P. W. McOwan, “Conditional mutual information based boosting for facial expression recognition,” in Proc. British Machine Vision Conference, Sep. 2005, vol. 1, pp. 399-408.
[24]C. Shan and T. Gritti, “Learning discriminative LBP-histogram bins for facial expression recognition,” in Proc. British Machine Vision Conference, 2008, pp. 2-12.
[25]C. Shan, S. Gong, and P. W. McOwan, “Facial expression recognition based on local binary patterns: A comprehensive study,” Image and Vision Computing, vol. 27, no. 6, pp. 803-816, 2009.
[26]T. Ahonen, A. Hadid, and M. Pietikäinen, “Face recognition with local binary patterns,” in Proc. European Conference on Computer Vision, T. Pajdla and J. Matas, Ed., LNCS 3021, 2004, pp. 469-481.
[27]O. G. Cula and K. J. Dana, “Compact representation of bidirectional texture functions,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, 2001, vol. 1, pp. 1041-1047.
[28]T. Leung and J. Malik, “Representing and recognizing the visual appearance of materials using three-dimensional textons,” International Journal of Computer Vision, vol. 43, no. 1, pp. 29-44, 2001.
[29]C. Zhu, C.-E. Bichot, and L. Chen, “Multi-scale color local binary patterns for visual object classes recognition,” in Proc. International Conference on Pattern Recognition, 2010, pp. 3065-3068.
[30]G. W. Yuan, Y. Gao, D. Xu, and M. R. Jiang, “A new background subtraction method using texture and color information,” in Proc. International Conference on Intelligent Computing, 2011, pp. 541-548.
[31]P. Viola and M. J. Jones, “Robust real-time face detection,” International Journal of Computer Vision, vol. 57, no. 2, pp. 137-154, 2004.
[32]J. Meng, Y. Gao, X. Wang, T. Lin, and J. Zhang, “Face recognition based on local binary patterns with threshold,” in Proc. IEEE International Conference on Granular Computing, 2010, pp. 352-356.
[33]M. Heikkiläa, M. Pietikäinen, and C. Schmid, “Description of interest regions with local binary patterns,” Pattern Recognition, vol. 42, no. 3, pp. 425-436, 2009.
[34]T. Jabid, M. H. Kabir, and O. Chae, “Robust facial expression recognition
based on local directional pattern,” ETRI Journal, vol. 32, no. 5, pp.784-794,
2010.
[35]M. Kabir, T. Jabid, and O. Chae, “Local directional patterm varinace (LDPv): A robust feature descriptor for facial expression recognition,” International Arab Journal of Information Technology, vol. 9, no. 4, pp. 382-391, 2012.
[36]J. A. R. Castillo, A. R. Rivera, and O. Chae, “Facial expression recognition
based on local sign directional pattern,” in Proc. IEEE International Conference on Image Processing, 2012, pp. 2613-2616.
[37]C. Y. Wee and R. Paramesran, “On the computational aspects of Zernike moments,” Image and Vision Computing, vol. 25 , no. 6, pp. 967-980, Jun. 2007.
[38]S. K. Hwang and W. Y. Kim, “A novel approach to the fast computation of Zernike moments,” Pattern Recognition, vol. 35, no. 12, pp. 2905-2911, Dec. 2002.
[39]J. Revaud, G. Lavou´e, and A. Baskurt, “Optimal similarity and rotation angle retrieval using Zernike moments,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 4, pp. 627-636, Apr. 2009.
[40]E. Sariyanidi, V. Dagli, S. C. Tek, B. Tunc, and M. Gokmen, “Local Zernike moments: A new representation for face recognition,” in Proc. IEEE International Conference on Image Processing, Oct. 2012, pp. 585-588.
[41]S. M. Lajevardi and Z. M. Hussain, “Higher order orthogonal moments for invariant facial expression recognition,” Digital Signal Processing, vol. 20, no. 6, pp. 1771-1779, Dec. 2010.
[42]H. R. Kanan and M. Ahmady, “Recognition of facial expressions using locally weighted and adjusted order pseudo Zernike moments,” in Proc. IEEE Conference on Pattern Recognition, 2012, pp. 3419-3422.
[43]M. X. Zheng and X. B. Jia, “Joint LBP and DCT model for visual speech,” in J. Luo, Ed., Affective Computing and Intelligent Interaction, AISC 137, 2012, pp. 101-107, 2012.
[44]C. C. Chang and C. J. Lin, LIBSVM: a library for support vector machines. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm, 2001.
[45]M. Pateraki, H. Baltzakis, P. Kondaxakis, and P. Trahanias, “Tracking of facial features to support human-robot interaction,” in Proc. IEEE International Conference on Robotics and Automation, May. 2009, pp. 3755-3760.
[46]M. K. Hu, “Visual pattern recognition by moment invariants,” IEEE Transactions on Information Theory, vol. 8, no. 2, pp. 179-187, 1962.
[47]M. R. Teague, “Image analysis via the general theory of moments,” Journal of the Optical Society of America, vol. 70, pp. 920-930, Aug, 1980.
[48]C. Cortex and V. Vapnik, “Support-vector networks,” International Journal of Machine Learning, vol. 20, no. 3, pp. 273-297, 1995.
[49]莊蕙如，萃取表情重要特徵進行表情辨識與表情強度分析。國立清華大學資訊工程學系碩士論文，98年8月。
[50]莊順旭，基於權重式區域方向特徵和區域二元化形態之表情辨識方法。中華大學資訊工程學系碩士論文，99年8月。
[51]劉懿萱，植基於全域與局部紋理特徵之臉部表情辨識。國立高雄大學電機工程學系碩士論文，101年6月。
[52]柯忠宏，植基於二階段區域紋理特徵之臉部表情辨識。國立高雄大學電機工程學系碩士論文，102年6月。