臺灣博碩士論文加值系統

在本篇論文中，我們提出了一個具有權重的區域方向性描述紋理方法並應用於臉部表情辨識。我們將臉部影像切割成若干區塊，利用方向性資訊來計算微小樣式的結構，將各區塊微小樣式結構的分佈以權重的概念來表示表情的紋理結構特徵，搭配邊緣偵測後的水平投影來輔助判斷，最終利用支持向量機來做臉部表情辨識。我們一共使用三個資料庫來進行臉部表情辨識實驗。JAFFE 資料庫和Cohn-Kanade 資料庫，各包括七種表情，分別為中性表情、開心、傷心、驚訝、害怕、生氣和厭惡。另外我們也使用CMU-PIE 資料庫，其中包括了中性表情和微笑共兩種表情。由實驗結果可以看出，本篇提出的方法得到的結果較其他方法好。

In this thesis, a weighted local directional texture descriptor is proposed which is applied in facial expression recognition. We partition the face images into several regions, calculate the distribution of the structure of micro patterns using the directional information for each region to represent the facial texture feature with the concept of weight, and the horizontal projection of the result of edge detection is also used to help identify facial expression. Finally, SVM (support vector machine) is used to classify the facial expression. We use three databases to test the performance of the proposed facial expression recognition system. There are seven expressions in both JAFFE database and Cohn-Kanade database, which are neutral, happiness, sadness, surprise, fear, anger, and disgust. For CMU-PIE database, there are two expressions,
which are neutral and smiling. As the experimental result, the proposed method outperforms the other methods in facial expression recognition.

摘要. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Overview of Facial Expression Recognition . . . . . . . . . . . . . . . . 1
1.2 Organization of This Thesis . . . . . . . . . . . . . . . . . . . . . . . . 3
2 Background and Related Works . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1 Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1.1 Face Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1.2 Bicubic interpolation . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1.3 Histogram Equalization . . . . . . . . . . . . . . . . . . . . . . 6
2.2 Feature Extraction Algorithm Analysis . . . . . . . . . . . . . . . . . . 8
2.2.1 Gabor Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2.2 Local Binary Pattern . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2.3 Local Directional Pattern . . . . . . . . . . . . . . . . . . . . . 14
2.3 Classifier Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3.1 K-Nearest Neighbors algorithm . . . . . . . . . . . . . . . . . . 16
2.3.2 Sparse Representation Classier . . . . . . . . . . . . . . . . . . 17
2.3.3 Support Vector Machine . . . . . . . . . . . . . . . . . . . . . . 19
2.4 The Methods of System Evaluation . . . . . . . . . . . . . . . . . . . . 22
2.4.1 K-fold Cross Validation . . . . . . . . . . . . . . . . . . . . . . 23
2.4.2 Leave-one-out Cross Validation . . . . . . . . . . . . . . . . . . 23
3 The Proposed System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.1 System Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.2 Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.3 Facial Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.4 Expression Classification . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.1 Database and Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.1.1 Cohn-Kanade . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.1.2 Japanese Female Facial Expression . . . . . . . . . . . . . . . . 38
4.1.3 CMU-PIE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.2 Experimental Analysis and Comparison . . . . . . . . . . . . . . . . . . 43
5 Conclusions and Future Works . . . . . . . . . . . . . . . . . . . . . . . . . . 46
5.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
5.2 Future Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

[1] F. Ahmed and M. Kabir, Directional ternary pattern ( dtp) for facial expression recognition, in Consumer Electronics (ICCE), 2012 IEEE International Conference on, Jan 2012, pp. 265-266.
[2] N. S. Altman, An introduction to kernel and nearest-neighbor nonparametric regression, The American Statistician, pp. 175-185.
[3] C. J. Burges, A tutorial on support vector machines for pattern recognition, in Data Mining and Knowledge Discovery, vol. 2, 1998, pp. 121-167.
[4] C.-C. Chang and C.-J. Lin, LIBSVM: A library for support vector machines,ACM Transactions on Intelligent Systems and Technology, vol. 2, pp. 27:1-27:27, 2011, software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
[5] S. C. Chen and S. C. Tai, Automatic facial expression recognition system using neural network, 2005.
[6] T. Chen and F. Su, Facial expression recognition via gabor wavelet and structured sparse representation, in Network Infrastructure and Digital Content (IC-NIDC),
2012 3rd IEEE International Conference on, Sept 2012, pp. 420-424.
[7] S. Dongcheng, C. Fang, and D. Guangyi, Facial expression recognition based on gabor wavelet phase features, in Image and Graphics (ICIG), 2013 Seventh International Conference on, July 2013, pp. 520-523.
[8] D. Gabor, Theory of communication, J. Inst. Electr. Eng, vol. 93, no. 26, pp.429-457, 1946.
[9] C. Garcia and G. Tziritas, Face detection using quantized skin color regions merging and wavelet packet analysis, Multimedia, IEEE Transactions on, vol. 1,no. 3, pp. 264-277, Sep 1999.
[10] R. C. Gonzalez and R. E. Woods, Digital Image Processing, 2nd, Ed. Reading, MA: Addison-Wesley, 1992.
[11] T. Jabid, M. Kabir, and O. Chae, Local directional pattern (ldp) for face recognition, in Consumer Electronics (ICCE), 2010 Digest of Technical Papers International Conference on, Jan 2010, pp. 329-330.
[12] T. Kanade, J. Cohn, and Y. Tian, Comprehensive database for facial expression analysis, in Automatic Face and Gesture Recognition, 2000. Proceedings. Fourth IEEE International Conference on, 2000, pp. 46-53.
[13] J.-Y. Kim, L.-S. Kim, and S.-H. Hwang, An advanced contrast enhancement using partially overlapped sub-block histogram equalization, Circuits and Systems for Video Technology, IEEE Transactions on, vol. 11, no. 4, pp. 475-484, Apr 2001.
[14] Kohavi and Ron, A study of cross-validation and bootstrap for accuracy estimation and model selection, in Proceedings of the Fourteenth International Joint Conference on Arti cail Intelligence, 1995.
[15] G. Lipori, Manual annotations of facial ducial points on the cohn kanade database, Milan Patent. [Online]. Available: http://lipori.dsi.unimi.it/download.html.
[16] M. Lyons, S. Akamatsu, M. Kamachi, and J. Gyoba, Coding facial expressions with gabor wavelets, in Automatic Face and Gesture Recognition, 1998. Proceedings. Third IEEE International Conference on, Apr 1998, pp. 200-205.
[17] M. J. Lyons, J. Budynek, and S. Akamatsu, Automatic classification of single facial images, Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 21, no. 12, pp. 1357-1362, Dec 1999.
[18] A. Mehrabian, Communication without words, vol. 2, p. 53 56, 1968.
[19] T. Ojala, M. Pietikainen, and T. Maenpaa, Multiresolution gray-scale and rotation invariant texture classification with local binary patterns, Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 24, no. 7, pp. 971-987, Jul 2002.
[20] P.Ekman and W.Friesen, Facial action coding system: A technique for measurement of facial movement, Consulting Psychologists Press, 1978.
[21] A. Ramirez Rivera, R. Castillo, and O. Chae, Local directional number pattern for face analysis: Face and expression recognition, Image Processing, IEEE Transactions on, vol. 22, no. 5, pp. 1740-1752, May 2013.
[22] R.Keys, Cubic convolution interpolation for digital image processing, Acoustics, Speech and Signal Processing, IEEE Transactions on, vol. 29, no. 6, pp. 1153-1160, 1981.
[23] C. Shan, S. Gong, and P. W. McOwan, Facial expression recognition based on local binary patterns: A comprehensive study, Image Vis. Comput, vol. 27, no. 6, pp. 803-816, 2009.
[24] T. Sim, S. Baker, and M. Bsat, The cmu pose, illumination, and expression (pie) database, in Proc. 5th Int. Conf. Autom, Face Gesture Recognition, 2002, pp. 1-9.
[25] P. Viola and M. J. Jones, Robust real-time face detection, International Journal of Computer Vision, vol. 57, no. 2, pp. 137-154, 2004.
[26] J. Wright, A. Yang, A. Ganesh, S. Sastry, and Y. Ma, Robust face recognition via sparse representation, Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 31, no. 2, pp. 210-227, Feb 2009.