由國際生物辨識組織的研究報告得知，利用人類生理特徵作為識別個人身分的生物辨識技術，將成為未來安全機制的主流。而在各種生物辨識方法中，人臉辨識在使用上最為簡便與人性化，若將其結合視訊監控系統，更可主動對範圍內的行人進行身分的比對與確認，也因此人臉辨識成為目前最熱門的研究主題之一。

在本文中，研究的重心是集中於以人臉外表特徵(appearance feature)為基礎之特徵擷取法上，其中最具代表性的方法是主成份分析法與獨立成份分析法。由於傳統的計算架構通常將整張人臉影像視為單一樣式(pattern)，而人臉局部特徵的統計資訊則有被忽略的可能，故後來有學者提出以次樣式(sub-pattern)為基礎的計算架構，試圖改善傳統方法之缺點。然而，我們進行一系列的實驗後發現，在某些情況下，次樣式特徵擷取架構的辨識率反而低於傳統方式。針對此一現象，本文進行深入的分析與探討，並提出合理的物理解釋。最後，為了同時考慮人臉整體與局部特徵的統計資訊，本文發展出一混合型特徵擷取架構，經由三個人臉資料庫測試的結果顯示，此架構有更佳的辨識性能。

此外，許多研究文獻指出：同一人的影像因受到光線變化所造成的改變，通常大於不同人影像間的差異，因此，光線變異對辨識性能的影響可見一斑。為了改善這個問題，本文提出一個光線補償方法，其作法是：先對影像中的局部區域進行直方圖等化，以強化局部特徵，接著透過鏡射與改良式影像平均的技術，融合左右兩邊的人臉資訊，以達到光線補償的效果。此方法具有以下特點：(a)架構簡單，容易實現；(b)能強化人臉主要特徵，並將非人臉主要特徵標準化；(c)能直接應用於單張影像，而無須預知光源方向，或使用多張影像進行訓練。

In a variety of biometric techniques, face recognition is a natural, non-intrusive, and user-friendly scheme. Especially, face recognition has the great advantage of being able to in places with large concourse of unaware visitors. After the Sept. 11, 2001 terrorist attacks, this active property of recognizing identity is not only garnering much more attention in both academia and industry, but also pushes face recognition turn into one of the most popular research topics.

In this dissertation, we mainly focus our attention on the investigation of appearance- based face recognition methods. Since the conventional appearance-based methods (e.g. Eigenfaces and Fisherfaces techniques) usually consider whole face image as a single pattern, some local statistical information of face image may be ignored. In order to emphasize the local facial features, some researches proposed their approaches which try to find the local appearance features by sub-pattern technique. Nevertheless, after a serious of experiments, we found that in some case the recognition accuracy of the sub-pattern based method is lower than that of the corresponding conventional method. In the former half of this dissertation, we presented our opinions to reasonably explain what causes this result. Then, by simultaneously considering global and local information of face images, we developed a novel hybrid approach for face recognition. The experimental results show that the proposed hybrid approach has better recognition performance than that obtained using other traditional methods.

In addition, some literatures pointed out that the intra-person variations caused by illumination change are often larger than the inter-person differences. Consequently, illumination variation is key factor that can significantly affect system face recognition performances. In the latter half of this dissertation, we proposed a novel shadow compensation method for dealing with this problem. Among others, this method has several advantages: (1) it is very simple so it is easily implemented in a real-time face recognition system; (2) it is able to reinforce key facial features and to standardize other parts of the face; (3) it can apply directly to single face image without any prior information of light source direction.

Abstract i
中文摘要 iii
誌謝 iv
Table of Contents v
List of Figures vii
List of Tables ix
Explanation of Symbols x

Chapter 1 Introduction 1
1.1 Background and Motivation 1
1.2 Survey of the Related Works 3
1.3 Dissertation Organization 7

Chapter 2 Overview of Two Basic Linear Projection Methods 9
2.1 Principal Component Analysis (PCA) 9
2.1.1 Calculating Eigenfaces 10
2.1.2 Using Eigenfaces to Classify a Face Image 10
2.2 Two-dimensional Principal Component Analysis (2DPCA) 11

Chapter 3 A Novel Hybrid Approach Based on Sub-pattern Technique and Whitened PCA for Face Recognition 13
3.1 Outline of This Chapter 13
3.2 Proposed Approaches 14
3.2.1 Image Partition for Constructing Sub-pattern Images 14
3.2.2 Sub-pattern Based PCA I (Sp-PCA I) 14
3.2.3 Sub-pattern Based PCA II (Sp-PCA II) 16
3.3 Experimental Results and Analysis 17
3.3.1 Yale Database - Expression Variant Test and Leave Out One Test 18
3.3.2 Yale B Database – Lighting Variant Test 19
3.3.3 Weizmann Database – Expression and Lighting Variant Test 20
3.3.4 Discussion 24
3.3.5 A Novel Hybrid Approach Combines PCA II and Sp-PCA I 24
3.4 Summary 25

Chapter 4 A Shadow Compensation Method Based on Facial Symmetry and Image Average for Face Recognition 27
4.1 Introduction 27
4.2 Proposed Method 30
4.3 Experiments and Results 35
4.3.1 Experiments on Yale B Face Database 35
4.3.2 Experiments on Weizmann Face Database 40
4.4 Discussion 41
4.4.1 The Mirror Operation’s Advantage and Limitation 41
4.4.2 The R-HEIA’s Robustness and Practicality 44
4.4.3 Comparisons with Other Well-known Methods 49

Chapter 5 Conclusions and Future Directions 50
5.1 Conclusions 50
5.2 Future Directions 51

References 52
Publications 58
Appendix – Mathematical Background of PCA 59

[1]A.K. Jain, A.A. Ross, “Introduction to biometrics,” in A.K. Jain, P. Flynn, A.A. Ross, editors, Handbook of Biometrics, pp. 1-22, Springer Science+Business Media, New York, 2008.
[2]A.F. Abate, M. Nappi, D. Riccio, G. Sabatino, “2D and 3D face recognition: a survey,” Pattern Recognition Letters, Vol. 28, pp. 1885-1906, 2007.
[3]S.Z. Li, A.K. Jain, “Chapter 1. Introduction,” in S.Z. Li, A.K. Jain, editors, Handbook of Face Recognition, pp. 1-11, Springer Science+Business Media, New York, 2005.
[4]C. Lin, K.C. Fan, “Triangle-based approach to the detection of human face,” Pattern Recognition, Vol. 34, pp. 1271-1284, 2001.
[5]I.S. Hsieh, K.C. Fan, C. Lin, “A statistic approach to the detection of human faces in color nature scene,” Pattern Recognition, Vol. 35, pp. 1583-1596, 2002.
[6]D. Zhang, W. Zuo, “Computational intelligence-based biometric technologies,” IEEE Computational Intelligence Magazine, Vol. 2, pp. 26-36, 2007.
[7]W. Zhao, R. Chellappa, R.J. Phillips, A. Rosenfeld, “Face recognition: a literature survey,” ACM Computing Surveys, Vol. 35, No. 4, pp. 399-458, 2003.
[8]R. Brunelli, T. Poggio, “Face recognition: features versus templates,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 15, No. 10, pp. 1042-1052, 1993.
[9]A. Lanitis, C.J. Taylor, T.F. Cootes, “Automatic interpretation and coding of face images using flexible models,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 19, No.7, pp. 743-756, 1997.
[10]M. Turk, A. Pentland, “Eigenfaces for recognition,” Journal of Cognitive Neuroscience, Vol. 3, No.1, pp. 71-86, 1991.
[11]M. Turk, A. Pentland, “Face recognition using Eigenfaces,” in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 586-591, 1991.
[12]M.S. Bartlett, H.M. Lades, T.J. Sejnowski, “Independent component re- presentation for face recognition,” in: Proceedings of SPIE Symposium on Electronic Image: Science and Technology, pp. 528-539, 1998.
[13]M.S. Bartlett, J.R. Movellan, T.J. Sejnowski, “Face recognition by independent component analysis,” IEEE Trans. on Neural Networks, Vol. 13, No. 6, pp. 1450-1464, 2002.
[14]A.J. Bell, T.J. Sejnowski, “An information-maximization approach to blind separation and blind deconvolution,” Neural Computation, Vol. 7, No.6, pp. 1129-1159, 1995.
[15]A.J. Bell, T.J. Sejnowski, “The independent components of natural scenes are edge filters,” Vision Research, Vol. 37, No.23, pp. 3327-3338, 1997.
[16]J. Yang, D. Zhang, J.Y. Yang, “Is ICA significantly better than PCA for face recognition?” in: Proceedings of the IEEE International Conference on Computer Vision, Vol. 1, pp. 198-203, 2005.
[17]J. Yang, D. Zhang, J.Y. Yang, “Constructing PCA baseline algorithms to reevaluate ICA-based face-recognition performance,” IEEE Trans. on Systems, Man, and Cybernetics, Part B: Cybernetics, Vol. 37, No. 4, pp. 1015-1021, 2007.
[18]R. Gottumukkal, V.K. Asari, “An improved face recognition technique based on modular PCA approach,” Pattern Recognition Letters, Vol. 25, No. 4, pp. 429 -436, 2004.
[19]S. Chen, Y. Zhu, “Subpattern-based principal component analysis,” Pattern Recognition, Vol. 37, pp. 1081-1083, 2004.
[20]K. Tan, S. Chen, “Adaptively weighted sub-pattern PCA for face recognition,” Neurocomputing, Vol. 64, pp. 505-511, 2005.
[21]J. Yang, D. Zhang, A.F. Frangi, J.Y. Yang, “Two-dimensional PCA: a new approach to appearance-based face representation and recognition,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 26, No. 1, pp. 131-137, 2004.
[22]D. Zhang, Z.H. Zhou, “(2D)2PCA: two-directional two-dimensional PCA for efficient face recognition,” Neurocomputing, Vol. 69, pp. 224-231, 2005.
[23]H. Kong, L. Wang, E.K. Teoh, X. Li, J.G. Wang, R. Venkateswarlu, “Generalized 2D principal component analysis for face image representation and recognition,” Neural Networks, Vol. 18, pp. 585-594, 2005.
[24]A. Xu, X. Jin, Y. Jiang, P. Guo, “Complete two-dimensional PCA for face recognition,” in: Proceedings of the 18th International Conference on Pattern Recognition, Vol. 3, pp. 481-484, 2006.
[25]M. Li, B. Yuan, “2D-LDA: a statistical linear discriminant analysis for image matrix,” Pattern Recognition Letters, Vol. 26, pp. 527-532, 2005.
[26]H. Xiong, M.N.S. Swamy, M.O. Ahmad, “Two-dimensional FLD for face recognition,” Pattern Recognition, Vol. 38, pp. 1121-1124, 2005.
[27]J. Yang, D. Zhang, X. Yong, J.Y. Yang, “Two-dimensional discriminant transform for face recognition,” Pattern Recognition, Vol. 38, pp. 1125-1129, 2005.
[28]X.Y. Jing, H.S. Wong, D. Zhang, “Face recognition based on 2D Fisherface approach,” Pattern Recognition, Vol. 39, pp. 707-710, 2006.
[29]S. Noushath, G.H. Kumar, P. Shivakumara, “(2D)2LDA: an efficient approach for face recognition,” Pattern Recognition, Vol. 39, pp. 1396-1400, 2006.
[30]Y. Qi, J. Zhang, “(2D)2PCALDA: an efficient approach for face recognition,” Applied Mathematics and Computation, Vol. 213, pp. 1-7, 2009.
[31]W. Yang, X. Yan, L. Zhang, C. Sun, “Feature extraction based on fuzzy 2DLDA,” Neurocomputing, Vol. 73, pp. 1556-1561, 2010.
[32]J. Yang, C. Liu, “Horizontal and vertical 2DPCA-based discriminant analysis for face verification on a large-scale database,” IEEE Trans. on Information Forensics and Security, Vol. 2, No. 4, pp. 781-792, 2007
[33]B. Niu, Q. Yang, S.C.K. Shiu, S.K. Pal, “Two-dimensional Laplacianfaces method for face recognition,” Pattern Recognition, Vol. 41, pp. 3237-3243, 2008.
[34]L. Sirovich, M. Kirby, “Low-dimensional procedure for the characterization of human faces,” Journal of Optical Society of America, Vol. 4, No. 3, pp. 519-524, 1987.
[35]M. Kirby, L. Sirovich, “Application of the KL procedure for the characterization of human faces,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 12, No. 1, pp. 103-108, 1990.
[36]Y. Xu, D. Zhang, J. Yang, J.Y. Yang, “An approach for directly extracting features from matrix data and its application in face recognition,” Neurocomputing, Vol. 71, pp. 1857-1865, 2008.
[37]Yale face database (http://cvc.yale.edu/projects/yalefaces/yalefaces.html).
[38]A.S. Georghiades, P.N. Belhumeur, D.J. Kriegman, “From few to many: illumination cone models for face recognition under variable lighting and pose,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 23, No. 6, pp. 643-660, 2001.
[39]Yale B face database (http://cvc.yale.edu/projects/yalefacesB/yalefacesB.html).
[40]Weizmann face database (http://www.faculty.idc.ac.il/moses/).
[41]K.C. Lee, J. Ho, D.J. Driegman, “Acquiring linear subspaces for face recognition under variable lighting,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 27, No. 5, pp. 684-695, 2005.
[42]S.I. Choi, C. Kim, C.H. Choi, “Shadow compensation in 2D images for face recognition,” Pattern Recognition, Vol. 40, pp. 2118-2125, 2007.
[43]Y. Adini, Y. Moses, S. Ullman, “Face recognition: the problem of compensating for changes in illumination direction,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 19, No. 7, pp. 721-731, 1997.
[44]W. Chen, M.J. Er, S. Wu, “Illumination compensation and normalization for robust face recognition using discrete cosine transform in logarithm domain,” IEEE Trans. on Systems, Man, and Cybernetics – Part B: Cybernetics, Vol. 36, No. 2, pp. 458-466, 2006.
[45]J. Ruiz-del-Solar, J. Quinteros, “Illumination compensation and normalization in eigenspace-based face recognition: a comparative study of different preprocessing approaches,” Pattern Recognition Letters, Vol. 29, pp. 1966-1979, 2008.
[46]X. Zou, J. Kittler, K. Messer, “Illumination invariant face recognition: a survey,” in: Proceedings of the First IEEE International Conference on Biometrics: Theory, Application, and Systems, 2007.
[47]S. Shan, W. Gao, B. Cao, D. Zhao, “Illumination normalization for robust face recognition against varying lighting conditions,” in: Proceedings of the IEEE International Workshop on Analysis and Modeling of Face and Gestures, pp. 157-164, 2003.
[48]X. Xie, K.M. Lam, “Face recognition under varying illumination based on a 2D face shape model,” Pattern Recognition, Vol. 38, pp. 221-230, 2005.
[49]Y.J. Song, Y.G. Kim, U.D. Chang, H.B. Kwon, “Face recognition robust to left/right shadows: facial symmetry,” Pattern Recognition, Vol. 39, pp. 1542- 1545, 2006.
[50]A. Shashua, T. Riklin-Raviv, “The quotient image: class-based re-rendering and recognition with varying illuminations,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 23, No. 2, pp. 129-139, 2001.
[51]H. Wang, S.Z. Li, Y. Wang, “Face recognition under varying lighting conditions using self quotient image,” in: Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 819-824, 2004.
[52]T. Chen, W. Yin, X.S. Zhou, D. Comaniciu, T.S. Huang, “Illumination normalization for face recognition and uneven background correction using total variation based image models,” in: Proceedings of the IEEE Computer Society Conference on CVPR, Vol. 2, pp. 532-539, 2005.
[53]R. Gross, V. Brajovic, “An image preprocessing algorithm for illumination invariant face recognition,” in: Proceedings of the Fourth International Conference on Audio- and Video-based Biometric Person Authentication, LNCS 2688, pp. 10-18, 2003.
[54]X. Tan, B. Triggs, “Enhanced local texture feature sets for face recognition under difficult lighting conditions,” in: Proceedings of the 2007 IEEE International Workshop on Analysis and Modeling of Face and Gestures, LNCS 4778, pp. 168- 182, 2007.
[55]R. Basri, D.W. Jacobs, “Lambertian reflectance and linear subspaces,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 25, No. 2, pp. 218-233, 2003.
[56]J.C. Lee, J. Ho, D. Kriegman, “Nine points of light: acquiring subspaces for face recognition under variable lighting,” in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, pp. 519-526, 2001.
[57]R. Jenkins, A.M. Burton, “100% accuracy in automatic face recognition,” Science, Vol. 319, pp. 435, 2008.
[58]X. Tan, S. Chen, Z.H. Zhou, F. Zhang, “Face recognition from a single image per person: a survey,” Pattern Recognition, Vol. 39, pp. 1725-1745, 2006.
[59]P.C. Hsieh, P.C. Tung, “A novel hybrid approach based on sub-pattern technique and whitened PCA for face recognition,” Pattern Recognition, Vol. 42, pp. 978- 984, 2009.
[60]S. Bhavani, A. Thawani, V. Sridhar, K.R. Ramakrishnan, “Illumination invariant face recognition for frontal faces using modified census transform,” in: Proceedings of the IEEE Region 10 International Conference on TENCON, 2007.
[61]L. Wang, X. Wang, X. Zhang, J. Feng, “The equivalence of two-dimensional PCA to line-based PCA,” Pattern Recognition Letters, Vol. 26, pp. 57-60, 2005.
[62]L. Wang, X. Wang, J. Feng, “On image matrix based feature extraction algorithms,” IEEE Trans. on Systems, Man, and Cybernetics – Part B: Cybernetics, Vol. 36, No. 1, pp. 194-197, 2006.
[63]Q. Gao, “Is two-dimensional PCA equivalent to a special case of modular PCA?” Pattern Recognition Letters, Vol. 28, pp. 1250-1251, 2007.
[64]Q. Gao, L. Zhang, D. Zhang, J. Yang, “Common on “on image matrix based feature extraction algorithms”,” IEEE Trans. on Systems, Man, and Cybernetics – Part B: Cybernetics, Vol. 37, No. 5, pp. 1373-1374, 2007.
[65]R.O. Duda, P.E. Hart, D.G. Stork, Pattern Classification, pp.114-117, Wiley, New York, 2001.