臺灣博碩士論文加值系統

多角度人臉辨識電腦視覺應用裡相當挑戰性的主題。本論文主要探討兩大情景，一是常見的對每位註冊者使用一張正面人臉影像，測試影像為註冊者不同視角之影像。另一則採用正面RGB-D影像於註冊，測試影像和前者相同為不同視角之RGB影像。第二種情景模擬僅註冊階段具備RGB-D攝影機，並可對所有不含深度資訊之RGB影像進行人臉辨識。本論文對第一情景提出兩種方法架構：基於全人臉與局部人臉區域的方法。前者為一人臉重建方法之延伸，透過採用不同數目之人臉特徵、使用多個參考模型來改善其效能。後者著重於對局部人臉區域進行重建，人臉區域是根據角度不變性的特徵所定義，辨識考慮各個區域在不同視角下的比對。此一基於局部人臉區域，進行三維重建用於多角度人臉辨識，在過去的文獻當中相當少被探討。在第二種基於RGB-D影像的情景中，儘管與第一種情景在辨識部分存在相似之處，其創新之部分在於處理深度讀取過程所產生之取樣雜訊。本論文提出對失真深度表面平滑之方法，此方法可大幅改善辨識效能。所有提出之方法皆評估於現有資料庫上，其效能與現有之方法相比不相上下。

Face recognition across pose is a challenging problem in computer vision. Two scenarios are considered in this thesis. One is the common setup with one single frontal facial image of each subject in the gallery set and the images of other poses in the probe set. The other considers a RGB-D image of the frontal face for each subject in the gallery, but the probe set is the same as in the previous case that only contains RGB images of other poses. The second scenario simulates the case that RGB-D camera can be available for user registration only and recognition can be performed on regular RGB images without the depth channel. Two approaches are proposed for handling the first scenario, one is holistic and the other is component-based. The former is extended from a face reconstruction approach and improved with different sets of landmarks for alignment and multiple reference models considered in the reconstruction phase. The latter focuses on the reconstruction of facial components obtained by the pose-invariant landmarks, and the recognition with different components considered at different poses. Such a component-based reconstruction for handling cross-pose recognition is rarely seen in the literature. Although the approach for handling the second scenario, i.e., the RGB-D based recognition, is partially similar to the approach for handling the first scenario, the novelty is on the handling of the depth readings corrupted by quantization noise, which are often encountered when the face is not close enough to the RGB-D camera at registration. An approach is proposed to resurface the corrupted depth map and substantially improve the recognition performance. All of the proposed approaches are evaluated on benchmark databases and proven comparable to state-of-the-art approaches.

中文摘要 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i
英文摘要 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii
誌謝 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
目錄 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
圖目錄 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi
表目錄 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Reviews on 2D-based methods . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Reviews on 3D-based methods . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Thesis organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2 Holistic 3D Face Reconstruction using A Single Reference Model . . . . . . . 8
2.1 3D Face Reconstruction Formulation . . . . . . . . . . . . . . . . . . . . 8
2.1.1 Reference Model Surface Rendering and Parameter Estimation . . 9
2.1.2 Irradiance Evaluation using Constrained Minimization . . . . . . 10
2.2 Cross-Pose Recognition Using SRC . . . . . . . . . . . . . . . . . . . . 12
2.2.1 Generation of Model-based Training Images . . . . . . . . . . . 12
2.2.2 Pose-Clustered Recognition with SRC . . . . . . . . . . . . . . . 14
2.3 An Extensive Experimental Study . . . . . . . . . . . . . . . . . . . . . 16
2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3 3D Reconstruction of Facial Components . . . . . . . . . . . . . . . . . . . . 23
3.1 Discriminative Response Map Fitting (DRMF) method . . . . . . . . . . 24
3.2 Landmark-based Component Reconstruction using Ethnicity and Gender
Reference Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.3 Component-based recognition with SRC . . . . . . . . . . . . . . . . . . 26
3.3.1 Component alignment and illumination normalization . . . . . . 27
3.3.2 Features and matching strategy . . . . . . . . . . . . . . . . . . . 28
3.4 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4 RGB-D based Face Reconstruction and Recognition . . . . . . . . . . . . . . . 33
4.1 Literature reviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.2 3D Reconstruction with RGB-D images . . . . . . . . . . . . . . . . . . 35
4.2.1 Resurfacing of Facial Depth Corrupted by Quantization Noise . . 36
4.2.2 Iterative Face Surface Estimation . . . . . . . . . . . . . . . . . 38
4.3 Landmark-assisted and SRC-based Recognition . . . . . . . . . . . . . . 38
4.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
5 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
5.1 Scenario I: 3D Face Reconstruction on RGB Image . . . . . . . . . . . . 50
5.2 Scenario II: 3D Face Reconstruction on RGB-D Images . . . . . . . . . . 51
5.3 Recognition and Feature Selection . . . . . . . . . . . . . . . . . . . . . 52
參考文獻 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

[1] R. Gross, I. Matthews, and S. Baker, ``Appearance-based face recognition and light fields,'' TPAMI, vol. 26, pp. 449--465, 2004.
[2] A. Li, S. Shan, and W. Gao, ``Coupled bias-variance tradeoff for cross-pose face recognition,'' IEEE Transactions on Image Processing, vol. 21, no. 1, pp. 305--315, 2012.
[3] V. Blanz and T. Vetter, ``Face recognition based on fitting a 3D morphable model,''TPAMI, vol. 25, pp. 1063--1074, Sep. 2003.
[4] J. Heo and M. Savvides, ``Gender and ethnicity specific generic elastic models from a single 2d image for novel 2d pose face synthesis and recognition,'' TPAMI, vol. 34, no. 12, pp. 2341--2350, 2012.
[5] U.Prabhu,J.Heo,andM.Savvides, ``Unconstrained pose-invariant face recognition using 3d generic elastic models,'' TPAMI, vol. 33, pp. 1952--1961, 2011.
[6] X. Zhang and Y. Gao, ``Heterogeneous specular and diffuse 3-D surface approximation for face recognition across pose,'' IEEE Trans. Inf. Forensics and Security, vol. 7, no. 2, pp. 1952--1961, 2012.
[7] A.Asthana,T.K.Marks,M.J.Jones,K.H.Tieu,andM.V.Rohith,``Fullyautomatic
pose-invariant face recognition via 3d pose normalization,'' in ICCV, pp. 937--944, 2011.
[8] D. Yi, Z. Lei, and S. Z. Li, ``Towards pose robust face recognition,'' in Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, pp. 3539--3545, IEEE, 2013.
[9] T. Sim, S. Baker, and M. Bsat, ``The CMU pose, illumination, and expression (PIE) database, ''in Proc. IEEE Conf. Automatic Face and Gesture Recognition, pp.46--51, 2002.
[10] X. Zhu and D. Ramanan, ``Face detection, pose estimation, and landmark localization in the wild,'' in Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pp. 2879--2886, IEEE, 2012.
[11] I. Kemelmacher-Shlizerman and R. Basri, ``3D face reconstruction from a single image using a single reference face shape,'' TPAMI, vol. 33, pp. 394--405, Feb. 2011.
[12] B. Y. Li, A. S. Mian, W. Liu, and A. Krishna, ``Using kinect for face recognition under varying poses, expressions, illumination and disguise,'' in Applications of Computer Vision (WACV), 2013 IEEE Workshop on, pp. 186--192, IEEE, 2013.
[13] C. Ciaccio, L. Wen, and G. Guo, ``Face recognition robust to head pose changes based on the rgb-dsensor, '' in Biometrics: Theory, Applications and Systems(BTAS), IEEE Intl Conf. on, pp. 1--6, IEEE, 2013.
[14] X. Zhang and Y. Gao, ``Face recognition across pose: A review,'' Pattern Recognition, vol. 42, pp. 2876--2896, Nov. 2009.
[15] S. J. Prince, J. H. Elder, J. Warrell, and F. M. Felisberti, ``Tied factor analysis for face recognition across large pose differences,'' TPAMI, vol. 30, pp. 970--984, June 2008.
[16] C. D. Castillo and D. W. Jacobs, ``Using stereo matching for 2-D face recognition across pose,'' in CVPR, pp. 1--8, 2007.
[17] D. Jiang, Y. Hu, S. Yan, L. Zhang, H. Zhang, and W. Gao, ``Efficient 3D reconstruction for face recognition,'' Pattern Recognition, vol. 38, pp. 787--798, June 2005.
[18] R. Gross, I. Matthews, J. Cohn, T. Kanade, and S. Baker, ``Multi-pie,'' Image and Vision Computing, vol. 28, pp. 807--813, May 2010.
[19] B. Heisele, T. Serre, and T. Poggio, ``A component-based framework for face detection and identification,'' International Journal of Computer Vision, vol. 74, no. 2, pp. 167--181, 2007.
[20] P.-H. Lee, G.-S. Hsu, T. Chen, and Y.-P. Hung, ``Facial trait code,'' IEEE Trans. Circuits Syst. Video Techn., vol. 23, no. 4, pp. 648--660, 2013.
[21] G. Goswami, S. Bharadwaj, M. Vatsa, and R. Singh, ``On rgb-d face recognition using kinect,'' in Biometrics: Theory, Applications and Systems (BTAS), IEEE Intl Conf. on, pp. 1--6, IEEE, 2013.
[22] P. Phillips, P. J.Flynn, T. Scruggs, K. W. Bowyer, J. Chang, K. Hoffman, J. Marques, J.Min, and W. Worek, ``Overview of the face recognition grand challenge,'' in CVPR, vol. 1, pp. 947--954, 2005.
[23] M. Alexa, J. Behr, D. Cohen-Or, S. Fleishman, D. Levin, and C. T. Silva, ``Computing and rendering point set surfaces,'' IEEE Trans. Vis. Comput. Graph., vol. 9, no. 1, pp. 3--15, 2003.
[24] L. Ding and A. M. Martinez, ``Features versus context: An approach for precise and detailed detection and delineation of faces and facial features, '' TPAMI, vol. 32, no. 11, pp. 2022--2038, 2010.
[25] J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma, ``Robust face recognition via sparse representation,'' IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 2, pp. 210--227, 2009.
[26] A. Yang, A. Ganesh, S. Sastry, and Y. Ma, ``Fast l1-minimization algorithms and an application in robust face recognition: A review,'' Technical Report, no. UCB/EECS-2010-13, 2010.
[27] W. Deng, J. Hu, and J. Guo, ``Extended SRC: Undersampled face recognition via intraclass variant dictionary,'' IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 9, pp. 1864--1870, 2012.
[28] A. Y. Yang, S. S. Sastry, A. Ganesh, and Y. Ma, ``Fast l1-minimization algorithms and an application in robust face recognition: A review,'' in ICIP, pp. 1849--1852, 2010.
[29] W. Deng, J. Hu, J. Guo, W. Cai, and D. D. Feng, ``Robust, accurate and efficient face recognition from a single training image: A uniform pursuit approach,'' Pattern Recognition, vol. 43, no. 5, pp. 1748--1762, 2010.
[30] A. Asthana, S. Zafeiriou, S. Cheng, and M. Pantic, ``Robust discriminative response map fitting with constrained local models,'' in Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, pp. 3444--3451, IEEE, 2013.
[31] X.TanandB.Triggs, ``Enhanced local texture feature sets for face recognition under difficult lighting conditions,'' Image Processing, IEEE Trans. on, vol. 19, pp. 1635--1650, June 2010.
[32] B. Holt, E.-J. Ong, H. Cooper, and R. Bowden, ``Putting the pieces together: Connected poselets for human pose estimation,'' in Computer Vision Workshops (ICCV Workshops), 2011 IEEE International Conference on, pp. 1196--1201, IEEE, 2011.
[33] N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, ``Indoor segmentation and support inference from rgbd images,'' in Computer Vision--ECCV 2012, pp. 746--760, Springer, 2012.
[34] P.Henry,M.Krainin,E.Herbst,X.Ren,andD.Fox,``Rgb-dmapping: Usingkinect-
style depth cameras for dense 3d modeling of indoor environments,'' The International Journal of Robotics Research, vol. 31, no. 5, pp. 647--663, 2012.
[35] N. Dalal and B. Triggs, ``Histograms of oriented gradients for human detection,'' in International Conference on Computer Vision & Pattern Recognition (C. Schmid, S. Soatto, and C. Tomasi, eds.), vol. 2, (INRIA Rhone-Alpes, ZIRST-655, av. de l'Europe, Montbonnot-38334), pp. 886--893, June 2005.
[36] G. Fanelli, M. Dantone, J. Gall, A. Fossati, and L. Van Gool, ``Random forests for real time 3d face analysis, ''International Journal of Computer Vision, vol. 101, no. 3, pp. 437--458, 2013.
[37] T. Huynh, R. Min, and J.-L. Dugelay, ``An efficient lbp-based descriptor for facial depth images applied to gender recognition using rgb-d face data,'' in Computer Vision-ACCV 2012 Workshops, pp. 133--145, Springer, 2013.
[38] P. N. Belhumeur, D. W. Jacobs, D. J. Kriegman, and N. Kumar, ``Localizing parts of faces using a consensus of exemplars,'' in CVPR, pp. 545--552, 2011.
[39] J. M. Saragih, S. Lucey, and J. F. Cohn, ``Deformable model fitting by regularized landmark mean-shift,'' International Journal of Computer Vision, vol. 91, no. 2, pp. 200--215, 2011.
[40] M. Heikkila and M. Pietikainen, ``A texture-based method for modeling the background and detecting moving objects, ''IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 28, no. 4, pp. 657--662, 2006.