臺灣博碩士論文加值系統

本論文提出一個適用於即時性(real-time)互動環境，以數位影像處理技術開發主動式多景深人臉辨識系統，以自然式及非接觸方式開發人機介面，不僅能有效識別資料庫內已註冊之使用者，並且能判斷不存在資料庫內的未註冊者。本系統以居家收視環境模擬，為了克服因收視環境背景過於複雜可能造成人臉影像的誤偵測，收視距離、時間及位置不一可能造成人臉辨識上的誤判，本系統利用Microsoft kinect sensor所提供的景深資訊及人型骨架(skeleton)資訊。當收視環境存在收視者時，可利用人型骨架的頭部關節部位快速擷取人臉影像可能所在的區域，經由人臉偵測演算法判斷是否為真，再將偵測後的人臉影像利用人臉影像遮罩去除可能因環境複雜所產生的雜訊，並依據景深資訊判斷收視者距離後選擇對應的資料庫來辨識人臉。人臉特徵擷取是運用灰度不變特性之多區塊局部二元樣板(multi-scale block local binary pattern, MBLBP)紋理描述，並將人臉影像以1×1、2×2及3×3階層式(hierarchical)拆解成14個子區塊後擷取紋理特徵，並個別統計成紋理直方圖以串接方式將影像局部與整體關係互相聯結並搭配卡方匹配檢定及積分系統辨識身分。此外，並搭配自適應更新機制，透過學習方式克服收視者可能會在不同時間及位置收視，因光源而產生不同的光影變化，因此設計自適應更新學習方式以增加資料庫內人臉影像紋理直方圖的強健性。

An active face recognition system (AFR) is proposed to act as the real-time human-machine interface for an interactive TV (iTV) system. This AFR system is developed to recognize multi-persons with multi-depth of field for the iTV to recommend interested TV programs for current group users. It is assumed to operate in the family environment with four to six members and designed to recognize registered users and reject (identify) unregistered ones for the program recommender. To improve the 2D face image based recognition method, the 3D scene depth information is also acquired to eliminate false positive recognition. The Microsoft Kinect Sensor is utilized to provide the scene depth and body skeleton information to enhance the processing time and stability of the AFR. With the help of body skeleton, the face image area can be quickly located for the AFR to justify faces within which efficiently. The detected face is first filtered by an average face alpha mask for noise free and its scene depth is used to select corresponding face models from databases for recognition. For robust face feature description, we proposed to adopt multi-scale block local binary pattern, MBLBP. One face image is hierarchically decomposed into 1×1, 2×2, and 3×3 sub-blocks, 14 sub-blocks in total, from which the MBLBP features are extracted and whose histograms are concatenated to act as the face feature. The chi-squared between two sub-block histograms is calculated as the distance for dissimilarity measurement. Votes for these 14 sub-blocks matching between the unknown user and that in the database are counted for user recognition. In addition to face recognition, the AFR can also perform self-learning function to update face models. To achieve active face recognition, it has to recognize users under different lightings and different locations. Once recognized one user, the AFR select sub-blocks with medium similarity measure to update the corresponding ones in the face model. Experiments verified that the proposed AFR can achieve high accuracy and robustness in recognize multi-users with multi-depth of field.

摘要 I
ABSTRACT II
誌謝 III
目錄 IV
圖目錄 VI
表目錄 IX
第一章 緒論 1
1.1 前言 1
1.2 研究背景與動機 2
1.3 研究項目與方法 4
1.4 論文架構 6
第二章 相關背景知識 7
2.1 人臉偵測 7
2.2 人臉辨識 9
2.3 紋理特徵描述 11
2.4 微軟Kinect Sensor 16
第三章 主動式多景深人臉辨識系統 21
3.1 應用環境描述及系統架構 21
3.2 透過骨架及深度資訊找尋感興趣之人臉影像區域 24
3.3 人臉偵測及影像前處理 26
3.3.1 OpenCV 人臉訓練分類器 26
3.3.2 人臉影像區域遮罩 29
3.4 紋理特徵描述及階層式拆解並統計直方圖 31
3.4.1 局部紋理特徵描述 31
3.4.2 階層式拆解並統計多景深紋理直方圖 33
3.5 人臉特徵辨識及自適應更新機制 36
3.5.1 直方圖匹配及積分系統 37
3.5.2 自適應更新機制 40
第四章 實驗結果 42
4.1 實驗平台及人臉資料庫 42
4.2 人臉辨識 43
4.2.1 靜態人臉影像辨識 43
4.2.2 主動式人臉影像辨識 57
第五章 結論與未來展望 65
5.1 結論 65
5.2 未來展望 66
參考文獻 67
附錄 69

[1]Google TV http://www.google.com/tv/
[2]Apple TV http://www.apple.com/appletv/
[3]Samsung Smart TV http://www.samsung.com/tw/consumer/televisions/televisions/smart-tv/UA55ES8000MXZW-features
[4]Youtube http://www.youtube.com/
[5]Kinect Sensor http://en.wikipedia.org/wiki/Kinect
[6]OpenCV http://en.wikipedia.org/wiki/OpenCV
[7]P. Viola and M. Jones, “Rapid Object Detection Using a Boosted Cascade of Simple Features,” IEEE Computer Society Conference on CVPR, vol.1, pp.511-518, 2001.
[8]M.A. Turk and A.P. Pentland, "Face recognition using eigenfaces," presented at Computer Vision and Pattern Recognition, 1991.
[9]A. Mohan, C. Papageorgiou and T. Poggio, “Example-based Object Detection in Images by Components,” IEEE Pattern Analysis and Machine Intelligence, vol.20, pp.39-51, 1998.
[10]J. Yang and A. Waibel, “A Real-Time Face Tracker,” in Proc. 3rd Workshop Applications of Computer Vision, pp. 142-147, 1996.
[11]D. Benjamin, Zarit, J. Boaz, Super, and K.H. Francis, “Comparison of Five Color Models in Skin Pixel Classification,” Int. Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-time Systems, pp. 58 - 63, Sept.1999.
[12]Color space http://en.wikipedia.org/wiki/Color_space
[13]L.-H. Zou and Y.-C. Liu, “A new algorithm of counting human based on segmentation of human faces in color image,” Int. Conference Computational Intelligence and Security, no. 5376172, pp. 505-509, CIS 2009
[14]A. Chen, L. Pan, Y. Tong, N. Ning, “Face detection technology based on skin color segmentation and template matching” 2nd International Workshop on Education Technology and Computer Science, ETCS 2010, no. 5458635, pp. 708-711, 2010.
[15]D. Chai and A. Bouzerdoum, “A Bayesian Approach to Skin Color Classification in YCbCr Color Space,” TENCON 2000. Proceedings, IEEE, Kuala Lumpur Malaysia, vol. 2, pp. 421-424, Sept. 2000.
[16]廖維中，「以色彩分析為基礎之臉部遮蔽偵測應用於A T M 監視系統」，淡江大學資訊工程學系碩士論，民94。
[17]J. Zhu, M. I. Vai and P. U. Mak, “Face recognition using 2D DCT with PCA,” in 4nd Chinese Conf. Biometric Recognition (Sinobiometrics’03) at Beijing, P. R. China, Dec. 7-8, 2003.
[18]G. Guo, S. Z. Li and K. Chan, “Face recognition by support vector machines,” 4th IEEE Int. Conf. Automatic Face and Gesture Recognition, pp. 196-201, 2000.
[19]Support vector machine http://en.wikipedia.org/wiki/Support_vector_machine
[20]T. Ojala, M. Pietikainen,,D. Harwood, “A comparative study of texture measures with classification based on feature distributions,” Pattern Recognition, vol. 29, pp. 51–59, 1996.
[21]Timo Ahonen, Abdenour Hadid, and Matti Pietik‥ainen, “Face recognition with local binary patterns,” ECCV, LNCS 3021, pp. 469–481, 2004.
[22]S. Liao, X. Zhu, Z. Lei, L. Zhang, and S.-Z. Li, “Learning multi-scale block local binary patterns for face recognition,” ICB. 2007, pp. 828-837, 2007.
[23]Taskeed Jabid, Md. Hasanul Kabir, and Oksam Chae, “Local Directional Pattern (LDP) for face recognition,” Int. Conference on Consumer Electronics , ICCE, no. 5418801, pp. 329-330, 2010.
[24]Kinect SDK http://www.microsoft.com/en-us/kinectforwindows/
[25]OpenNI http://en.wikipedia.org/wiki/OpenNI
[26]T. Leyvand, C. Meekhof, Y. Wei, J. Sun, and B. Guo, “Kinect identity: technology and experience,” IEEE Computer, pp.94-96, 2911
[27]J. Shotton et al, “Real-time human pose recognition in parts from single depth images,” IEEE International Conference on Computer Vision and Pattern Recognition, CVPR 2011.
[28]Kinect SDK skeletonhttp://msdn.microsoft.com/en-us/library/hh438998.aspx
[29]Y. Freund and R. Schapire, “A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting,” Journal of Computer and System Sciences, pp.119-139, Aug. 1997.
[30]M. Martinez-Zarzuela, F. J. Diaz-Pernas, F. Lopez Gonzalez, D. Gonzalez Ortega, J. F. Higuera, M. Anton-Rodriguez, D. Boto-Giralda, “AdaBoost face detection on the GPU using haar-like features”, IWINAC 2011, Part II, LNCS 6687, pp. 333–342, 2011.
[31]Nvidia CUDA http://www.nvidia.com/object/cuda_home_new.html
[32]Feret face database http://www.nist.gov/itl/iad/ig/colorferet.cfm/
[33]李承翰，「主動式多解析度人臉辨識系統」，台灣科技大學電機工程系碩士論，民100。