臺灣博碩士論文加值系統

近年來，有鑑於更有效率且更友善的人機介面之需求漸漸提高，使得各種與人臉處理相關的研究迅速地成長。除了對人提供服務之外，一個好用的系統最重要的是與人們的自主性互動關係，其中人臉辨識系統已廣泛地應用於身份辨識、門禁監控與人機介面等領域，最近由於「智慧生活」科技的提倡，人臉辨識技術已延伸至人與機器最佳化介面的應用；有鑑於此，我們在本論文中提出一個具有即時自動人臉追蹤與辨識功能的機器人跟隨系統，主要包括人臉追蹤與人臉辨識兩大程序。

在人臉偵測方面，首先，我們基於影像中的膚色區塊取出可能為人臉的區域，再利用區域面積和人臉長寬比例等幾何資訊來偵測候選人臉；接著，從先前偵測出來的人臉區域找出精確的眼睛和嘴巴的區域，再利用等腰三角形的基礎，依據眼睛與嘴巴的相對位置判斷所獲得的人臉區域是否為一個真正的人臉。

在人臉追蹤程序，我們利用粒子濾除器(particle filter)技術來動態追蹤人臉。由於我們考慮了頭髮的顏色，當人臉背對攝影機時，系統也能持續對其作正確的追蹤。另外，我們以運動與顏色兩者資訊當作特徵，使在追蹤的過程中儘可能讓背景的影響達到最低。根據人臉在畫面中的位置，對機器人的輪控馬達下達前進、左轉、右轉的指令，並且利用超音波測距，來判斷機器人與目標物的距離，進而下達停止或後退的指令，直到機器人跟隨至適當的距離，便開始進行人臉辨識的程序，判定是否為主人。

至於在人臉辨識程序，是在人臉偵測與追蹤程序後，我們以二維哈爾小波轉換(two-dimensional Haar wavelet transform)取出人臉影像的低頻部份，如此可以克服傳統影像特徵擷取的缺點，有效降低影像的資料維度，另外，我們改進線性鑑別分析(linear discriminant analysis)找不到反矩陣與主成份分析(principal component analysis)無法辨識不同類別的缺點，而使用有辨別力的共通向量(discriminative common vectors)建立具有鑑別性的人臉特徵參數模型。最後，我們使用最小歐機里德距離的決策方式，連續取10張人臉辨識後，投票判斷最有可能的人。

根據實驗結果顯示，我們所提方法的人臉追蹤率在一般的情況下高於 95%，而於人臉遭受暫時性遮蔽的情況下高於 88%，又人臉的辨識率經過統計得知：在一般的情況下，可以達到 93% 以上，而在複雜的背景下，仍有 80% 以上的表現；此外，對於機器人的整體系統執行效能相當令人滿意，在人臉追蹤和辨識程序上，分別至少達到每秒 5 個和 2 個畫面的速度。

Owing to the demand of more efficient and friendly human-computer interfaces, the researches on face processing have been rapidly grown in recent years. In addition to offering some kinds of service for human beings, one of the most important characteristics of a favorable system is to autonomously interact with people. Accordingly, face recognition has been broadly applied in the areas, such as biometric identity authentication, entrance guard, and human-computer interfaces. More recently, the technique of face recognition has been markedly extended to the applications of the optimality of human-computer interfaces due to the promotion of “intelligent life.” In view of the above-mentioned facts, a completely automatic real-time face tracking and recognition system installed on a person following robot is presented in this thesis, including face tracking and face recognition procedures.

As to face detection, it is first based on skin color blocks and geometrical properties applied to eliminate the skin color regions that do not belong to the face in the HSV color space. Then we find the proper ranges of two eyes and one mouth according to the positions of pupils and the center of a mouth. Subsequently, we utilize the foundation of an isosceles triangle formed by the relative positions of two eyes and one mouth to judge whether the detected skin color regions a human face.

In the face tracking procedure, we employ an improved particle filter to dynamically locate a human face. Since we have considered the hair color information of a human head, the particle filter will keep tracking even if the person is back to the sight of a camera. We further adopt both the motion and color cues as the features to make the influence of the background as low as possible. According to the position of the human face in an image, we issue a series of commands (moving forward, turning left or turning right) to drive the motors of wheels on a robot, and judge the distance between the robot and a person with the aid of three ultrasonic sensors to issue a set of commands (stop or turn backward) until the robot follows to a suitable distance from the person. At this moment, the system starts the recognition procedure that identifies whether the person is the master of the robot or not.

In the face recognition procedure, after the face detection and tracking procedure, we have captured a face image and apply the two-dimensional Haar wavelet transform (2D-HWT) to acquire the low-frequency data of the face image. This method is able to overcome the drawbacks of extracting face features in traditional manners. Additionally, we improve the shortcomings of principal component analysis (PCA) which can not effectively distinguish from different classes and those of linear discriminant analysis (LDA) which may find no inverse matrix. And we then employ the discriminative common vectors (DCV) algorithm to setup the discriminative models of face features received from different persons. Finally, we utilize the minimum Euclidean distance to measure the similarity of the face image and a candidate person and decide the most likely person by the majority vote of ten successive recognition results from a face image sequence.

Experimental results reveal that the face tracking rate is more than 95% in general situations and over 88% when the face suffers from temporal occlusion. As for the face recognition, the rate is more than 93% in general situations and still reaches 80% in complicated backgrounds; besides this, the efficiency of system execution is very satisfactory, which respectively attains 5 and 2 frames per second at least in the face tracking and recognition modes.

銘謝 i
中文摘要 ii
Abstract iv
Contents vii
List of Figures ix
List of Tables xiii
Chapter 1 Introduction 1
1.1 Overview 1
1.2 Background and motivation 1
1.3 Our proposed methods 5
1.4 System description 6
1.5 Thesis organization and system architecture 12
Chapter 2 Related Work 14
2.1 Reviews of face detection 15
2.2 Reviews of face Tracking 20
2.3 Reviews of face recognition 22
Chapter 3 Face Detection 25
3.1 Color space transformation 26
3.1.1 Skin color detection using the HSV model 27
3.1.2 Hair color detection using YCbCr model 29
3.2 Connected component labeling 31
3.3 Face geometry filtering 32
3.4 Pupils and mouth detection 35
3.5 Triangle geometry modeling 39
Chapter 4 Face Tracking 44
4.1 Object description and finding 44
4.2 The particle filter 46
4.3 Our proposed method 53
4.3.1 Improved particle filter 54
4.3.2 Robot control 60
Chapter 5 Face Recognition 63
5.1 Preliminary processing 64
5.2 Wavelet transformation 66
5.3 Principal components analysis 70
5.4 Linear discriminant analysis 76
5.5 Discriminative common vectors 83
5.6 Similarity measurement 90
Chapter 6 Experimental Results and Discussions 92
6.1 System interface description 93
6.2 The results of face detection 100
6.3 The results of face tracking 104
6.4 The results of face recognition 110
6.4.1 Face databases 110
6.4.2 Experiments on the ORL and Yale databases 113
6.4.3 Experiments on the robot 116
Chapter 7 Conclusions and Future Works 127
7.1 Conclusions 127
7.2 Future works 129
References 130

[1]G. L. Foresti, C. Micheloni, L. Snidaro, and C. Marchiol, “Face detection for visual surveillance,” in Proceedings of the 12th IEEE International Conference on Image Analysis and Processing, pp.115-120, 2003.
[2]H. Zhao and R. Shibasaki, “A Real-Time System for Monitoring Pedestrians,” Application of Computer Vision, vol. 1, pp. 378-385, 2005
[3]E. Hjelm�犘, and B. K. Low, “Face detection: a survey,” Computer Vision and Image Understanding, vol. 83, pp. 236-274, 2001.
[4]K. C. Fan, Y. K. Wang, and B. F. Chen, “Introduction of tracking algorithms,” Image and Recognition, Vol. 8, No. 4, pp. 17-30, 2002.
[5]W. Zhao, R. Chellappa, P. J. Phillips, and A. Rosenfeld, “Face recognition: a literature survey,” ACM Computing Surveys, vol. 35, no. 4, pp. 399-458, 2003.
[6]K. H. An, D. H. Yoo, S. U. Jung, and M. J. Chung, “Robust multi-view face tracking,” in Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Edmonton, Alberta, Canada, pp. 1905-1910, 2005.
[7]S. Z. Li and J. Lu, “Face recognition using the nearest feature line method,” IEEE Transactions on Neural Networks, vol. 10, pp. 439-443, 1999.
[8]R. Brunelli and T. Poggio, “Face recognition: features versus templates,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.15, pp. 1042, 1993.
[9]M. J. Er, S. Wu, J. Lu, and H. L. Toh, “Face recognition with radial basis function (RBF) neural networks,” IEEE Transactions on Neural Networks, vol. 13, pp. 697-710, 2002.
[10]M. Soriano, S. Huovinen, B. Martinkauppi, and M. Laaksonen, “Using the skin locus to cope with changing illumination conditions in color-based face tracking,” in Proceedings of the IEEE Nordic Signal Processing Symposium, pp. 383-386, 2000.
[11]R. C. Gonzalez and R. E. Woods, Digital Image Processing, 2nd Ed., Addison-Wesley, Reading, Massachusetts, 1992.
[12]S. L. Phung, A. Bouzerdoum, and D. Chai, “A novel skin color model in YCbCr color space and its application to human face detection,” in Proceedings of IEEE International Conference on Image Processing, vol. 1, pp. 289-291, 2002.
[13]L. Sigal, S. Sclaroff, and V. Athitsos, “Skin color-based video segmentation under time-varying illumination,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, pp. 862-877, 2004.
[14]B. Fr�爿a and C. K�佒lbeck, “Robust face detection at video frame rate based on edge orientation features,” in Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition, pp. 342-347, 2002.
[15]Z. Liu, X. He, J. Zhou, and G. Xiong, “A novel method for eye region detection in gray-level image,” in Proceedings of IEEE International Conference on Circuits and Systems and West Sino Expositions, vol. 2, pp. 1118-1121, 2002.
[16]W. Huang, Q. Sun, C. P. Lam, and J. K. Wu, “A robust approach to face and eyes detection from images with cluttered background,” in Proceedings of the international conference on Pattern Recognition, vol. 1, pp. 110-114, Aug, 1998.
[17]K. M. Lam and H. Yan, “An improved method for locating and extracting the eye in human face images,” in Proceedings of the international conference on Pattern Recognition, vol. 3, pp. 411-415, 1996.
[18]C. Lin, “Face detection, pose classification, and face recognition based on triangle geometry and color features,” Master Thesis, Department of Computer Science Information Engineering, National Central University, Jhongli, Taiwan, 2001.
[19]S. H. Jeng, H. Y. M. Liao, C. C. Han, M. Y. Chern, and Y. T. Liu, “Facial feature detection using geometrical face model: an efficient approach,” Pattern Recognition, vol. 31, pp.273-282, 1998.
[20]K. Y. Wang, “A real-time face tracking and recognition system based on particle filtering and AdaBoosting techniques,” Master Thesis, Department of Computer Science Information Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan, 2006.
[21]W.S. Lee, H.J. Lee, and J.H. Chung, “Wavelet-based FLD for face recognition,” in Proceedings of the IEEE Midwest Symposium on Circuits and Systems, Lansing MI, pp. 734-737, 2000.
[22]Y. Nara, J. Yang, and Y. Suematsu, “Face recognition using improved principal component analysis,” in Proceedings of the International Symposium on Micromechatronics and Human Science, pp. 77 - 82, 2003.
[23]M. Turk and A. Pentland, “Face recognition using eigenfaces,�舡n Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 586-591, 1991.
[24]Y. Bing, J. Lianfu, and C. Ping, “A new LDA-based method for face recognition,” in Proceedings of the International Conference Pattern Recognition, vol. 1, pp. 168-171, 2002.
[25]K. Etemad and R. Chellappa, “Discriminant analysis for recognition of human face images,” Journal of the Optical Society of America A, Vol.14, pp.1724-1733, 1997.
[26]H. Cevikalp, M. Neamtu, M. Wilkes, and A. Barkana, “Discriminative common vectors for face recognition”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol 27, pp. 4-13, 2005.
[27]H. Cevikalp and M. Wilkes, “Face recognition by using discriminative common vectors,” in Proceedings of the International Conference on Pattern Recognition, Vol. 1, pp. 326-329, 2004.
[28]M.B. Gulmezoglu, V. Dzhafarov, and A. Barkana, “The common vector approach and its relation to principal component analysis,” IEEE Transactions on Speech and Audio Processing, vol. 9, no. 6, 2001.
[29]H. Lu and W. Shi, “Accurate active shape model for face alignment,” in Proceedings of the 17th IEEE International Conference on Tools with Artificial Intelligence, vol. 13, pp. 1119-1128, 2005.
[30]M. Bicego, G. Iacono, and V. Murino, “Face recognition with multilevel B-Splines and Support Vector Machines,” ACM WBMA’03, 2003.
[31]M. H. Yang, N. Ahuja, and D. Kriegman, “Face detection using mixtures of linear subspaces,” in Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 70-76, 2000.
[32]A. M. Martinez and A. C. Kak, “PCA versus LDA,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, pp. 228-233, 2001.
[33]C. Y. Chang and S. Y. Fu, “Image classification using a module RBF neural network,” in Proceedings of the First International Conference on Innovative Computing, Information and Control, vol. 2, pp. 270-273, 2006.
[34]C. A. Waring and X. Liu, “Face detection using spectral histograms and SVMs,” IEEE Transactions on System, vol. 35, pp. 467- 476, 2005.
[35]S. Birchfield, “Elliptical head tracking using intensity gradients and color histograms,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 232-237, 1998.
[36]P. Fieguth and D. Terzopoulos. “Color-based tracking of heads and other mobile objects at video frame rates,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 21-27, 1997.
[37]C. Y. Tang, Y. P. Hung, and Z. Chen, “Automatic detection and tracking of human head using an active stereo vision system,” in Proceedings of Asian Conference on Computer Vision, vol. 1, pp. 632-639, 1998.
[38]N. Herodotou, K. N. Plataniotis, and A. N. Venetsanopoulos, “Automatic location and tracking of the facial region in color video sequences,” Signal Processing and Image Communication, pp. 359-388, 1999.
[39]Z. Tang, Z. Miao, “Fast background subtraction and shadow elimination using improved Gaussian mixture model,” in Proceedings of the IEEE International Workshop on Haptic Audio Visual Environments and their Applications Ottawa, pp. 38-41, 2007.
[40]H. P. Graf, E. Cosatto, D. Gibbon, and M. Kocheisen, “Multi-modal system for locating heads and faces,” in Proceedings of the Second International Conference on Automatic Face and Gesture Recognition, pp. 88-93, 1996.
[41]K. Suzuki, I. Horiba, and N. Sugie, “Linear-time connected-component labeling based on sequential local operations,” Source Computer Vision and Image Understanding Archive, vol. 89 , no. 1, pp. 1-23, 2003.
[42]Y. H. Ching, “Visual tracking for a moving object using optical flow technique,” Master Thesis, Department of Mechanical and Electro- Mechanical Engineering, National Sun Yat Sen University, Kaohsiung, 2003.
[43]G. Welch and G. Bishop, “An introduction to the Kalman filter,” Technical Report TR95-041, Department of Computer Science, University of North Carolina at Chapel Hill, NC, 2004.
[44]S. Caifeng, W. Yucheng, T. Tieniu, and F. Ojardias, “Real time hand tracking by combining particle filtering and mean shift,” in Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 669-674, 2004.
[45]M. Montemerlo, S. Thrun, and W. Whittaker, “Conditional particle filters for simultaneous mobile robot localization and people-tracking,” in Proceedings of the IEEE International Conference on Robotics and Automation, vol. 1, pp. 695-701, 2002.
[46]K. Nummiaro, E. Koller-Meier, and L. Van Gool, “An adaptive color-based particle filter,” Image and Vision Computing, vol. 21, no. 1, pp. 99-110, 2003.
[47]C. Liu and H. Wechsler, “Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition,” IEEE Transactions on Image Processing, vol.11, pp.467- 476, 2002.
[48]L. Sirovich and M. Kirby, “Low-dimensional procedure for the characterization of human faces,�紃ournal of the Optical Society of America A, vol.4, no. 3, 1987.
[49]M. Kirby and L. Sirovich, “Application of the Karhunen-Loeve procedure for the characterization of human faces,�粁EEE Transactions on Pattern Analysis and Machine Intelligence, vol. 12, no. 1, 1990.
[50]P. Belhumeur, J. Hespanha, and D. Kriegman, “Eigenfaces vs. Fisherfaces: recognition using class specific linear projection,�� IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, 1997.
[51]R. Huang, Q. Liu, H. Lu, and S. Ma, “Solving the small size problem of LDA,” in Proceedings of the International Conference on Pattern Recognition, vol. 3, pp. 29-32, 2002.
[52]L. F. Chen, H. Y. M. Liao, M. T. Ko, J. C. Lin, and G. J. Yu, “A new LDA-based face recognition system which can solve the small sample size problem,” in Proceedings of the International Conference on Pattern Recognition, vol. 33, pp. 1713-1726, 2000.
[53]C. C. Wu, “Face recognition using discriminant wavelet features,” Master Thesis, Department of Computer Science Information Engineering, National Cheng Kung University, Tainan, Taiwan, 2001.