臺灣博碩士論文加值系統

人臉偵測在計算機視覺領域中是很常被研究的主題，且其在多種的臉部分析演算法上也扮演很重要的角色，包括像是臉部辨識、人臉之年齡估測及表情辨識等等之應用，人臉偵測可以被稱為這些演算法的基石。人臉偵測的最終目標，即為給定一張任意的圖像，並且去檢測此張圖像是否有人臉的存在，倘若存在著人臉，即回傳人臉在此張圖像中的位置及範圍。
近年來，有越來越多人著重於解決比較特殊且難以被偵測的人臉例子，包括像是姿勢、照明情況及含有遮蔽物的情況等等，而此篇論文主要針對多角度的人臉偵測的例子提供解決的辦法。一直以來，總有許多人嘗試修改知名的‟Viola & Jones(Adaboost)”此人臉偵測的方法，此方法在人臉偵測研究的歷史中可被視為一個里程碑，然而，其卻只在正臉的偵測上有顯著的效果，在面對側臉或是其他多角度人臉的例子，其效果相對於近期的演算法較為略遜一籌，因此，在此篇論文中，我們針對彩色人臉的影像提出了一套偵測的方法，基於Adaboost正臉偵測之演算法的架構，加入五官特徵擷取的條件，去解決多角度的人臉偵測之問題。
在此篇論文中，首先，我們先透過膚色濾波器(Skin-filter)及Viola-Jones正臉偵測器找出正臉的例子，再針對無法成功找到人臉的例子進行五官特徵擷取，包含嘴巴、鼻子及耳朵偵測。我們藉由顏色、輪廓及邊緣的資訊去找到嘴巴及鼻子的候選區域，再者，透過近來熱門的深度學習的方式 ( Faster R-CNN ) 來抽取耳朵的特徵並且找到耳朵的候選區域，最後藉由三者特徵的相對位置去分別判定是否為鼻子、嘴巴及耳朵的正確所在位置，再進一步偵測側臉。除此之外，針對仰頭的例子，我們使用邊緣偵測以及型態學影像處理相關的演算法，再加上一些色彩的資訊，去找出眼睛及鼻孔的候選區域，透過計算眼睛及鼻孔的中心點位置，去找出眼睛及鼻孔的正確所在位置，再進一步偵測人臉。最後透過非極大抑制演算法（Non-Maximum Suppression）去改善偵測率。經由這些處理，此篇論文的方法可以解決許多透過Adaboost的演算法無法偵測的例子，並且大幅提升辨識率，更可以成功解決許多多角度人臉偵測的問題。

Face detection is one of the most research topics in the computer vision field and it also plays an important role on many applications of the face analysis algorithms, such as face recognition, age identification, facial expression recognition, and so on. It is the foundation of many applications. The final goal of face detection is given an arbitrary image, and to detect whether the face exists in this image or not. If the image contains the face, the position and range of the face in the image will be returned.
Many people contribute to solving particular cases which cannot be detected easily because of pose variation, illumination variation, and occlusion. Therefore, we will provide the solutions on multi-view face detection in this thesis. In recent years, many researchers have intended to modify the well-known Viola and Jones (Adaboost) face detection algorithm. This Viola-Jones detector can be regarded as a milestone in the history of face detection. Nevertheless, its sufficient effectiveness is confined to frontal face detection. It is unable to get better detection rates on multi-view face detection. Hence, in this thesis, we propose a robust face detection algorithm based on the “Adaboost” machine learning algorithm and novel methods of facial features extraction to solve the multi-view face detection problems.
First, we apply a skin-filter and Viola-Jones detector to conduct frontal face detection. Second, we extract the facial features of other face images which cannot be found the locations of the faces by first step through our proposed methods, e.g., mouth detection, nose detection, and ear detection. We make use of information of color, edge and contour to extract the facial features such as mouth and nose. Then, we propose a novel method based on the popular deep learning algorithm by improving the techniques of the Faster R-CNN to conduct ear detection. By these proposed methods of facial feature detection, we can obtain the locations of these prominent facial features and proceed to detect the correct locations of the profile faces which contain head-up and head-down cases. In addition, for some cases with head-raised, we apply the edge detection algorithm, morphological operations, and color information to detect eye and nostril candidates. By calculating the locations of the center points of eye and nostril candidates, we can obtain the correct locations of eye and nose to detect the face. Finally, we use the non-maximum suppression algorithm to improve the detection rates. We perform the proposed system on some popular multi-view face databases (e.g., FEI database, CVL database, Pointing’04 database, and so on). Our proposed methods can attain higher detection rates in this novel system, and the effectiveness will be demonstrated in this thesis.

口試委員會審定書 #
誌謝 i
中文摘要 ii
ABSTRACT iv
CONTENTS vi
LIST OF FIGURES ix
LIST OF TABLES xix
Chapter 1 Introduction 1
1.1 Motivation 1
1.2 Multi-view Face Detection 1
Chapter 2 Review of Some Common Face Detection Algorithms 4
2.1 Face Detection Based on Haar-like Features and Adaboost Algorithm 4
2.2 Face Detection Based on Color and Edge Information 7
2.3 Face Detection Based on Normalized Pixel Difference Feature 12
2.4 Multi-view Face Detection Based on Deformable Models 14
2.5 Multi-view Face Detection Based on Deep Learning Algorithms 16
2.6 Multi-view Face Detection Based on Improved Adaboost Algorithms 23
Chapter 3 Review of Some Common Facial Feature Detection Methods 27
3.1 Eye Detection Based on Color and Edge Information 27
3.2 Mouth Detection Based on Color, Contour and Edge Information 32
3.3 Ear Detection Based on Some Robust Methods and Convolution Neural Network 36
Chapter 4 Background Knowledge 44
4.1 Support Vector Machine 44
4.2 Some Fuzzy Concepts Based on Triangular Norms and Negations 46
4.3 Some Common Edge Detection Algorithms 50
4.4 Faster R-CNN Algorithms 55
4.5 Otsu''s Thresholding Method 57
Chapter 5 Proposed Multi-view Face Detection Algorithm 59
5.1 Overview of the Proposed Algorithm 59
5.2 Skin Filter 61
5.3 Frontal Face Detection 62
5.4 Profile Face Detection 65
5.4.1 Mouth Detection 65
5.4.2 Nose Detection 69
5.4.3 Ear Detection 74
A. Object Refocus Filter and Gradient Map 74
B. Faster R-CNN Detector 78
5.4.4 Facial Features Verification 79
5.5 Post Processing 80
5.5.1 Non-Maximum Suppression 80
5.5.2 Flipping and Blending Auxiliary Detection 81
5.6 Further Detection 82
5.6.1 Face Detection For Head-Raised Cases 82
5.6.2 Light Compensation 86
Chapter 6 Simulation Result 88
6.1 Databases 88
6.2 Performance Measurement 89
6.2.1 The Simulation Result of Ear Detection 90
6.2.2 The Simulation Result of Face Detection 95
6.3 Analysis 103
Chapter 7 Conclusions and Future Work 116
7.1 Conclusions 116
7.2 Future Work 116
REFERENCE 118

A.Review Paper
[1]P. Viola and M. J. Jones, “Robust real-time face detection.” International Journal of Computer Vision, vol. 57, issue 2, pp. 137-154, 2004.
[2]P. Viola and M. J. Jones, “Rapid object detection using a boosted cascade of simple features.” Computer Vision and Pattern Recognition. Proceedings of the IEEE Computer Society Conference on, vol. 1, pp. 1-1, 2001.
[3]Y. Yong and X. Ding, “Real-Time Multi-View Face Detection and Pose Estimation Based on Cost-Sensitive Adaboost.” Tsing Hua Science and Technology, vol. 10, pp. 152-157, 2005.
[4]C. Huang, H. Ai, B. Wu, and S. Lao, “Boosting Nested Cascade Detector for Multi-View Face Detection.” Pattern Recognition. Proceedings of the International Conference on, pp. 415-418, 2004.
[5]S. Kang, B. Choi, and D. Jo, “Faces detection method based on skin color modeling.” Journal of Systems Architecture, vol. 64, pp. 100-109, 2016.
[6]D. Ghimire and J. Lee, “A Robust Face Detection Method Based on Skin Color and Edges.” Journal of Information Processing Systems, vol. 9, issue 1, pp. 141-156, 2013.
[7]R.L. Hsu, M. Abdel-Mottaleb, A.K. Jain, “Face detection in color images.” IEEE Trans. Patt. Anal. Mach. Intell., vol. 24, pp. 696-706, 2002.
[8]B. Yang, J. Yan, Z. Lei, and S. Z. Li, "Aggregate channel features for multi-view face detection." International Joint Conference on Biometrics, IEEE, pp. 1-8. 2014.
[9]H. Li, Z. Lin, X. Shen, J. Brandt, and G. Hua “A convolutional neural network cascade for face detection.” International Conference of Computer Vision, IEEE, pp. 5325-5334, 2015.
[10]S. S. Farfade, M. Saberian, and L.J. Li, “Multi-view Face Detection Using Deep Convolutional Neural Networks.” Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, vol. 3, pp. 643-650, 2015.
[11]S. Liao, A. K. Jain, and S. Z. Li, “A fast and accurate unconstrained face detector.” IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, issue 2, pp. 211-223, 2016.
[12]J. Orozco, B. Martinez, and M. Pantic. “Empirical analysis of cascade deformable models for multi-view face detection.” Image and Vision Computing, vol. 42, pp. 47-61, 2015.
[13]A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks.” Advances in Neural Information Processing Systems, pp. 1097-1105, 2012.
[14]Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition.” Proceedings of the IEEE, vol. 86, issue 11, pp. 2278-2324, 1998.
[15]K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, “Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks.” IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1499-1503, 2016.
[16]Y. H. Tsai, Y.C. Lee, and J.J. Ding, “Flipping and blending based highly robust in-plane and out-of-plane color face detection.” International Conference on ICME, pp. 427-432, 2017.
[17]T. Rajpathak, R. Kumar, and E. Schwartz, “Eye Detection Using Morphological and Color Image Processing.” Proceeding of Florida Conference on Recent Advances in Robotics, 2009.
[18]M. Turkan, M. Pardàs, and A. E. Cetin, “Human eye localization using edge projections.” International Conference on Computer Vision Theory and Applications, pp. 410-415, 2007.
[19]D. Donoho, I. Johnstone, and Iain M. Johnstone, “Ideal Spatial Adaptation by Wavelet Shrinkage.” Biometrika, vol. 81, pp. 425-455, 1994.
[20]U. Saeed, and J. L. Dugelay, “Combining edge detection and region segmentation for lip contour extraction.” Proceedings of the international conference on Articulated motion and deformable objects, pp. 11-20, 2010.
[21]T. F. Chan, and L. A. Vese, “Active contours without edges.” IEEE Transactions on Image Processing, vol. 10, issue 2, pp.266-277, 2001.
[22]H. Kalbkhani, and M. C. Amirani, “An Efficient Algorithm for Lip Segmentation in Color Face Images Based on Local Information.” Journal of World''s Electrical Engineering and Technology, vol. 10, issue 2, pp.12-16, 2012.
[23]L. Yuan and Z.-C. Mu., “Ear detection based on skin-color and contour information.” In Proc. of ICMLC, vol. 4, pp. 2213–2217, 2007.
[24]P. P. Sarangi, M. Panda, B. S. P Mishra and S. Dehuri, “An Automated Ear Localization Technique Based on Modified Hausdorff Distance.” International Conference on Computer Vision and Image Processing, vol. 460, pp. 229-240, 2017.
[25]J. G. Allen, R. Y. D. Xu, and J. S. Jin, “Object Tracking Using CamShift Algorithm and Multiple Quantized Feature Spaces.” Proceeding of Pan-Sydney Area Workshop on Visual Information Processing, pp. 3-7, 2004.
[26]Y. Zhang and Z. Mu, “Ear Detection under Uncontrolled Conditions within Multiple Scale Faster Region-Based Convolutional Neural Networks.” Symmetry, 2017.
B.Background Knowledge
[27]C. W. Hsu, C. C. Chang, and C. J. Lin, “A Practical Guide to Support Vector Classification.” Department of Computer Science, National Taiwan University, 2016.
[28]Tzong-Shiun Lin,“Support Vector Machines 簡介.”, CMLab Graphics, National Taiwan University, 2010.
[29]C. C. Chang and C. J. Lin, “LIBSVM: A library for support vector machines.” ACM Trans. Intelligent Systems and Technology, vol. 2, issue 3, article 27, 2011.
[30]Fuzzy Logic Triangular Norms, available in http://fakulty.osu.cz/prf/rsk/uploaded/3504_Lect2_T-norms.pdf
[31]Fuzzy Negations Illustration, available in http://fuzzy.cs.ovgu.de/studium/fuzzy/txt/flogic.pdf
[32]R. M. Haralick, “Digital Step Edges from Zero Crossing of Second Directional Derivatives.” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-6, issue 1, pp. 58-68, 1984.
[33]L.Sobel, “Camera Models and Machine Perception.” PhD theses, Stanford University, Standford, CA, 1970.
[34]J. Prewitt, “Object Enhancement and Extraction.” Picture Process. Psychopict, pp.75–149, 1970.
[35]J. Canny, “Finding Edges and Lines in Images.” MIT Artif. Intell. Lab., Cambridge, MA, Tech. Rep. AI-TR-720, 1983.
[36]S. Ren, K. He, R. Girshick, and J. Sun,” Faster R-CNN: Towards real-time object detection with region proposal networks.” IEEE Trans. Pattern Anal. Machine Intell., vol. 39, issue 6, pp. 1137-1149, 2017.
[37]R. Girshick, and Microsoft Research, “Fast R-CNN.” Int. Conf. Computer Vision, ICCV, 2015.
[38]N. Otsui, “A Threshold Selection Method from Gray-Level Histograms.” IEEE Transactions on Systems, Man, and Cybernetics, vol. 9, issue 1, pp. 62-66, 1979.
[39]L. Xu, J. Ren, Q. Yan, R. Liao, and J. Jia, “Deep edge-aware filters.” Int. Conf. Machine Learning, pp. 1669-1678, 2015.
[40]L. Xu, C. Lu, Y. Xu, and J. Jia, “Image smoothing via L0 gradient minimization.” ACM Trans. Graph., vol. 30, issue 6, pp. 174, 2011.
C.Databases
[41]The USTB database will be available at http://www1.ustb.edu.cn/resb/en/visit/visit.htm
[42]The FEI database will be available at http://fei.edu.br/~cet/facedatabase.html
[43]The CVL database will be available at http://www.lrv.fri.uni-lj.si/facedb.html
[44]The GTAV database will be available at https://gtav.upc.edu/en/research-areas/face-database
[45]The Pointing''04 database will be available at http://www-prima.inrialpes.fr/Pointing04/data-face.html
[46]The Color Feret database will be available at https://www.nist.gov/itl/iad/image-group/color-feret-database
D.Performance Measurement
[47]D. M. W. Powers, "Evaluation: From precision, recall and F-measure to ROC, informedness, markedness & correlation." Journal of Machine Learning Technologies, vol. 2, issue 1, pp. 37–63, 2011.
E.Compared Algorithms
[48]J. Chen, Y. Deng, G. Bai, and G. Su, “Face image quality assessment based on learning to rank.” IEEE Signal Processing Lett., vol. 22, issue 1, pp. 90-94, 2014.
[49]M. Najibi, P. Samangouei, R. Chellappa, and L. Davis, “SSH: Single Stage Headless Face Detector.” International Conference on Computer Vision, vol. 3, 2017.
[50]G. Ghiasi and C. Fowlkes. “Occlusion coherence: Localizing occluded faces with a hierarchical deformable part model.” IEEE Conf. Computer Vision and Pattern Recognition, pp. 2385-2392, 2014.
[51]S. Yadav and N. Nain, “Fast Face Detection Based on Skin Segmentation and Facial Features.” Signal-Image Technology & Internet-Based Systems, pp. 663-668, 2015.
[52]S. Yadav and N. Nain, “A novel approach for face detection using hybrid skin color model.” Journal of Reliable Intelligent Environments, vol. 2, issue 3, pp. 145-158, 2016.