臺灣博碩士論文加值系統

本研究探討基於深度學習之90度側臉辨識。首先使用傳統機器學習方法，分別以MB-LBP與Haar-like 對90度側臉影像進行特徵擷取。再以Adaboost 演算法訓練出90度側臉分類器。另一方面，使用深度學習方法中的Faster R-CNN進行訓練與分類。
實驗以三種分類方法來進行：MB-LBP特徵分類器、Haar-like特徵分類器與深度學習Faster R-CNN 演算法。實驗結果顯示深度學習Faster R-CNN 演算法在90度側臉在無背景干擾影像與在自然背景影像中的偵測成功率皆達99%以上。為了驗證在背景影像的干擾的情況下的可能影響，我們也將90度側臉影像隨機貼入具相同自然背景影像中做測試，也能達到良好的效能。

關鍵字：90度側臉，人臉辨識，機械學習，MB-LBP，Haar-like，特徵擷取，深度學習，Faster R-CNN。

In this paper, we explore 90-degree angle side-view face recognition based on deep learning. First, using the traditional machine learning method, the feature extraction is performed on the 90-degree side-view face image by MB-LBP and Haar-like respectively. Then, the 90-degree side-view face classifier is trained by Adaboost algorithm. On the other hand, the training and classification are performed with Faster R-CNN, one of deep learning methods.
The experiment was carried out in three classification methods: MB-LBP feature classifier, Haar-like feature classifier and deep learning Faster R-CNN algorithm. Experimental results show that the depth learning Faster R-CNN algorithm has a success rate of more than 99% in the 90-degree side-view face without background interference images and in natural background images. In order to verify the possible effects in the case of background image interference, we also randomly placed the 90-degree side-view face image into the same natural background image for testing, which also can achieve good performance.

Keywords: Side-view face, face recognition, machine learning, MB-LBP, Haar-like, feature extraction, deep learning, Faster R-CNN.

中文摘要 I
英文摘要 II
誌謝 III
目錄 IV
圖目錄 VI
表目錄 VIII
第1章 緒論 ..1
1.1研究背景與動機 1
1.2論文架構 2
第2章 傳統機械學習應用90度側臉之辨識 3
2.1傳統之人臉正面辨識 3
2.2傳統機械學習方法應用於90度側臉之辨識 5
2.2.1局部二進制模式 7
2.2.2哈爾特徵 9
2.2.3積分影像 11
2.3.4 Adaboost演算法 12
第3章 基於深度學習之90度側臉辨識 16
3.1卷積神經網路 16
3.1.1 卷積原理 16
3.1.2 池化原理 18
3.1.3 深度學習流程 19
3.1.4 全連接層 19
3.1.5 反向傳播 20


3.2 R-CNN 21
3.3 FAST R-CNN 23
3.2.1 ROI POOLING 24
3.2.2 MULTI-TASK 25
3.2.3 BOUNDING BOX REGRESSION 27
3.3 FASTER RCNN 29
3.3.1 RPN 網路 31
第4章 模擬結果與討論 34
4.1 90度側臉偵測 34
4.2傳統機械學習之90度側臉偵測結果 43
4.3深度學習90度側臉偵測結果 45
4.4 90度背景側臉偵測結果 46
第5章 結論與未來研究方向 49
5.1結論 49
5.2未來研究方向 49
參考文獻 50

[1] T. Ahonen, A. Hadid, and M. Pietikainen, “Face description with local binary patterns: Application to face recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 12, pp. 2037-2041, 2006.
[2] A. Sharifara, M. S. M. Rahim, and Y. Anisi, “A general review of human face detection including a study of neural networks and Haar feature-based cascade classifier in face detection,” International Symposium on Biometrics Security Technologies, pp. 73-78, Aug. 2014.
[3] J.-H. Huang, “A Potential-based approach for shape matching and recognition,” Pattern Recognition, vol. 29, no. 3, pp. 463-470, 1996.
[4] R. Brunelli and T. Poggio, “Face recognition: Features versus templates,” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1042-1052, 1993.
[5] M. Turk and A. Pentland, “Eigen faces for recognition,” Journal of Cognitive Neuroscience, vol. 3, no. 1, pp. 71-86, 1991
[6] K.-K. Sung and T. Poggio, “Example-based learning for view-based human face detection,” Image Understanding Workshop , pp.843-850, 1994.
[7] A. Pentland, B. Moghaddam, and T. Starner, “View-based and modular eigenspaces for face recognition,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 84-91, 1994.
[8] S.-H. Jeng, H.-Y. Mark Liao, C.-C. Han, M.-Y. Chern, and Y.-T. Liu, “An efficient approach for facial feature detection using geometrical face model,” Pattern Recognition, vol. 31, no. 3, pp. 273-282, 1998.
[9] X.-I. Jia and M.-S. Nixon, “Extend the feature vector for automatic face recognition,” IEEE Trans. Pattern Analysis and Machin Intelligence, vol. 17, no. 12, pp. 1167-1176 , 1995.
[10] X. Zhu and D. Ramanan, “Face detection, pose estimation, and landmark localization in the wild,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 2879-2886, 2012.
[11] T. Kanade, “Picture Processing by Computer Complex and Recognition of Human Faces,” PhD Thesis, 1973.
[12] K. Bong, S. Choi, C. Kim, and H.-J. Yoo, “Low-power convolutional neural network processor for a face-recognition system,” IEEE Micro, vol. 37, no. 6, pp. 30-38, 2017.
[13] A. Rahardja, A. Sowmya, and W.H. Wilson, “A neural network approach to component versus holistic recognition of facial expressions in images,” SPIE Intelligent Robots and Computer Vision X: Algorithms and Techniques, vol. 1607, pp. 62-70, 1991.
[14] C. Neubauer, “Evaluation of convolution neural networks for visual recognition,” IEEE Transactions on Neural Networks, vol. 9, no. 4, pp. 685-696, 1998.
[15] K. Shinjiro and J. Ohya, “Automatic skin-color distribution extraction for face detection and tracking,” International Conference on Signal Processing , vol. II, pp. 1415-1418, 2000.
[16] J. L. Crowley and K. Schwerdt, “Robust tracking and compression for vedio communication,” IEEE Computer Society International Conference on Computer Vision Workshop on Facie and Gesture Recognition, 1999.
[17] X. Zhang and R.-M. Mersereau, “Lip feature extraction towards an automatic speechreading system,” IEEE International Conference on Image Processing, vol. 3, pp. 226-229, 2000.
[18] R.-L. Hsu, M. Abdel-Mottaleb, and A.K. Jain, “Face detection in color images,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 696-706, 2002.
[19] H. Yao and W. Gao, “Face detection and location based on skin chrominance and lip chrominance transformation from color images,” Pattern Recognition, vol. 34, no. 8, pp. 1555-1564, 2001.
[20] T. Wark, D. Thambiratnam, and S. Sridharan. “Person authentication using lip information,” IEEE TENCON, pp. 153-156, 1997.
[21] C. Morimoto and M. Flickner, “Real-time multiple face detection using active illumination,” IEEE International Conference on Automatic Face and Gesture Recognition, pp. 8-13, 2000.
[22] P. Viola and M. Jones, “Robust real-time face detection,” International Journal of Computer Vision, vol. 57, no. 2, pp. 137-154, 2004.
[23] P. Zhu, W. Zuo, L. Zhang, S. C.-K. Shiu, and D. Zhang, “Image set-based collaborative representation for face recognition,” IEEE Transactions Forensics Security, vol. 9, no. 7, pp. 1120-1132, 2014.
[24] R. Lienhart and J. Maydt, “An extended set of haar-like features for rapid Object Detection,” IEEE International Conference on Image Processing, pp. 900-903, 2002.
[25] A. Dasgupta, A. George, S. Happy, and A. Routray, “A vision-based system for monitoring the loss of attention in automotive drivers,” IEEE Transactions on Intelligent Transportation Systems , vol. 14, no. 4, pp. 1825-1838, 2013.
[26] P. Vadakkepat, P. Lim, L. C. D. Silva, L. Jing, and L. L. Ling, “Multimodal approach to human-face detection and tracking,” IEEE Transactions on Industrial Electronics and Control Instrumentation, vol. 55, no. 3, pp. 1385-1393, 2008.
[27] W. Yang, X. Sun, and Q. Liao, “Cascaded elastically progressive model for accurate face alignment,” IEEE Transactions on Systems Man and Cybernetics: Systems, vol. 47, no. 9, 2017.
[28] T. Mikolov, M. Karafiat, L. Burget, J. Cernocky, and S. Khudanpur, “Recurrent neural network based language model, ” Inter-speech, 2010.
[29] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
[30] H.-C. Shin, “Deep convolutional neural networks for computer aided detection: CNN architectures, dataset characteristics and transfer learning,” IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1285-1298, 2016.
[31] S. Miao, Z. J. Wang, and R. Liao, “A CNN regression approach for real-time 2D/3D registration,” IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1352-1363, 2016.
[32] G. Girish, S. N. CL, and P. K. Das, “Face recognition using mb-lbp and pca: A comparative study,” Computer Communication and Informatics International Conference on, pp. 1-6, 2014.
[33] P. Viola and M. Jones, “Rapid object detection using a boosted cascade of simple features,” IEEE Computer Vision and Pattern Recognition, 2001.
[34] R. Lienhart and J. Maydt, “An extended set of Haar-like features for rapid object detection,” IEEE International Conference on Image Processing, vol. 1, pp. 900-903, 2002.
[35] S. Zhang, C. Bauckhage, and A. B. Cremers, “Informed Haar-like features improve pedestrian detection,” IEEE Computer Vision and Pattern Recognition, pp. 947-954, 2014.
[36] S.A.A.M. Faudzi and N. Yahya, “Evaluation of LBP-based face recognition techniques,” 5th International Conference on Intelligent and Advanced Systems, vol. 1, no. 2, pp. 1-6, 2014.
[37] J. Ren, X. D. Jiang, and J. Yuan, “Dynamic texture recognition using enhanced LBP features,” IEEE International Conference Acoustics Speech and Signal Processing, pp. 2400-2404, 2013.
[38] J. Ren, X. D. Jiang, J. Yuan, and G. Wang, “Optimizing LBP structure for visual recognition using binary quadratic programming,” IEEE Signal Processing Letters, vol. 21, no. 11, pp. 1346-1350, 2014.
[39] X. Hong, G. Zhao, M. Pietikainen, and X. Chen, “Combining LBP difference and feature correlation for texture description,” IEEE Transactions on Image Processing, vol. 23, no. 6, pp. 2557-2568, 2014.
[40] H. Yang and Y. Wang, “A LBP-based face recognition method with hamming distance constraint”, IEEE International Conference on Image and Graphics, pp. 645-649, 2007.
[41] A. Satpathy, X. D. Jiang, and H. L. Eng, “LBP-based edge-texture features for object recognition,” IEEE Transactions on Image Processing , vol. 23, no. 5, pp. 1953-1964, 2014.
[42] J. Ren, X. Jiang, and J. Yuan, “Learning LBP structure by maximizing the conditional mutual information,” Pattern Recognition, vol. 48, no. 10, pp. 3180-3190, 2015.
[43] Y. Freund and R.E. Schapire, “A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting,” European Conference on Computational Learning Theory, pp. 23-37, 1995.
[44] N. Zhang, J. Donahue, R. Girshick, and T. Darrell, “Part-based r-cnns for fine-grained category detection,” European Conference on Computer Vision, pp. 834-849, 2014.
[45] R. Girshick,“Rich feature hierarchies for accurate object detection and semantic segmentation,” IEEE Conference on Computer Vision and Pattern Recognition, 2014.
[46] R. Girshick, “Fast r-cnn,” IEEE International Conference on Computer Vision, 2015.
[47] S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: towards real-time object detection with region proposal networks,” IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 39, no. 6, pp. 1137-1149, 2017.
[48] The FEI face database. ,from the World Wide Web:
http://fei.edu.br/~cet/facedatabase.html
[49] Cohn Kanade face database. ,from the World Wide Web:
http://www.consortium.ri.cmu.edu/data/ck/
[50] Psychological image collection at stirling. ,from the World Wide Web:
http://pics.psych.stir.ac.uk/
[51] The ORL database of faces, from the World Wide Web:
https://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html
[52] Yale face database. ,from the World Wide Web:
http://vision.ucsd.edu/content/yale-face-database