臺灣博碩士論文加值系統

許多人臉相關應用，包括人臉辨識、情緒檢測、及近年蓬勃發展的醫學美容等，都依賴精準的人臉特徵資訊。人工標註特徵點有著低效、不穩定、不夠客觀等各種問題，故須發展一套自動且有效提取人臉特徵點的技術。現今以深度學習網路自動提取人臉特徵點的方法，多注重在解決正臉特徵點定位的問題。本論文考慮使用效能良好的深層卷積神經網路(Deep Convolutional Neural Networks)模型，進行90度側臉21個特徵點的定位，並針對該模型提出三項主要改善方法。其一為提升神經網路的深度，藉由更大尺寸的輸入值，以及多層次的卷積、池化層交互堆疊，放大特徵提取的效果。其二，對於輸入的影像不僅僅採用全彩的三通道圖，亦考量採用灰階圖，加強對輪廓和線條類特徵的敏感度。其三，在兩階段的網路皆進行區域的分類與個別訓練，提升網路模型的專精度，防止因複雜形狀導致網路模型收斂效果不佳的問題。其四，在第二階段的部分網路採用非正方形的圖片輸入，透過更適當的擷取區塊最大程度的凸顯該部位的特徵值。實驗結果顯示，本論文提出的方法相較於文獻的深層卷積神經網路，以3個像素點（臉長的1.5%）為容錯閥值的情形下，對90度側面人臉的特徵點定位，其點平均誤差值可降低2.11像素，正確率則可提升38.14%。

Many face-related applications including face recognition, emotion detection, and medical cosmetology, rely on accurate facial features information. Manual labeling are inefficient, unstable, and subjective, and therefore an efficient automatic facial landmarking technique has been a crucial research topic. Current automatic facial feature extraction techniques based on deep learning networks are mostly applied to frontal facial landmarking. This thesis explores the method of deep convolutional neural networks for detecting 21 features on profile faces. Three major improvements on the model are proposed. First, a deeper network with bigger input sizes and more convolution and pooling layers was used to better extract the features. Second, this thesis used not only color images but also gray-scale images as inputs to emphasize the contours or edges. Third, we separated the model of both first layer and second layer in five regions and local models were used in subsequent training for better model converging under complex shapes. Fourth, some networks in second layer used non-square image as input, because a more suitable capture block can maximize the features value of the part. Experimental results substantiated the superiority of the proposed method. Compared with the original deep convolutional neural networks, the proposed model not only decreases the facial location deviation by 2.11 pixels, but also increases the accuracy of facial landmarking by 38.14% under a 3-pixel (1.5% facial height) error tolerance.

摘　要 i
ABSTRACT ii
誌　謝 iii
目　錄 iv
表目錄 viii
圖目錄 ix
第一章 緒論 1
1.1 研究背景與動機 1
1.2 研究目的 2
1.3 論文架構 2
第二章 人臉特徵點定位相關文獻 3
2.1以模型為基礎的人臉特徵點檢測方法 3
2.1.1 Active Shape Models (ASM) 3
2.1.2 Active Appearance Models (AAM) 4
2.1.3 Constrained Local Models (CLM) 4
2.1.4 Cascaded Pose Regression (CPR) 5
2.2 Neural Network 5
2.2.1 Neural Network (NN) 5
2.2.1.1 Activation Function 6
2.2.1.2 Forward-Propagation 6
2.2.1.3 Cost Function 7
2.2.1.4 Backward-Propagation 8
2.2.1.5 Gradient 9
2.2.1.6 Vanishing Gradient 9
2.2.2 Support Vector Machine (SVM) 11
2.3 Deep Learning (DP) 11
2.3.1 Solution 12
2.3.1.1 Restricted Boltzmann Machine (RBM) and Deep Belief Network (DBN) 12
2.3.1.2 New Activation Function 13
2.3.1.3 Optimizer 15
2.3.2 Deep Neural Network (DNN) 16
2.3.2.1 DNN做法 16
2.3.2.2 Dropout 16
2.3.3 Convolutional Neural Network (CNN) 17
2.3.3.1 CNN 做法 17
2.3.3.2 Convolution 17
2.3.3.3 Pooling 18
2.3.3.4 Fully Connected 19
2.4 以CNN解決人臉特徵點定位問題之相關文獻 19
2.4.1 Deep Convolutional Neural Networks (DCNN) 19
2.4.2 Face++ Deep Convolutional Neural Networks (Face++ DCNN) 20
2.4.3 Tweaked Convolutional Neural Networks (TCNN) 22
第三章 研究方法 25
3.1 系統架構 25
3.2 側臉特徵點與區域定義 27
3.3 訓練樣本前處理 28
3.4 DCNN神經網路訓練模型 31
3.5 DCNN神經網路測試流程 33
第四章 實驗結果與分析 35
4.1實驗環境 35
4.1.1 軟硬體設備 35
4.2 實驗方法 35
4.2.1 特徵點誤差值計算方法 36
4.2.2特徵點標註工具 36
4.3特徵點定位結果 37
4.3.1 Layer1彩圖特徵點定位結果 37
4.3.2 Layer1灰階圖特徵點定位結果 39
4.3.3 Layer1後的擷取區塊-正方形 41
4.3.4 Layer1後的擷取區塊-非正方形 42
4.3.5 Layer2彩圖特徵點定位結果 43
4.3.6 Layer2灰階圖特徵點定位結果 44
4.3.7非正方形區塊特徵點定位結果 46
4.4 特徵點準確度分析 48
4.4.1第一階段(彩圖/灰階)特徵點誤差統計 49
4.4.2第二階段(彩圖/灰階)特徵點誤差統計 50
4.4.3第二階段非正方形區塊特徵點誤差統計 51
4.4.4第一階段各部位特徵點誤差統計 53
4.4.5第二階段各部位特徵點誤差統計 54
4.4.6第二階段非正方形區塊各部位特徵點誤差統計 55
4.5實驗數據分析 55
第五章 結論與未來工作 63
5.1 結論 63
5.2 未來工作 63
參考文獻 64

[1] T. Cootes, and C. Taylor, “Active shape models—‘smart snakes’,” in Proc. British Machine Vision Conference. BMVC, Vol.3, Leeds, West Yorkshire, England, 1992, pp. 266–275.
[2] T. Cootes, G. Edwards and C. Taylor, “Active Appearance Models,” in Proc. European Conference on Computer Vision. ECCV, Vol.2, Germany, 1998, pp. 484–498.
[3] D. Cristinacce and T. Cootes, “Feature detection and tracking with constrained local models,” in Proc. British Machine Vision Conference. BMVC, Vol.3, Edinburgh, Scotland, 2006, pp. 929–938.
[4] Y. Sun, X. Wang and X. Tang, “Deep Convolutional Network Cascade for Facial Point Detection,” in Proc. Computer Vision and Pattern Recognition. CVPR, 2013, pp. 3476-3483.
[5] E. Zhou, H. Fan, Z. Cao, Y. Jiang and Q. Yin, “Extensive Facial Landmark Localization with Coarse-to-Fine Convolutional Network Cascade,” in Proc. International Conference on Computer Vision. ICCV, 2013, pp. 386-391.
[6] Y. Wu, T. Hassner, K.G. Kim, G. Medioni and P. Natarajan, “Facial Landmark Detection with Tweaked Convolutional Neural Networks,” arXiv:1511.04031, 21 Mar 2016.
[7] OpenCV: Open Source Computer Vision Library, https://opencv.org/
[8] M. Zimbler and J. Ham, “AESTHETIC FACIAL ANALYSIS,” in Cummings Otolaryngology: Head and Neck Surgery, Chapter 21, 2004.
[9] K. Zhamg, Z. Zhang, Z. Li and Y. Qiao, “Joint Face Detection and Alignment using
Multi-task Cascaded Convolutional Networks,” IEEE Signal Processing Letters, Vol.13, Issue10, 10 Oct 2016.
[10] Z. Zhang, P. Luo, C.C. Loy and X. Tang, “Facial Landmark Detection by Deep Multi-task Learning” in Proc. European Conference on Computer Vision. ECCV, 2014, pp. 94–108.
[11] S. Guo, G. Tan, H. Pan, L. Chen and C. Gao, “Face alignment under occlusion based on local and global feature regression” Vol.76, Issue6, Mar 2017, pp 8677–8694.
[12] M. Kowalski, J. Naruniec, and T. Trzcinski, “Deep Alignment Network: A Convolutional Neural Network for Robust Face Alignment,” in Proc. Computer Vision and Pattern Recognition. CVPR, 2017.
[13] J. Zhang, S. Shan, M. Kan and X. Chen, “Coarse-to-Fine Auto-Encoder Networks (CFAN) for Real-Time Face Alignment” in Proc. European Conference on Computer Vision. ECCV, 2014, pp. 1–16.
[14] J. Dai, H. Qi, Y. Xiong, Y. Li, G. Zhang, H. Hu and Y. Wei, “Deformable convolutional networks,” in Proc. International Conference on Computer Vision. ICCV, 2017.
[15] F. Chollet, “Xception: Deep Learning with Depthwise Separable Convolutions,” in Proc. Computer Vision and Pattern Recognition. CVPR, 2017.
[16] Wikipedia 人工神經網絡, https://zh.wikipedia.org/wiki/人工神經網絡
[17] Wikipedia卷積神經網絡, https://zh.wikipedia.org/wiki/卷積神經網絡
[18] Wikipedia Relu, https://zh.wikipedia.org/wiki/線性整流函數
[19] Wikipedia Delaunay, https://zh.wikipedia.org/wiki/德勞內三角化
[20] Tensorflow: Open Source Software Library for Dataflow Programming Across a Range of Tasks. https://www.tensorflow.org/
[21] NVIDIA GPU: http://www.nvidia.com.tw/Download/index.aspx?lang=tw
[22] NVIDIA cuDNN: https://developer.nvidia.com/cudnn
[23] Posts Tagged ‘Adam Optimizer’, https://smist08.wordpress.com/tag/adam-optimizer/