人臉辨識領域近年來發展逐漸飽和除了改善網路結構，改善損失函數也是一種方法，也讓卷積神經網路從資料中學習到更多的特性，由於深度學習演算法的突破，漸漸發展出基於深度學習的人臉辨識，因此本研究提出「Inception-ResNet-v2」深度卷積神經網路於人臉即時辨識，人臉識別主要包含四個重要步驟：人臉偵測、影像前處理、特徵擷取、特徵比對。
人臉偵測部分，我們採用快速且錯誤率低的多工聯結卷積網路(Multi-Task Cascaded Convolutional Networks , MTCNN)，影像前處理使用人臉對齊透過數據增強(Data Augmentation)的方法進行，使訓練資料多樣化，提升學習模型對未知資料的適應能力。辨識方面透過深度卷積神經網路，提取訓練資料特徵，透過歐式距離進行特徵比對與人臉分類器實現人臉即時識別。
本研究有二個主要貢獻，第一個貢獻是在Inception-ResNet-v1、Inception-ResNet-v2、Inception-v3和Inception-v4四種深度卷積神經網路在softmax loss with center loss損失函數下，得出Inception-ResNet-v1訓練速度最快和Inception-ResNet-v2效果最佳。第二個貢獻是使用預訓練模型經過triplet loss微調後準確率提升。

In recent years, face recognition studies have been developed to full extent and become saturated. It is also important to be noted that this technique can also be advanced by optimizing Loss function in addition to improving the infrastructure of current network. Furthermore, the breakthrough arithmetic methods on using “Deep Learning” to learn much complex features which describe human’s face are flourishing as well. In this study, we include four major steps to accomplish the magnificent result of “Inception-ResNet-v2” of Deep Convolutional Neural Network(DCNN) on face recognition : face detection, pre-image management , features extraction and feature comparison.
Face detection, a speedy and low error rate network, Multi-Task Cascaded Convolutional Networks (MTCNN) is adopted in dealing with this job. pre-image management, it is divided into two groups: training and testing schemes. Both of them not only provide the necessary data and data augmentation for the DCNN, but also diversify the data, which result in promoting and increasing adaptability for our learning model to unknown new data. face recognition, by means of DCNN, the characteristics of related training can be obtained. Meanwhile, by virtue of Euclidean Distance operated in the comparison of unique characteristics, the practice and realization of real-time face recognitions can be achieved.
This research has two main contributions. The first contribution is obtained under the softmax loss with center loss function of the four deep convolutional neural networks Inception-ResNet-v1, Inception-ResNet-v2, Inception-v3 and Inception-v4. Inception-ResNet-v1 has the fastest training speed and Inception-ResNet-v2 has the best effect. The second contribution is the use of pre-trained model after triplet loss fine-tuning to improve accuracy.

目錄
中文摘要……………………………………………………………………………….i
Abstract………………………………………………………………………………ii
誌謝…………………………………………………………………………………...iii
目錄………………………………………………………………………………….. iv
圖目錄………………………………………………………………………………...vi
表目錄……………………………………………………………………………….viii
第一章 緒論.............................1
1.1研究背景.............................1
1.2研究動機.............................1
1.3研究目的.............................2
1.4研究方法.............................3
1.5研究架構.............................4
1.6論文架構.............................5
第二章 文獻探討..........................6
2.1 文獻回顧............................6
2.2 深度學習協助影像處理識別.............6
2.3 人臉相關前處理技術..................10
2.3.1 數據增強....................10
2.3.2 影像正規化處理...............13
2.3.3 OpenCV仿射變換..............14
2.4 人臉偵測相關技術....................16
2.4.1 OpenCV人臉偵測..............16
2.4.2 Dlib人臉偵測................18
2.4.3 人臉關鍵點偵測...............19
2.5 人臉識別演算法......................23
2.5.1 傳統特徵擷取..................23
2.5.2 卷積神經網路之特徵擷取.........24
2.5.3 特徵空間.....................27
2.5.4 卷積神經網路損失函數..........28
第三章 人臉識別方法.....................33
3.1 MTCNN人臉偵測......................34
3.1.1 NMS演算法.....................37
3.2人臉預處理..........................37
3.2.1人臉對齊.......................37
3.3人臉特徵之深度卷積神經網路............38
3.3.1 Inception組成結構.............39
3.3.2 Inception-ResNet組成結構......43
3.3.3 Bottleneck結構...............48
3.4特徵比對............................50
3.4.1 歐式距離......................50
3.5人臉即時識別.........................51
3.5.1 人臉訓練模型結構................51
3.5.2 支援向量機.....................52
第四章 實驗結果與分析....................53
4.1實驗結果.............................53
4.1.1人臉資料庫......................54
4.1.2超參數與訓練....................54
4.2實驗過程.............................55
4.3效能評估.............................56
4.3.1系統速度........................57
4.4比較與討論...........................58
第五章 結論與未來展望....................63
5.1 結論...............................63
5.2 未來展望...........................64
參考文獻...............................65

[1]“Deep MNIST for Experts,” 2017. [Online]. Available: https://www.tensorflow.org/get_started/mnist/pros#deep-mnist-for-experts
[2]M.A. Turk, A.P. Pentland, 1991,“Face recognition using eigenfaces”, Proc. IEEE Conf. on Computer Vision and Pattern Recognition, pp. 586-591.
[3]“Image Classification in 5 Methods,” 2017. [Online]. Available: https://medium.com/towards-data-science/image-classificcation-in-5methods83742ae3645.
[4]黃文堅, 唐源, 2017, Tensorflow 實戰. 北京市: 電子工業出版社.
[5]“前饋神經網路泛化性能力的系統分析＂, 2017. [Online]. Available: http://123.57.41.99/jweb_xtgcllysj/CN/article/downloadArticleFile.do?attachType=PDF &id=109012.
[6]F. Schroff, D. Kalenichenko and J. Philbin, 2015, “FaceNet: A Unified Embedding for Face Recognition and Clustering,”Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 07-12-June-2015, pp. 815-823, Oct. 14,
[7]C. Szegedy, S. Ioffe and V. Vanhoucke, 2017,“Inception-v4: InceptionResNet and the Impact of Residual Connections on Learning,”31st AAAI Conference on Artificial Intelligence, AAAI 2017, pp. 4278-4284.
[8]Jiankang Deng,Jia Guo,Niannan Xue, 2019, “ArcFace: Additive Angular Margin Loss for Deep Face Recognition”, arXiv:1801.07698v3.
[9] A.Krizhevsky, I.Sutskever, and G. E. Hinton, 2012, “ImageNet classification with deep convolutional neural networks,” Advances in neural information processing systems, pp. 1097-1105.
[10] K. Simonyan and A. Zisserman, 2015, “Very deep convolutional networks for large-scale image recognition,” International Conference on Learning Representation (ICLR).
[11] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, 2016, “Deep Residual Learning for Image Recognition,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770 – 778
[12] R. K. Srivastava, K. Greff, and J. Schmidhuber, 2015, “Training very deep networks,” Advances in Neural Information Processing Systems, pp. 2368–2376.
[13] Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, “Identity Mappings in Deep Residual Networks,” Computer Vision and Pattern Recognition ,ECCV 2016.
[14] “Face alignment using MTCNN,” 2017. [Online]. Available: https://github.com/davidsandberg/facenet.
[15] P.Viola, O. M.Way, andM. J.Jones, 2004, “Robust Real-time Face Detection,” vol. 57, no.2,pp. 137–154.
[16] N.Dalal and B.Triggs, 2005, “Histograms of Oriented Gradients for Human Detection.”, In CVPR.
[17] Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, Yu Qiao, 2016, “Joint Face Detection and Alignment using Multi-Task Cascaded Convolutional Networks”, IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1499-1503.
[18]C. Szegedy, W. Liu and Y. Jia, 2015, “Going deeper with convolutions,”Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 0712-June-2015, pp. 1-9, Oct. 14.
[19] M. D. Zeiler and R. Fergus, 2014, “Visualizing and understanding convolutional networks,”Lecture Notes in Computer Science, LNCS,vol. 8689, no. PART 1, pp.818-833.
[20] Haibing Wu, Xiaodong Gu, 2015, “Towards Dropout Training for Convolutional Neural Networks”, Neural Networks 71, pp. 1–10.
[21] “ 深度學習最全優化方法總結比較, ＂ 2017. [Online]. Available: https://zhuanlan.zhihu.com/p/22252270.
[22] Diederik Kingma, Jimmy Ba, 2015, “Adam: A Method for Stochastic Optimization”, ICLR.
[23] Jiankang Deng, Jia Guo, Yuxiang Zhou, Jinke Yu, Irene Kotsia, Stefanos Zafeiriou, 2019, “RetinaFace: Single-stage Dense Face Localisation in the Wild” , arXiv:1905.00641v2 [cs.CV] May. 4.
[24] T. Ojala, M.Pietikäinen, and D. Harwood, 2002, “Multiresolution gray-scale and rotation invariant texture classification with local binary patterns” , pami, vol 24, no.7, July .
[25] “OpenCV函數warpAffine”, 2019. [Online]. Available：https://docs.opencv.org/2.4/doc/tutorials/imgproc/imgtrans/warp_affine/warp_affine.html.
[26]Yi Sun, Ding Liang, Xiaogang Wang, Xiaoou Tang, 2015,“ , DeepID3: Face Recognition with Very Deep Neural Networks”, arXiv:1502.00873v1.
[27]M. Turk and A. Pentland, 1991,“Eigenfaces for recognition”, Journal of Cognitive Neuroscience, 3 (1): 71–86. doi:10.1162/jocn.1991.3.1.71.
[28]Y. Wen, K. Zhang, Z. Li, and Y. Qiao, 2016, “A Discriminative Feature Learning Approach for Deep Face Recognition,” In European Conference on Computer Vision (ECCV).
[29]Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou,Zhifeng Li, and Wei Liu, 2018, “CosFace: Large Margin Cosine Loss for Deep Face Recognition”, arXiv:1801.09414v2.
[30]A. Neubeck , L. Van Gool, 2006, “Efficient Non-Maximum Suppression”, 18th International Conference on Pattern Recognition (ICPR), 20-24 August.
[31]Yann LeCun, Leon Bottou, Yoshua Bengio, and Patrick Haffner, ”Gradient based learning applied to document-recognition” , Proc. of the IEEE, November, 1998.
[32]Tommy Huang , “卷積神經網路” , 2018 . [Online]. Available : https://medium.com//@chih.sheng.huang821.
[33]Wei Wu, Chuanchang Liu, Zhiyuan Su, “Novel Real-time Face Recognition from Video Streams”, 2017 International Conference on Computer Systems, Electronics and Control (ICCSEC), 25-27 Dec. 2017.
[34] “Opencv Cascade Object Detection”, 2018. [Online]. Available : https://chtseng.wordpress.com/2018/06/15/opencv-cascade-object-detection/.
[35] “Labeled Faces in the Wild Home”, 2017. [Online]. Available : http://vis-www.cs.umass.edu/lfw/index.html.
[36]Q. Cao, L. Shen, W. Xie, O. M. Parkhi, and A. Zisserman, 2017, “ Vggface2: A dataset for recognising faces across pose and age.”, arXiv : 1710.08092.
[37]Y. Guo, L. Zhang, Y. Hu, X. He, and J. Gao, 2016, “Ms-celeb-1m : A dataset and benchmark for large-scale face recognition.”, In European Conference on Computer Vision, pages 87–102. Springer.
[38]D. Yi, Z. Lei, S. Liao, and S. Z. Li, 2014, “Learning face representation from scratch.”, arXiv preprint:1411.7923.
[39]G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller, 2007, “Labeled faces in the wild : A database for studying face recognition in unconstrained environments.”, Technical report, Technical Report 07-49, University of Massachusetts, Amherst.
[40]Raia Hadsell, Sumit Chopra, Yann LeCun, 2005, “Dimensionality Reduction by Learning an Invariant Mapping”, in CVPR 2006, November.
[41]Christian Szegedy Google Inc, Vincent Vanhoucke, Sergey Ioffe and Jonathon Shlens, 2015, “Rethinking the Inception Architecture for Computer Vision”, arXiv:1512.00567v3.
[42]Adrian Rosebrock, 2017, “detect-eyes-nose-lips-jaw-dlib-opencv-python” https://www.pyimagesearch.com.