臺灣博碩士論文加值系統

近年來科技發展快速，尤其於電腦科學領域發展更受各界關注，驅使得人工智慧（Artificial Intelligence）、大數據（Big Data）、深度學習（Deep Learning）、機器學習（Machine Learning）等相關技術也成為各版面討論度熱度最高的名詞。也因為如此各領域爭相投入大量資源與資金研發相關技術，進而將此類技術運用在各行業。像是網路科技業、金融科技業、AI機器人、電腦視覺、數據分析、服務業、行銷業等，同時漸漸與結合在生活當中。同時，也衍伸出擴增實境（Augmented Reality，簡稱AR）、虛擬實境（Virtual Reality，縮寫VR）、虛擬主播(VTuber )及深偽技術(Deep fake) 的新名詞。對於深度學習、影像處理及人臉辨識的技術及需求增加。
過去在人臉辨識技術常受環境影響使得辨識度不佳的情況時常發生。而現在能使用基於深度學習的人臉辨識技術來增加辨識度及拓展其應用。其中，Google 於2019年推出相關應用MediaPipe。MediaPipe可用於構建多模式音訊、影片或任何時間序列資料的框架。透過MediaPipe 框架的幫助下，可以為 TensorFlow、TF Lite 等推理模型以及媒體處理功能構建相關服務的機器學習管道。
本研究主要針對MediaPipe Face Mesh做為研究方向。MediaPipe Face Mesh 是可以用在移動設備上以即時估計 468 個 3D 人臉標記。採用機器學習來判斷 3D 臉部表面，用以人臉五官為特徵做為實驗基礎，偵測的部份用MediaPipe Face Mesh的人臉偵測結果，而人臉偵測結合了雙眼及嘴巴所現成之三角形並得到三角形重心，同時實驗包含了張眼、閉眼、抬頭、低頭、左右搖擺等實驗數據來研究其應用精準度及實用性。也對轉頭30度、60度及90度分別進行實驗來確認錯誤率。實驗結果顯示，本論文所提之臉部特徵及動作用於Media Pipe可以達到理想辨識結果。

In recent years, the field of computer science has developed rapidly, including Artificial Intelligence, Big Data, machine learning, deep learning, etc. These technologies have gradually become the most discussed terms on the Internet. It also makes countries willing to invest a lot of money in research and development of related technologies, and then apply such technologies in various industries. For example, the Internet technology industry, financial industry, intelligent robots, computer vision, data analysis, etc., are gradually living with our life. At the same time, new terms such as Augmented Reality (AR), Virtual Reality (VR), VTuber, and Deepfake have also been developed. The demand for deep learning, image processing, and face recognition have increased.
In the past, it often happened that facial recognition technology was often affected by the environment and the recognition was poor. Now, face recognition technology based on deep learning can be used to increase recognition and expand its application. Among them, Google launched the related application MediaPipe in 2019. MediaPipe is a framework mainly used to build multimodal audio, video, or any time series data. With the help of the MediaPipe framework, machine learning pipelines can be built for inference models like TensorFlow, TF Lite, and media processing functions.
This research mainly focuses on MediaPipe FaceMesh as the research direction. MediaPipe FaceMesh estimates 468 3D face markers in real-time even on mobile devices. Machine learning is used to determine the 3D face surface, and the facial features are used as the experimental basis. The detection part uses the face detection results of MediaPipe FaceMesh, and the face detection combines the eyes and the mouth. The triangle is obtained and the center of gravity of the triangle is obtained. At the same time, the experiment includes things such as opening eyes, closing eyes, looking up, bowing, and swinging left and right to study its application accuracy. Experiments were also carried out on the face at 30 degrees, 60 degrees, and 90 degrees to confirm the error rate. The experimental results show that Media Pipe can achieve ideal recognition results when it comes to face recognition.

中文摘要 i
英文摘要 iii
誌謝 iv
目錄 v
表目錄 vi
圖目錄 vii
第一章 導論 1
1.1 研究動機 1
1.2 研究目的 1
1.3 論文架構 2
第二章 文獻探討 3
2.1 人臉偵測 (Face Detection) 3
2.2 人臉校正 (Face Alignment) 4
2.3 3D人臉重建(3D Morphable Model) 4
第三章 研究方法 6
3.1 傳統臉部偵測方法 6
3.2 Media Pipe 8
3.3 Media Pipe Face Mesh 9
第四章 研究結果 10
4.1 實驗方法 10
4.1.1 左右轉頭 10
4.1.2 張嘴、閉嘴 11
4.1.3 抬頭、低頭 11
4.1.4 左右擺頭 12
4.1.5 張眼、閉眼 12
4.2 實驗結果 13
4.3 技術應用實例探討 14
4.3.1 技術應用 14
4.3.2 行銷應用 16
第五章 結論 18
5.1 結論 18
5.2 未來展望 18
參考文獻 20

1.P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features" On Computer vision and Pattern Recognition, 2001
2.Google ML Kit Face-Detection https://developers.google.com/ml-kit/vision/face-detection
3.Xiangyu Zhu, Xiaoming Liu, Zhen Lei, Stan Z. Li "Face Alignment in Full Pose Range: A 3D Total Solution" on IEEE 2018, P.146~155
4.A. Bulat and G. Tzimiropoulos "How far are we from solving the 2D & 3D Face Alignment Problem? ( and a dataset of 230,000 3D facial landmarks) " On Computer Vision Laboratory, The University of Nottingham, United Kingdom, 2017
5.Jianzhu Guo, Xiangyu Zhu, Yang Tang , Fan Yang, Zhen Lei, Stan Z. Li "Towards Fast, Accurate and Stable 3D Dense Face Alignment" ECCV, 2020
6.V. Blanz and T. Vetter "A Morphable Model For The Synthesis Of 3D Faces" Tubingen, Germany, 1999
7.Anh Tuan Tran, Tal Hassner, Iacopo Masi, and Gerard Medioni "Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Network" On IEEE Xplore, USA, 2017
8.M. Turk, A. Pentland "Eigenfaces for Recognition" J Cogn Neurosci 1991; 3 (1): 71–86. doi: https://doi.org/10.1162/jocn.1991.3.1.71
9.W.S. McCulloch and W. Pitts"A Logigal Calculus of The Ideas Immanent in Nervos Activity" Bulletin of Mathematical Biology Vol.52, No.1/2, pp.99-115, 1990
10.G.E. Hinton and R.R. Salakhtdinov "Reducing the Dimensionality of Data with Neural Networks" 2006 Vol 313, Issue 5786 pp.504-507 doi: 10.1126/science.1127647
11.MediaPipe https://google.github.io/mediapipe/
12.A. Yuan and A. Vakunov "Face and Hand tracking in the browser with MediaPipe and TensorFlow.js" Google, 2020
13.K. Sokal "MediaPipe 3D Face Transform" Google Developers, USA 2020
14.Y. Kartynnik, A. Ablavatski, I.Grishchenko, M.Grundmann "Real-time Facial Surface Geometry from Monocular Video on Mobile GPUs", USA, 2019
15.A.Ablavatski and I.Grishchenko "Real-Time ARself-Expression with Machine Learning" Google AI, 2019
16.A.Ablavatski, A.Vakunov, I.Grashchenko, K.Raveendran, M.Zhdanovich "Real-Time Pupil Tracking from Monocular Video for Digital Puppetry" USA, 2020
17.MediaPipe Face Mesh https://google.github.io/mediapipe/solutions/face_mesh