臺灣博碩士論文加值系統

近年來，隨著人工智慧領域的迅速發展，深度學習技術的突破性進展已經引起
了各種創新和變革。然而，這種技術的應用也帶來了一系列新興的挑戰和風險，其
中最引人注目的便是深度偽造技術（Deepfake）。深度偽造技術利用深度學習算法，
能夠以驚人的精度生成虛假的影片、音訊以及圖片，使得觀眾難以分辨真偽。儘管
深度偽造技術在影片製作、視覺效果和娛樂產業方面帶來了前所未有的可能性，但
同時也引發了對虛假訊息和影像真實性的深刻關切。
在這樣的背景下，為了應對深度偽造技術可能帶來的社會問題，一些研究開始
探索利用深度學習技術來對抗深度偽造。本項研究就是在這樣的背景下展開的，旨
在利用深度學習技術製作一種能夠有效檢測深度偽造影片的檢測器。具體而言，研
究選擇了Xception檢測網路作為基礎模型，並結合了WFLW資料集中的人臉特徵
點檢測技術，以此來實現面部分割，進而檢測影片中的深度偽造。
為了確保所製作的檢測器的有效性和泛化能力，研究還進行了一系列實驗，比
較了檢測器對數個Deepfake資料集的效能。透過這些實驗，研究團隊將能夠評估
檢測器在不同情況下的表現，發現其優勢和不足之處，並進一步改進和優化檢測器
的性能。這樣的研究不僅有助於提高對抗深度偽造的能力，也為應對未來可能出現
的更加先進和複雜的深度偽造技術提供了重要參考和指導。
總體來說，這項研究旨在利用先進的深度學習技術，開發出一種能夠有效檢測
深度偽造影片的檢測器，從而維護訊息和影像的真實性，保護社會大眾免受虛假訊
息的侵害。這不僅是對科學技術的挑戰，也是對社會公正和穩定的貢獻。

In recent years, the rapid development of the field of artificial intelligence has led to
breakthroughs in deep learning technologies, prompting various innovations and
transformations. However, the application of these technologies also introduces a series
of emerging challenges and risks, among which deepfake technology is notably
prominent. Deepfake technology, utilizing deep learning algorithms, is capable of
generating false videos, audio, and images with astounding accuracy, making it difficult
for audiences to distinguish between real and fake. Although deepfake technology offers
unprecedented possibilities in film production, visual effects, and the entertainment
industry, it also raises serious concerns about misinformation and the authenticity of
images.
Against this backdrop, some research has begun to explore the use of deep learning
technologies to combat Deepfake. This research is conducted in such a context, aiming to
use deep learning to develop a detector capable of effectively identifying deepfake videos.
Specifically, the research utilizes the Xception network as the base model, combined with
facial landmark detection techniques from the WFLW dataset, to achieve facial
segmentation and thereby detect Deepfake in videos.
To ensure the effectiveness and generalizability of the detector, a series of
experiments were conducted comparing its performance across several deepfake datasets.
Through these experiments, the research team will be able to assess the detector’s
performance under different conditions, identify its strengths and weaknesses, and further
improve and optimize its capabilities. Such research not only helps enhance the ability to
counter Deepfake but also provides important references and guidance for addressing
more advanced and complex deepfake technologies that may emerge in the future.
Overall, this research aims to use advanced deep learning technologies to develop a
detector that can effectively identify deepfake videos, thereby maintaining the
authenticity of information and images and protecting the public from the harms of
misinformation and improper influences. This represents not only a challenge to science
and technology but also a contribution to social justice and stability.

摘要 i
Abstract ii
誌 謝 iv
目 錄 v
表 目 錄 vii
圖 目 錄 viii
第一章 緒論 1
1.1研究背景與動機 1
1.2研究目的 1
1.3論文架構 2
第二章 文獻探討 4
2.1 常見人臉偽造方法 4
2.1.1 Faceswap 4
2.1.2 Face2Face 5
2.1.3 FaceShifter 6
2.2人臉檢測架構 7
2.2.1 MesoNet 8
2.2.2 CapsuleNet 9
2.2.3 DSP-FWA 10
2.2.4 Cross-Net 11
2.2.5 Inception 12
2.2.6 Xception 13
2.3人臉特徵點檢測方法 14
2.3.1 AFLIT 14
2.3.2 BAFA 16
2.4人臉抓取方法 17
2.4.1 MTCNN 17
2.4.2 RetinaFace 18
2.4.3 BlazeFace 19
第三章 研究方法 20
3.1 深偽影像資料集 22
3.1.1 Celeb-DF 22
3.1.2 FaceForensics++ 23
3.1.3 Unrestricted Adversarial Faces in Video 24
3.1.4 DeepFake Detection Challenge 25
3.2 人臉特徵點檢測 26
3.2.1 Wider Facial Landmarks in-the-wild資料集 26
3.2.2 特徵點標註 26
3.2.3 臉部分割 27
3.3 Xception檢測網路 28
3.3.1 深度卷積（Depthwise Convolution） 29
3.3.2 逐點卷積（Pointwise Convolution） 30
3.4 深偽人臉檢測系統 31
第四章 研究結果與分析 32
4.1 資料集 32
4.2 環境配置 32
4.3 模型訓練 33
4.4 效能評估指標 34
4.5 各資料集驗證結果比較 35
第五章 結論與未來展望 39
5.1結論 39
5.2 未來展望 39
第六章 參考文獻 40

[1] Chollet, F. (2017). Xception: Deep Learning with Depthwise Separable Convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[2] Faceswap [Online]. Available: https://faceswap.dev/
[3] Thies, J., Zollhöfer, M., Stamminger, M., Theobalt, C., & Nießner, M. (2016). Face2Face: Real-time Face Capture and Reenactment of RGB Videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[4] Li, L., Bao, J., Yang, H., Chen, D., & Wen, F. (2020). FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping. arXiv preprint arXiv:1912.13457.
[5] 卷積神經網路 (CNN) 的發展. Dec. 31, 2019. [Online]. Available: https://medium.com/ai-academy-taiwan/%E5%8D%B7%E7%A9%8D%E7%A5%9E%E7%B6%93%E7%B6%B2%E8%B7%AF-cnn-%E7%9A%84%E7%99%BC%E5%B1%95-4c5d29e60c55
[6] Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018). MesoNet: A Compact Facial Video Forgery Detection Network. In Proceedings of the IEEE International Workshop on Information Forensics and Security (WIFS).
[7] Nguyen, H. H., Yamagishi, J., & Echizen, I. (2019). Capsule-Forensic: Using Capsule Networks to Detect Forged Images and Videos. arXiv preprint arXiv:1910.12467.
[8] Li, Y., Chang, M. C., & Lyu, S. (2018). In Ictu Oculi: Exposing AI Generated Fake Face Videos by Detecting Eye Blinking. In Proceedings of the IEEE International Workshop on Information Forensics and Security (WIFS).
[9] Hulzebosch, N., Ibrahimi, S., & Worring, M. (2020). Detecting CNN-Generated Facial Images in Real-World Scenarios. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).
[10] Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015). Going Deeper with Convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[11] Johnston, B. (2018). A Review of Image-Based Automatic Facial Landmark Identification Techniques. In EURASIP Journal on Image and Video Processing.
[12] Zhang, Z., Luo, P., Loy, C. C., & Tang, X. (2016). Learning Deep Representation for Face Alignment with Auxiliary Attributes. In IEEE Transactions on Pattern Analysis and Machine Intelligence.
[13] Zhang, K., Zhang, Z., Li, Z., & Qiao, Y. (2016). Joint Face Detection and Alignment using Multi-Task Cascaded Convolutional Networks. In IEEE Signal Processing Letters, 23(10), 1499-1503.
[14] Deng, J., Guo, J., Zhou, Y., Yu, J., Kotsia, I., & Zafeiriou, S. (2020). RetinaFace: Single-Stage Dense Face Localisation in the Wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 5203-5212.
[15] Bazarevsky, V., Grishchenko, I., Raveendran, K., Kartynnik, Y., Grundmann, M., & Kwatra, V. (2019). BlazeFace: Sub-Millisecond Neural Face Detection on Mobile GPUs. arXiv preprint arXiv:1907.05047.
[16] Li, Y., Chang, M. C., & Lyu, S. (2020). Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[17] Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Nießner, M. (2019). FaceForensics++: Learning to Detect Manipulated Facial Images. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV).
[18] Yang, X., Li, Y., & Lyu, S. (2019). Exposing Deep Fakes Using Inconsistent Head Poses. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP).
[19] Dolhansky, B., Howes, R., Pflaum, B., Baram, N., & Ferrer, C. C. (2020). The DeepFake Detection Challenge (DFDC) Preview Dataset. arXiv preprint arXiv:2006.07397.
[20] Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., ... & Kudlur, M. (2016). TensorFlow: A System for Large-Scale Machine Learning. In Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI).