跳到主要內容

臺灣博碩士論文加值系統

(44.222.189.51) 您好!臺灣時間:2024/05/20 14:48
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果 :::

詳目顯示

: 
twitterline
研究生:何宗翰
研究生(外文):Ho, Tsung-Han
論文名稱:結合高頻資訊及種族偏差研究的熱影像到可見光多尺度人臉合成系統
論文名稱(外文):A Multi-Scale Thermal-to-Visible Face Synthesis System with Studies of High-Frequency Information and Race Bias Issues
指導教授:朱威達
指導教授(外文):Chu, Wei-Ta
口試委員:連震杰胡敏君黃敬群
口試日期:2021-07-26
學位類別:碩士
校院名稱:國立成功大學
系所名稱:資訊工程學系
學門:工程學門
學類:電資工程學類
論文種類:學術論文
論文出版年:2021
畢業學年度:109
語文別:英文
論文頁數:38
中文關鍵詞:熱影像人臉辨識生成對抗網路多尺度合成高頻資訊種族偏差
外文關鍵詞:Thermal Face RecognitionGenerative Adversarial NetworkMulti-Scale SynthesisHigh-Frequency InformationRace Bias
相關次數:
  • 被引用被引用:0
  • 點閱點閱:105
  • 評分評分:
  • 下載下載:4
  • 收藏至我的研究室書目清單書目收藏:0
我們提出一種基於對抗式學習的多尺度熱影像到可見光人臉合成系統來實現熱感應人臉辨識。其中包含一個由單一生成器及三個判別器所組成的生成對抗網路。生成器負責將熱感應人臉影像轉變成可見光人臉影像,而三個判別器則分別負責不同尺度特徵配對以及高頻資訊的檢驗。我們進一步提出一個新的熱影像人臉資料集,稱做 NCKU-VTF,主要包含在各種視覺條件下所拍攝的亞洲人種。這個資料集不僅對熱感應人臉辨識構成技術上的挑戰,也讓我們點出目前現有的熱感應人臉辨識方法存在著種族偏差的問題。針對這個問題,我們研究模型微調及利用混合不同種族的數據增強如何影響辨識表現。總體而言,我們提出的多尺度熱影像到可見光人臉合成系統在 EURECOM 資料集和 NCKU-VTF 資料集均達到了現存方法中最好的表現。而高頻資訊和種族偏差的研究也讓未來熱感應人臉辨識研究多了一條新的道路。
We propose a multi-scale thermal-to-visible face synthesis system based on adversarial learning to achieve thermal face recognition. A generative adversarial network is constructed by one generator that transforms a given thermal face into a face in the visible spectrum, and three discriminators that especially examine multi-scale feature matching and high-frequency components, respectively. We further provide a new paired thermal-visible face dataset called NCKU-VTF that mainly contains Asian subjects captured in various visual conditions. This new dataset not only poses technical challenges to thermal face recognition but also enables us to point out the race bias issue in current thermal face recognition methods. Specific to this issue, we investigate how model fine-tuning and race-mixed data augmentation in-fluence recognition performance. Overall, the proposed multi-scale thermal-to-visible face recognition system achieves the state-of-the-art performance on both the EURECOM dataset and the NCKU-VTF dataset. The studies of high-frequency information and race bias issues pave new ways for future thermal face recognition researches.
摘要 i
Abstract ii
誌謝 iii
Table of Contents iv
List of Tables vi
List of Figures vii
Chapter 1. Introduction 1
1.1. Motivation 1
1.2. Contributions 3
1.3. Thesis Organization 3
Chapter 2. Related Work 5
2.1. Thermal Face Recognition 5
2.1.1. Feature Mapping 5
2.1.2. Image Transformation 6
2.2. Thermal Face Datasets 7
2.3. Short Summary 8
Chapter 3. NCKU-VTF Dataset 9
3.1. Environment Setup 9
3.2. Settings of the Collecting Process 9
Chapter 4. The Proposed Method 14
4.1. MultiScale Synthesis Framework 14
4.2. Loss Functions 15
4.3. High-Frequency Information 19
Chapter 5. Experiments 22
5.1. Datasets 22
5.2. Evaluation Protocol 22
5.3. Performance on the EURECOM Dataset 23
5.4. Performance on the NCKU-VTF Dataset 25
5.5. Ablation Studies 29
5.6. Studies of Race Bias 30
5.6.1. Model Finetuning 30
5.6.2. Data Augmentation 31
Chapter 6. Conclusion 33
6.1. Conclusion 33
6.2. Future Work 33
References 35
[1] Adrian Bulat and Georgios Tzimiropoulos. How far are we from solving the 2d & 3d face alignment problem? (and a dataset of 230,000 3d facial landmarks). In Proceedings of IEEE International Conference on Computer Vision, pages 1021–1030, 2017.
[2] Cunjian Chen and Arun Ross. Matching thermal to visible face images using hidden factor analysis in a cascaded subspace learning framework. Pattern Recognition Letters, 72:25–32, 2016.
[3] Cunjian Chen and Arun Ross. Matching thermal to visible face images using a semantic-guided generative adversarial network. In Proceedings of IEEE International Conference on Automatic Face & Gesture Recognition, 2019.
[4] Qifeng Chen and Vladlen Koltun. Photographic image synthesis with cascaded refinement networks. In Proceedings of IEEE International Conference on Computer Vision, pages 1511–1520, 2017.
[5] Xin Chen, Patrick J. Flynn, and Kevin W. Bowyer. Ir and visible light face recognition. Computer Vision and Image Understanding, 99(3):332–358, 2005.
[6] Jonghyun Choi, Shuowen Hu, S. Susan Young, and Larry S. Davis. Thermal to visible face recognition. In Proceedings of SPIE The International Society for Optical Engineering, 2012.
[7] Wei-Ta Chu and Ping-Shen Huang. Thermal face recognition based on multiscale image synthesis. In Proceedings of International Conference on Multimedia Modelling, 2021.
[8] Wei-Ta Chu and Jo-Ning Wu. A parametric study of deep perceptual model on visible to thermal face recognition. In Proceedings of IEEE International Conference on Visual Communications and Image Processing, 2018.
[9] Rinon Gal, Dana Cohen, Amit Bermano, and Daniel CohenOr. Swagan: A stylebased wavelet-driven generative model. arXiv preprint arXiv:2102.06108, 2021.
[10] Yue Gao, Fangyun Wei, Jianmin Bao, Shuyang Gu, Dong Chen, Fang Wen, and Zhouhui Lian. High-fidelity and arbitrary face editing. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021.
[11] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Proceedings of Advances in Neural Information Processing Systems, 2014.
[12] Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. Ms-celeb-1m: A dataset and benchmark for large-scale face recognition. In Proceedings of European Conference on Computer Vision, 2016.
[13] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016.
[14] Harold Hotelling. Relations between two sets of variates. Biometrika, 28:321–377, 1936.
[15] Shuowen Hu, Jonghyun Choi, Alex L. Chan, and William Robson Schwartz. Thermal-to-visible face recognition using partial least squares. Journal of the Optical Society of America A, 32(3), 2015.
[16] Seyed Mehdi Iranmanesh, Ali Dabouei, Hadi Kazemi, and Nasser M. Nasrabadi. Deep cross polarimetric thermal-to-visible face recognition. In Proceedings of International Conference on Biometrics, 2018.
[17] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017.
[18] Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for realtime style transfer and super-resolution. In Proceedings of European Conference on Computer Vision, 2016.
[19] Brendan Klare and Anil K. Jain. Heterogeneous face recognition using kernel prototype similarities. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(6):1410–1422, 2013.
[20] Brahmastro Kresnaraman, Daisuke Deguchi, Tomokazu Takahashi, Yoshito Mekada, Ichiro Ide, and Hiroshi Murase. Reconstructing face image from the thermal infrared spectrum to the visible spectrum. Sensors, 16(4):568, 2016.
[21] David G. Lowe. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2):91–110, 2004.
[22] Khawla Mallat, Naser Damer, Fadi Boutros, Arjan Kuijper, and Jean-Luc Dugelay. Cross-spectrum thermal to visible face recognition based on cascaded image synthesis. In Proceedings of International Conference on Biometrics, 2019.
[23] Khawla Mallat and Jean-Luc Dugelay. A benchmark database of visible and thermal paired face images across multiple variations. In Proceedings of International Conference of the Biometrics Special Interest Group, 2018.
[24] Domenick Poster, Matthew Thielke, Robert Nguyen, Xing Di, Cedric Nimpa Fondje, Vishal M. Patel, Benjamin S. Riggan, Nasser M. Nasrabadi, Srinivasan Rajaraman, Nathaniel J. Short, and Shuowen Hu. A large-scale, time-synchronized visible and thermal face dataset. In Proceedings of IEEE Winter Conference on Applications of Computer Vision, 2021.
[25] Benjamin S. Riggan, Nathaniel J. Short, and Shuowen Hu. Optimal feature learning and discriminative framework for polarimetric thermal to visible face recognition. In Proceedings of IEEE Winter Conference on Applications of Computer Vision, 2016.
[26] Benjamin S. Riggan, Nathaniel J. Short, and Shuowen Hu. Thermal to visible synthesis of face images using multiple regions. In Proceedings of IEEE Winter Conference on Applications of Computer Vision, 2018.
[27] Benjamin S. Riggan, Nathaniel J. Short, Shuowen Hu, and Heesung Kwon. Estimation of visible spectrum faces from polarimetric thermal faces. In Proceedings of IEEE International Conference on Biometrics Theory, Applications and Systems, 2016.
[28] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander Berg, and Li Fei-Fei. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3), 2015.
[29] M. Saquib Sarfraz and Rainer Stiefelhagen. Deep perceptual mapping for thermal to visible face recognition. In Proceedings of British Machine Vision Conference, 2015.
[30] Shreyas Saxena and Jakob Verbeek. Heterogeneous face recognition with cnns. In Proceedings of ECCV TASK-CV Workshops, 2016.
[31] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
[32] Lingxiao Song, Man Zhang, Xiang Wu, and Ran He. Adversarial discriminative heterogeneous face recognition. In Proceedings of AAAI Conference on Artificial Intelligence, 2018.
[33] Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022, 2016.
[34] Jingdong Wang, Ke Sun, Tianheng Cheng, Borui Jiang, Chaorui Deng, Yang Zhao, Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang, Wenyu Liu, and Bin Xiao. Deep high-resolution representation learning for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, PP(99):1–1, 2020.
[35] Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. High-resolution image synthesis and semantic manipulation with conditional gans. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8798–8807, 2018.
[36] Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Chen Change Loy, Yu Qiao, and Xiaoou Tang. Esrgan: Enhanced super-resolution generative adversarial networks. In Proceedings of European Conference on Computer Vision Workshops, 2018.
[37] Xiang Wu, Ran He, Zhenan Sun, and Tieniu Tan. A light cnn for deep face representation with noisy labels. IEEE Transactions on Information Forensics and Security, 13(11):2884–2896, 2018.
[38] Dong Yi, Zhen Lei, Shengcai Liao, and Stan Li. Learning face representation from scratch. arXiv preprint arXiv:1411.7923, 2014.
[39] Teng Zhang, Arnold Wiliem, Siqi Yang, and Brian Lovell. Tv-gan: Generative adversarial network based thermal to visible face recognition. In Proceedings of International Conference on Biometrics, 2018.
連結至畢業學校之論文網頁點我開啟連結
註: 此連結為研究生畢業學校所提供,不一定有電子全文可供下載,若連結有誤,請點選上方之〝勘誤回報〞功能,我們會盡快修正,謝謝!
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top