跳到主要內容

臺灣博碩士論文加值系統

(100.28.0.143) 您好!臺灣時間:2024/07/23 09:34
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果 :::

詳目顯示

我願授權國圖
: 
twitterline
研究生:潘慈閔
研究生(外文):Pan, Tzu-Min
論文名稱:一個用於單張影像反射去除的輕量版GAN
論文名稱(外文):A Light Weight Modified GAN for Single Image Reflection Removal
指導教授:戴顯權戴顯權引用關係
指導教授(外文):Tai, Shen-Chuan
口試委員:戴顯權郭淑美吳宗憲郭忠民鄺獻榮
口試日期:2023-07-10
學位類別:碩士
校院名稱:國立成功大學
系所名稱:電機工程學系
學門:工程學門
學類:電資工程學類
論文種類:學術論文
論文出版年:2023
畢業學年度:111
語文別:英文
論文頁數:58
中文關鍵詞:單張影像去反射深度學習大核注意力多尺度注意力
外文關鍵詞:single image reflection removaldeep learningLarge Kernel AttentionMulti-scale attention
相關次數:
  • 被引用被引用:0
  • 點閱點閱:77
  • 評分評分:
  • 下載下載:0
  • 收藏至我的研究室書目清單書目收藏:0
隨著科技的進步,具有拍照功能的便攜式設備越來越普及,人們可以隨時隨地輕鬆拍照,記錄美好瞬間。然而通過透明玻璃拍攝的圖像通常包含反射,會產生視覺噪點並降低圖像質量,若使用這些照片來做高階視覺任務像是語意分割或物件偵測的時候,也會影響其表現。本文提出一種結合多尺度注意力及生成對抗網路的單張影像去反射演算法。此方法包含一個生成器和一個判別器,其中生成器使用了多尺度注意力轉換模塊來產生乾淨的影像,而判別器則用來判別真實的影像相對於生成的影像更真實的機率。
實驗結果顯示,本文提出的方法,相較於其他方法,在參數量上有顯著的減少,在客觀影像評估標準及主觀的影像品質與目前最佳方法相符。
With the advancement of technology, portable devices with camera functions have become more and more popular, and people can easily take pictures whenever they want to record beautiful moments. However, images taken through transparent glass often contain reflections, which can create visual noise and degrade image quality. These reflections can also affect performance when using these photos for high-level vision tasks such as semantic segmentation or object detection. This Thesis proposes a single image reflection removal algorithm combining multi-scale attention and generative adversarial networks. It contains a generator and a discriminator. The generator uses a multi-scale attention transformer block to produce a clean output image. The discriminator estimates the probability that the real image is relatively more realistic than the image generated by the generator.
The experimental results show that the proposed method has significantly reduced the number of parameters and is consistent with the current best methods in terms of objective measurement and subjective visual quality.
摘要 i
Abstract ii
Acknowledgments iii
Contents iv
List of Tables vi
List of Figures vii
Chapter 1 Introduction 1
1.1 Overview 1
Chapter 2 Background and Related Works 4
2.1 Introduction of Reflection Properties 4
2.2 Related Works 7
2.3 VGG19 Network 8
2.4 Relativistic GAN 10
2.5 Larger Kernel Attention 11
2.6 Normalization 14
2.6.1 Layer Normalization (LN) 15
2.6.2 Group Normalization (GN) 16
2.7 Activation Function 17
2.7.1 Mish 17
2.7.2 GELU 18
2.8 Quality Metrics 19
2.8.1 Peak Signal-to-Noise Ratio (PSNR) 19
2.8.2 Structural Similarity Index Measure (SSIM) 19
Chapter 3 The Proposed Algorithm 21
3.1 Proposed Network Architecture 22
3.2 Generator Architecture 24
3.3 Discriminator Architecture 29
3.4 Loss Function 30
3.4.1 Pixel loss 30
3.4.2 Perceptual loss 31
3.4.3 Gradient loss 32
3.4.4 Adversarial loss 32
3.4.5 Total loss 34
Chapter 4 Experiment Results 35
4.1 Experimental Dataset 35
4.2 Parameter and Experimental Setting 38
4.3 Experimental Results of Simulated Images 39
4.4 Ablation Study 52
Chapter 5 Conclusion and Future Work 53
5.1 Conclusion 53
5.2 Future Work 53
References 55
[1]H. Barrow, J. Tenenbaum, A. Hanson, and E. Riseman, "Recovering intrinsic scene characteristics," Comput. vis. syst, vol. 2, no. 3-26, p. 2, 1978.
[2]Q. Fan, J. Yang, G. Hua, B. Chen, and D. Wipf, "A Generic Deep Architecture for Single Image Reflection Removal and Image Smoothing," in 2017 IEEE International Conference on Computer Vision (ICCV), 22-29 Oct. 2017 2017, pp. 3258-3267, doi: 10.1109/ICCV.2017.351.
[3]R. Wan, B. Shi, L.-Y. Duan, A.-H. Tan, and A. C. Kot, "CRRN: Multi-Scale Guided Concurrent Reflection Removal Network," p. arXiv:1805.11802doi: 10.48550/arXiv.1805.11802.
[4]X. Zhang, R. Ng, and Q. Chen, "Single Image Reflection Separation with Perceptual Losses," in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 18-23 June 2018 2018, pp. 4786-4794, doi: 10.1109/CVPR.2018.00503.
[5]K. Wei, J. Yang, Y. Fu, D. Wipf, and H. Huang, "Single Image Reflection Removal Exploiting Misaligned Training Data and Network Enhancements," in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 15-20 June 2019 2019, pp. 8170-8179, doi: 10.1109/CVPR.2019.00837.
[6]B. Song, J. Zhou, and H. Wu, "Multistage Curvature-Guided Network for Progressive Single Image Reflection Removal," IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 10, pp. 6515-6529, 2022, doi: 10.1109/TCSVT.2022.3168828.
[7]M. Kang et al., "Scaling up GANs for Text-to-Image Synthesis," p. arXiv:2303.05511doi: 10.48550/arXiv.2303.05511.
[8]R. Wan, B. Shi, H. Li, Y. Hong, L. Y. Duan, and A. C. Kot, "Benchmarking Single-Image Reflection Removal Algorithms," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 2, pp. 1424-1441, 2023, doi: 10.1109/TPAMI.2022.3168560.
[9]A. P. Pentland, "A New Sense for Depth of Field," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-9, no. 4, pp. 523-531, 1987, doi: 10.1109/TPAMI.1987.4767940.
[10]S. Angulo et al., All-in-focus image reconstruction robust to ghosting effect (SPIE Optical Engineering + Applications). SPIE, 2018.
[11]B. Buttery and G. Davison, "The ghost artifact," Journal of Ultrasound in Medicine, vol. 3, no. 2, pp. 49-52, 1984, doi: https://doi.org/10.7863/jum.1984.3.2.49.
[12]R. Wan, B. Shi, H. Li, L. Y. Duan, and A. C. Kot, "Reflection Scene Separation From a Single Image," in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 13-19 June 2020 2020, pp. 2395-2403, doi: 10.1109/CVPR42600.2020.00247.
[13]Y. Yang, W. Ma, Y. Zheng, J. F. Cai, and W. Xu, "Fast Single Image Reflection Suppression via Convex Optimization," in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 15-20 June 2019 2019, pp. 8133-8141, doi: 10.1109/CVPR.2019.00833.
[14]Q. Zheng, B. Shi, J. Chen, X. Jiang, L. Y. Duan, and A. C. Kot, "Single Image Reflection Removal with Absorption Effect," in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 20-25 June 2021 2021, pp. 13390-13399, doi: 10.1109/CVPR46437.2021.01319.
[15]Y. L. Liu, W. S. Lai, M. H. Yang, Y. Y. Chuang, and J. B. Huang, "Learning to See Through Obstructions," in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 13-19 June 2020 2020, pp. 14203-14212, doi: 10.1109/CVPR42600.2020.01422.
[16]K. I. Diamantaras and T. Papadimitriou, "Blind separation of reflections using the image mixtures ratio," in IEEE International Conference on Image Processing 2005, 14-14 Sept. 2005 2005, vol. 2, pp. II-1034, doi: 10.1109/ICIP.2005.1530235.
[17]C. Lei and Q. Chen, "Robust Reflection Removal with Reflection-free Flash-only Cues," in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 20-25 June 2021 2021, pp. 14806-14815, doi: 10.1109/CVPR46437.2021.01457.
[18]A. Levin and Y. Weiss, "User Assisted Separation of Reflections from a Single Image Using a Sparsity Prior," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 9, pp. 1647-1654, 2007, doi: 10.1109/TPAMI.2007.1106.
[19]Y. Li and M. S. Brown, "Single Image Layer Separation Using Relative Smoothness," in 2014 IEEE Conference on Computer Vision and Pattern Recognition, 23-28 June 2014 2014, pp. 2752-2759, doi: 10.1109/CVPR.2014.346.
[20]R. Wan, B. Shi, A. H. Tan, and A. C. Kot, "Sparsity based reflection removal using external patch search," in 2017 IEEE International Conference on Multimedia and Expo (ICME), 10-14 July 2017 2017, pp. 1500-1505, doi: 10.1109/ICME.2017.8019527.
[21]R. Wan, B. Shi, L. Y. Duan, A. H. Tan, W. Gao, and A. C. Kot, "Region-Aware Reflection Removal With Unified Content and Gradient Priors," IEEE Transactions on Image Processing, vol. 27, no. 6, pp. 2927-2941, 2018, doi: 10.1109/TIP.2018.2808768.
[22]J. Yang, D. Gong, L. Liu, and Q. Shi, "Seeing deeply and bidirectionally: A deep learning approach for single image reflection removal," in Proceedings of the european conference on computer vision (ECCV), 2018, pp. 654-669.
[23]Q. Wen, Y. Tan, J. Qin, W. Liu, G. Han, and S. He, "Single Image Reflection Removal Beyond Linearity," in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 15-20 June 2019 2019, pp. 3766-3774, doi: 10.1109/CVPR.2019.00389.
[24]C. Li, Y. Yang, K. He, S. Lin, and J. E. Hopcroft, "Single Image Reflection Removal Through Cascaded Refinement," in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 13-19 June 2020 2020, pp. 3562-3571, doi: 10.1109/CVPR42600.2020.00362.
[25]Z. Zou, S. Lei, T. Shi, Z. Shi, and J. Ye, "Deep Adversarial Decomposition: A Unified Framework for Separating Superimposed Images," in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 13-19 June 2020 2020, pp. 12803-12813, doi: 10.1109/CVPR42600.2020.01282.
[26]Z. Dong, K. Xu, Y. Yang, H. Bao, W. Xu, and R. W. H. Lau, "Location-aware Single Image Reflection Removal," in 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 10-17 Oct. 2021 2021, pp. 4997-5006, doi: 10.1109/ICCV48922.2021.00497.
[27]龍柏安, "一個用於反光去除的增強式生成對抗網路," 碩士, 電腦與通信工程研究所, 國立成功大學, 台南市, 2021. [Online]. Available: https://hdl.handle.net/11296/bxspc9
[28]W. T. Chen, K. Y. Chen, I. H. Chen, H. Y. Fang, J. J. Ding, and S. Y. Kuo, "Missing Recovery: Single Image Reflection Removal Based on Auxiliary Prior Learning," IEEE Transactions on Image Processing, vol. 32, pp. 643-656, 2023, doi: 10.1109/TIP.2022.3230544.
[29]K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition," p. arXiv:1409.1556doi: 10.48550/arXiv.1409.1556.
[30]A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," Commun. ACM, vol. 60, no. 6, pp. 84–90, 2017, doi: 10.1145/3065386.
[31]A. Jolicoeur-Martineau, "The relativistic discriminator: a key element missing from standard GAN," arXiv preprint arXiv:1807.00734, 2018.
[32]M.-H. Guo, C.-Z. Lu, Z.-N. Liu, M.-M. Cheng, and S.-M. Hu, "Visual Attention Network," p. arXiv:2202.09741doi: 10.48550/arXiv.2202.09741.
[33]S. Ioffe and C. Szegedy, "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift," p. arXiv:1502.03167doi: 10.48550/arXiv.1502.03167.
[34]J. Lei Ba, J. R. Kiros, and G. E. Hinton, "Layer Normalization," p. arXiv:1607.06450doi: 10.48550/arXiv.1607.06450.
[35]Y. Wu and K. He, "Group Normalization," p. arXiv:1803.08494doi: 10.48550/arXiv.1803.08494.
[36]Y. Wang, Y. Li, G. Wang, and X. Liu, "Multi-scale Attention Network for Single Image Super-Resolution," p. arXiv:2209.14145doi: 10.48550/arXiv.2209.14145.
[37]R. Wan, B. Shi, L. Y. Duan, A. H. Tan, and A. C. Kot, "Benchmarking Single-Image Reflection Removal Algorithms," in 2017 IEEE International Conference on Computer Vision (ICCV), 22-29 Oct. 2017 2017, pp. 3942-3950, doi: 10.1109/ICCV.2017.423.
電子全文 電子全文(網際網路公開日期:20240831)
連結至畢業學校之論文網頁點我開啟連結
註: 此連結為研究生畢業學校所提供,不一定有電子全文可供下載,若連結有誤,請點選上方之〝勘誤回報〞功能,我們會盡快修正,謝謝!
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top