跳到主要內容

臺灣博碩士論文加值系統

(44.211.117.197) 您好!臺灣時間:2024/05/22 01:12
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果 :::

詳目顯示

我願授權國圖
: 
twitterline
研究生:陳昭榮
研究生(外文):CHEN, ZHAO-RONG
論文名稱:基於視覺注意力預測與生成對抗網路進行單張影像之反射消除
論文名稱(外文):Single Image Reflection Removal Using Visual Attention Prediction and Generative Adversarial Networks
指導教授:郭天穎郭天穎引用關係
指導教授(外文):KUO, TIEN-YING
口試委員:楊士萱陳建中蘇柏齊
口試委員(外文):YANG, SHIH-HSUANCHEN, JIANN-JONESU, PO-CHYI
口試日期:2022-09-02
學位類別:碩士
校院名稱:國立臺北科技大學
系所名稱:電機工程系
學門:工程學門
學類:電資工程學類
論文種類:學術論文
論文出版年:2022
畢業學年度:110
語文別:中文
論文頁數:44
中文關鍵詞:反射反射消除圖像恢復深度學習生成對抗網路
外文關鍵詞:ReflectionReflection RemovalImage RestorationDeep LearningGenerative Adversarial Network
相關次數:
  • 被引用被引用:0
  • 點閱點閱:94
  • 評分評分:
  • 下載下載:1
  • 收藏至我的研究室書目清單書目收藏:0
當隔著玻璃等透明介質拍攝玻璃之後的場景時,例如隔窗拍攝窗外風景,經常會出現反射現象並破壞照片。因此,本論文的研究即為如何消除反射現象在照片上所造成的影響。雖已有文獻針對單張影像反射消除進行研究,但反射消除結果不盡理想,且偏向對整張影像進行恢復並未著重關注於反射區域。如何達到了解反射區域,是一項需克服的問題。而實驗中所收集到的真實影像有限,資料集缺少多樣性亦會影響模型的效能,因此資料集使用藉由合成方式所製作的合成影像。
本文提出針對單張影像反射消除之深度學習模型,使用生成對抗網路,其生成模型與判別模型的相互對抗,使網路會遵循自然影像分布,讓生成結果能更接近真實影像。生成模型預測反射區域在影像中出現的位置,並生成反射區域之注意力圖,注意力圖將引導生成網路關注於反射區域,讓反射區域細節恢復得更完善。此外,利用判別模型使得影像上反射區域所生成的影像能更逼近真實影像,達到反射消除。本文與相關文獻模型相比,反射消除結果在傳統評估方式上有所提升,並且在視覺上觀感更佳。

When shooting a scene behind a transparent medium like glass, such as shooting the scenery through window, reflection often appears and destroys the photos. Therefore, the research of this paper is how to eliminate the reflection effect on photos. Although there have been literature studies on reflection removal for a single image, the results of reflection removal are not satisfactory, and the restoration doesn't focus on the reflection area but on the entire image. How to get the reflection area accurately is a problem to be overcome. Since the number of real photos that could be collected is restricted, and the lack of the diversity in the dataset will affect the effectiveness of the model, we use synthetic methods to produce synthetic images.
This paper proposes a deep learning model for single image reflection removal based on the Generative Adversarial Network. The adversary between the generative model and the discriminant model makes the network follow the natural image distribution, so that the generated results can be closer to the real image. The generative model predicts the location of the reflection area in the image, and generates an attention map of the reflection area. The attention map will guide the generation network to focus on the reflection area, so that the details of the reflection area can be restored better. In addition, the discriminative model is used to make the image generated on the reflection area closer to the real image, so as to achieve reflection removal. When compared to existing methods, our proposed method achieves a significant improvement in the traditional evaluation metrics and the deep learning-based evaluation metrics, as well as giving a better human visual experience.

摘要 i
ABSTRACT ii
致謝 iv
目錄 v
表目錄 vii
圖目錄 viii
第一章 緒論 1
1.1 研究動機與目的 1
1.2 研究方法 2
1.3 研究貢獻 3
1.4 論文組織架構 3
第二章 相關知識及文獻回顧 4
2.1 影像反射消除方法 4
2.1.1 多張影像方法 5
2.1.2 基於深度學習之單張影像方法 5
2.1.3 深度學習影像反射消除方法總結 8
第三章 本論文提出方法 9
3.1 本文演算法流程 9
3.2 單張影像反射消除模型 10
3.2.1 資料集介紹 10
3.2.2 單張影像反射消除模型介紹 11
3.2.3 單張影像反射消除使用之損失函數與超參數設定 16
第四章 實驗結果與分析 19
4.1 實驗設備及環境 19
4.2 模型評估方法 19
4.2.1 影像反射消除評估方式 20
4.3 反射消除模型消融實驗 20
4.4 模型結果與分析 22
4.4.1 影像反射消除結果 22
4.4.2 失敗案例分析 37
第五章 結論 40
參考資料 41

[1]Q. Fan, J. Yang, G. Hua, B. Chen, and D. Wipf, "A generic deep architecture for single image reflection removal and image smoothing," in Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 3238-3247.
[2]K. Wei, J. Yang, Y. Fu, D. Wipf, and H. Huang, "Single image reflection removal exploiting misaligned training data and network enhancements," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 8178-8187.
[3]N. Kong, Y.-W. Tai, and J. S. Shin, "A physically-based approach to reflection separation: from physical modeling to constrained optimization," IEEE transactions on pattern analysis and machine intelligence, vol. 36, no. 2, pp. 209-221, 2013.
[4]X. Guo, X. Cao, and Y. Ma, "Robust separation of reflection from multiple images," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 2187-2194.
[5]Y. Li and M. S. Brown, "Exploiting reflection change for automatic reflection removal," in Proceedings of the IEEE international conference on computer vision, 2013, pp. 2432-2439.
[6]T. Xue, M. Rubinstein, C. Liu, and W. T. Freeman, "A computational approach for obstruction-free photography," ACM Transactions on Graphics (TOG), vol. 34, no. 4, pp. 1-11, 2015.
[7]J. Yang, H. Li, Y. Dai, and R. T. Tan, "Robust optical flow estimation of double-layer images under transparency or reflection," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 1410-1419.
[8]A. Levin and Y. Weiss, "User assisted separation of reflections from a single image using a sparsity prior," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 9, pp. 1647-1654, 2007.
[9]X. Zhang, R. Ng, and Q. Chen, "Single image reflection separation with perceptual losses," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4786-4794.
[10]R. Wan, B. Shi, L.-Y. Duan, A.-H. Tan, and A. C. Kot, "Crrn: Multi-scale guided concurrent reflection removal network," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 4777-4785.
[11]R. Wan, B. Shi, H. Li, L.-Y. Duan, A.-H. Tan, and A. C. Kot, "CoRRN: Cooperative reflection removal network," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 12, pp. 2969-2982, 2019.
[12]J. Yang, D. Gong, L. Liu, and Q. Shi, "Seeing deeply and bidirectionally: A deep learning approach for single image reflection removal," in Proceedings of the european conference on computer vision (ECCV), 2018, pp. 654-669.
[13]C. Li, Y. Yang, K. He, S. Lin, and J. E. Hopcroft, "Single image reflection removal through cascaded refinement," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 3565-3574.
[14]K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778.
[15]X. Shi, Z. Chen, H. Wang, D.-Y. Yeung, W.-K. Wong, and W.-c. Woo, "Convolutional LSTM network: A machine learning approach for precipitation nowcasting," Advances in neural information processing systems, vol. 28, 2015.
[16]I. Goodfellow et al., "Generative adversarial nets," Advances in neural information processing systems, vol. 27, 2014.
[17]P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, "Image-to-image translation with conditional adversarial networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1125-1134.
[18]R. Szeliski, S. Avidan, and P. Anandan, "Layer extraction from multiple images containing reflections and transparency," in Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No. PR00662), 2000, vol. 1, pp. 246-253: IEEE.
[19]B. Sarel and M. Irani, "Separating transparent layers through layer information exchange," in European Conference on Computer Vision, 2004, pp. 328-341: Springer.
[20]K. Gai, Z. Shi, and C. Zhang, "Blind separation of superimposed moving images using image statistics," IEEE transactions on pattern analysis and machine intelligence, vol. 34, no. 1, pp. 19-32, 2011.
[21]S. N. Sinha, J. Kopf, M. Goesele, D. Scharstein, and R. Szeliski, "Image-based rendering for scenes with reflections," ACM Transactions on Graphics (TOG), vol. 31, no. 4, pp. 1-10, 2012.
[22]C. Sun, S. Liu, T. Yang, B. Zeng, Z. Wang, and G. Liu, "Automatic reflection removal using gradient intensity and motion cues," in Proceedings of the 24th ACM international conference on Multimedia, 2016, pp. 466-470.
[23]B.-J. Han and J.-Y. Sim, "Reflection removal using low-rank matrix completion," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 5438-5446.
[24]H. Farid and E. H. Adelson, "Separating reflections and lighting using independent components analysis," in Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149), 1999, vol. 1, pp. 262-267: IEEE.
[25]P. Wieschollek, O. Gallo, J. Gu, and J. Kautz, "Separating reflection and transmission images in the wild," in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 89-104.
[26]A. Agrawal, R. Raskar, S. K. Nayar, and Y. Li, "Removing photography artifacts using gradient projection and flash-exposure sampling," in ACM SIGGRAPH 2005 Papers, 2005, pp. 828-835.
[27]Y. Y. Schechner, N. Kiryati, and R. Basri, "Separation of transparent layers using focus," International Journal of Computer Vision, vol. 39, no. 1, pp. 25-39, 2000.
[28]P. Kalwad, D. Prakash, V. Peddigari, and P. Srinivasa, "Reflection removal in smart devices using a prior assisted independent components analysis," in Digital Photography XI, 2015, vol. 9404, pp. 31-40: SPIE.
[29]K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014.
[30]S. Kim, Y. Huo, and S.-E. Yoon, "Single image reflection removal with physically-based training images," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 5164-5173.
[31]Y.-C. Chang, C.-N. Lu, C.-C. Cheng, and W.-C. Chiu, "Single image reflection removal with edge guidance, reflection classifier, and recurrent decomposition," in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021, pp. 2033-2042.
[32]J. Hu, L. Shen, and G. Sun, "Squeeze-and-excitation networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 7132-7141.
[33]V. Mnih, N. Heess, and A. Graves, "Recurrent models of visual attention," Advances in neural information processing systems, vol. 27, 2014.
[34]K. Gregor, I. Danihelka, A. Graves, D. Rezende, and D. Wierstra, "Draw: A recurrent neural network for image generation," in International conference on machine learning, 2015, pp. 1462-1471: PMLR.
[35]B. Zhao, X. Wu, J. Feng, Q. Peng, and S. Yan, "Diversified visual attention networks for fine-grained object classification," IEEE Transactions on Multimedia, vol. 19, no. 6, pp. 1245-1256, 2017.
[36]M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman, "The pascal visual object classes (voc) challenge," International journal of computer vision, vol. 88, no. 2, pp. 303-338, 2010.
[37]R. Wan, B. Shi, L.-Y. Duan, A.-H. Tan, and A. C. Kot, "Benchmarking single-image reflection removal algorithms," in Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 3922-3930.
[38]O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmentation," in International Conference on Medical image computing and computer-assisted intervention, 2015, pp. 234-241: Springer.
[39]D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," arXiv preprint arXiv:1412.6980, 2014.
[40]M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, "Gans trained by a two time-scale update rule converge to a local nash equilibrium," Advances in neural information processing systems, vol. 30, 2017.
[41]R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, "The unreasonable effectiveness of deep features as a perceptual metric," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 586-595.
[42]C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, "Rethinking the inception architecture for computer vision," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2818-2826.


電子全文 電子全文(網際網路公開日期:20270920)
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top
無相關期刊