跳到主要內容

臺灣博碩士論文加值系統

(44.211.31.134) 您好!臺灣時間:2024/07/24 17:31
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果 :::

詳目顯示

我願授權國圖
: 
twitterline
研究生:曾泓硯
研究生(外文):Hung-Yen Tseng
論文名稱:無須運動估計之影像原始檔的連拍超解析
論文名稱(外文):Burst Super-Resolution in Raw Domain without Explicit Motion Estimation
指導教授:莊永裕
指導教授(外文):Yung-Yu Chuang
口試委員:吳賦哲葉正聖
口試委員(外文):Fu-Che WuJeng-Sheng Yeh
口試日期:2023-06-29
學位類別:碩士
校院名稱:國立臺灣大學
系所名稱:資訊工程學系
學門:工程學門
學類:電資工程學類
論文種類:學術論文
論文出版年:2023
畢業學年度:111
論文頁數:33
中文關鍵詞:超解析影像還原去噪連拍
外文關鍵詞:Super-ResolutionImage RestorationDenoiseDemosaicingBurst
DOI:10.6342/NTU202301669
相關次數:
  • 被引用被引用:0
  • 點閱點閱:43
  • 評分評分:
  • 下載下載:0
  • 收藏至我的研究室書目清單書目收藏:0
本篇論文想要解決影像重建任務中的超解析問題。在這樣的任務中,我們的輸入會是一連串的連拍、低解析度的影像,而我們希望最終重建的影像是一張高解析度且細節都有保留的清晰影像。由於在拍攝影像的過程中,會有各種不同的情況,導致最終拍攝出來的影像上會出現強弱不一的噪點,並且由於我們選用原始檔的格式當作輸入,所以在超解析的任務上,我們還需要同時處理去噪(Denoising) 以及去馬賽克 (Demosaicing) 這兩項任務。本篇論文提出了一個神經網路去學習如何根據低解析度的輸入,最終映射到一個高解析度的輸出。我們認為利用 RGB 的影像去訓練一個光流的網路對於 Raw 影像的對齊演算法並沒有起到非常好的效果。此外,擷取影像間的關聯性對於對齊的演算法也有不小的幫助。最終實驗出來的結果表明我們的方法在數值上以及視覺上表現得都比過去的那些方法還要好,也說明了我們的方法是有效的。
This thesis addresses the challenging task of image restoration, specifically Super-Resolution (SR). Recent advancements in Deep Learning have significantly improved the quality of image restoration results. In this work, we propose a method for reconstructing high-resolution images from a burst of raw, low-resolution images. Super-Resolution is a complex problem with no closed-form solution. Moreover, our input burst images are corrupted by shot and read noise in the raw domain, while the desired SR images are in the noise-free RGB domain. Hence, our method needs to address denoising and demosaicing tasks alongside Super-Resolution.
Our proposed method utilizes a deep neural network to learn the mapping between the input burst images and the corresponding high-resolution images. We also demonstrate that an optical flow estimation network pre-trained on RGB images performs inferiorly compared to our end-to-end training alignment module. Furthermore, we discover that the channel attention mechanism outperforms the global spatial attention mechanism for the alignment module.
Extensive experiments validate the effectiveness of our method, surpassing the state-of-the-art performance in both qualitative and quantitative evaluation metrics.
Verification Letter from the Oral Examination Committee i
Acknowledgements iii
摘要v
Abstract vii
Contents ix
List of Figures xiii
List of Tables xv
Chapter 1 Introduction 1
1.1 Motivation and Challenges . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2.1 Single Image SuperResolution . . . . . . . . . . . . . . . . . . . . 2
1.2.2 MultiFrame SuperResolution . . . . . . . . . . . . . . . . . . . . 3
1.2.3 Previous work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Chapter 2 Data Preprocessing 5
2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1.1 RGB Images Unprocessing . . . . . . . . . . . . . . . . . . . . . . 6
2.1.2 Burst Image Generation . . . . . . . . . . . . . . . . . . . . . . . . 6
2.1.3 Read and Shot Noise . . . . . . . . . . . . . . . . . . . . . . . . . 6
Chapter 3 Super Resolution 9
3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2 Alignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.2.1 Implicit Alignment module . . . . . . . . . . . . . . . . . . . . . . 11
3.2.2 Nonlinear Activation Free Module . . . . . . . . . . . . . . . . . . 12
3.3 Fusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.4 Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Chapter 4 Experiments 17
4.1 Dataset Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.2 Training Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.3 Evaluation Metric . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
4.4 Visualization Results . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Chapter 5 Denoising 21
5.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
5.2 Data preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
5.3 Denoise Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
5.3.1 Single Image Denoise . . . . . . . . . . . . . . . . . . . . . . . . . 22
5.3.2 Multiple Image Denoise . . . . . . . . . . . . . . . . . . . . . . . . 22
5.4 Recovered Image Quality . . . . . . . . . . . . . . . . . . . . . . . 23
5.4.1 Denoise Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
5.4.2 How Noise Level Effect Result . . . . . . . . . . . . . . . . . . . . 23
Chapter 6 Ablation Studies 25
6.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
6.2 Alignment schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
6.3 Reconstruction modules . . . . . . . . . . . . . . . . . . . . . . . . 26
6.4 Alignment module . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
6.4.1 Attention mechanism . . . . . . . . . . . . . . . . . . . . . . . . . 26
6.4.2 Swin Transformer . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
6.5 Denoising before superresolution . . . . . . . . . . . . . . . . . . . 27
Chapter 7 Conclusion 29
References 31
G. Bhat, M. Danelljan, L. Van Gool, and R. Timofte. Deep burst super-resolution. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9205–9214, 2021.
G. Bhat, M. Danelljan, F. Yu, L. Van Gool, and R. Timofte. Deep reparametrization of multi-frame super-resolution and denoising. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 2440–2450, 2021.
T. Brooks, B. Mildenhall, T. Xue, J. Chen, D. Sharlet, and J. T. Barron. Unprocessing images for learned raw denoising. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
L. Chen, X. Chu, X. Zhang, and J. Sun. Simple baselines for image restoration. arXiv preprint arXiv:2204.04676, 2022.
X. Chu, L. Chen, and W. Yu. Nafssr: Stereo image superresolution using nafnet. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1238–1247, 2022.
A. Dudhane, S. W. Zamir, S. Khan, F. S. Khan, and M.-H. Yang. Burst image restoration and enhancement. In CVPR, 2022.
S. Guo, Z. Yan, K. Zhang, W. Zuo, and L. Zhang. Toward convolutional blind denoising of real photographs, 2019.
A. Ignatov, L. Van Gool, and R. Timofte. Replacing mobile camera isp with a single deep learning model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2020.
Y. Jo, S. Yang, and S. J. Kim. Investigating loss functions for extreme super-resolution. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1705–1712, 2020.
B. Lecouat, J. Ponce, and J. Mairal. Lucas-kanade reloaded: End-to-end super-resolution from raw image bursts. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 2350–2359, 2021.
J. Liang, J. Cao, G. Sun, K. Zhang, L. Van Gool, and R. Timofte. Swinir: Image restoration using swin transformer. In 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), pages 1833–1844, 2021.
Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo. Swin transformer: Hierarchical vision transformer using shifted windows, 2021.
Z. Luo, Y. Li, S. Cheng, L. Yu, Q. Wu, Z. Wen, H. Fan, J. Sun, and S. Liu. Bsrt: Improving burst super-resolution with swin transformer and flow-guided deformable alignment. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 997–1007, 2022.
K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition, 2014.
Y. Tian, Y. Zhang, Y. Fu, and C. Xu. Tdan: Temporally-deformable alignment network for video super-resolution. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3357–3366, 2020.
X. Wang, K. C. Chan, K. Yu, C. Dong, and C. C. Loy. Edvr: Video restoration with enhanced deformable convolutional networks. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1954–1963, 2019.
X. Wang, K. Yu, S. Wu, J. Gu, Y. Liu, C. Dong, Y. Qiao, and C. C. Loy. Esrgan: Enhanced super-resolution generative adversarial networks. In L. Leal-Taixé and S. Roth, editors, Computer Vision - ECCV 2018 Workshops, pages 63–79, Cham, 2019. Springer International Publishing.
X. Zhu, H. Hu, S. Lin, and J. Dai. Deformable convnets v2: More deformable, better results. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9300–9308, 2019.
電子全文 電子全文(網際網路公開日期:20280717)
連結至畢業學校之論文網頁點我開啟連結
註: 此連結為研究生畢業學校所提供,不一定有電子全文可供下載,若連結有誤,請點選上方之〝勘誤回報〞功能,我們會盡快修正,謝謝!
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top