臺灣博碩士論文加值系統

視覺影像的資訊與刺激佔有人類日常生活中的絕大部分，而對於視覺條件受限的使用者而言，具有像差的視覺系統例如近視或遠視會使得外界影像無法於視網膜清晰成像。而針對此類像差視覺系統，一般的修正方法都是配戴眼鏡來改變光的行進方向，但此校正法有時會帶來些許不便。本研究欲針對這些具像差性質的視覺系統，提出一基於深度學習的影像前處理演算法，以達到透過裸眼即可獲得清楚影像的目的。
但由於現實中不存在模擬人眼的成像系統，因此本研究利用離焦相機系統取代近視人眼，以實際拍攝照片驗證此預處理影像對於校正像差系統的可行性。首先，我們會利用相機分別對同一物體收集對焦(清晰)及離焦(模糊)的影像，以預測此離焦系統的點擴散函數。再者，利用深度殘差神經網路加上已得知的點擴散函數，進一步產生我們欲得到的預處理影像。最後，利用同樣的離焦系統觀測此預處理影像，並期望可獲得較為清晰的視覺影像。
由於產生預處理影像的方法是基於影像去模糊的原理，因此本篇論文會先介紹影像去模糊的技術及數學原理，再進而架構出此影像預處理系統的數學模型。再者，我們會先回顧近年來關於此技術的相關研究，以及在這些研究中所使用到的影像預處理方法，然後提出此篇論文使用的神經網路架構及模型。於第三章，我們會介紹實驗的步驟及流程，包含實驗架設環境、如何收集資料庫、以及從拍攝的影像中預測系統參數，最後以多種方法產生預處理影像，再由同樣離焦相機拍攝，以評估最終觀測到之影像是否比預處理前所看到之模糊影像保有較高的清晰度。最後我們會比較不同方法產生出之預處理影像的好壞，並提出我們對於此技術於應用層面的可行性及可能遇到的問題。

Stimulus of visual information occupies the vast majority of human daily life. For users with limited visual conditions, aberrations in the visual system such as myopia or hyperopia will make it impossible for external images to be clearly imaged on the retina. For such aberration imaging systems, the general correction method is changing the direction of light by glasses, but this correction method sometimes brings some inconvenience. Thus, this research aims to propose a deep learning-based image pre-processing method for correcting these aberrated imaging systems, so as to achieve the goal of obtaining clear images through naked eyes.
However, since there is no imaging system that perfectly mimics the human eye in reality, a defocus camera system is utilized to replace the myopic human eye in this research to verify the feasibility of image pre-correction for aberrated optical system. Firstly, we will use the camera to collect the on-focus (clear) and out-of-focus (blurred) images of the same object to estimate the point spread function of the defocus system. Then, the deep residual neural network and the known point spread function are used to further generate the pre-corrected images. Finally, the same defocus system is used to observe the preprocessed image, and is expected to obtain a clearer visual image.
Since the method of generating preprocessed images is based on the principle of image deblurring, we will first introduce the concept of image deblurring, the mathematical principles, and the mathematical model of our image pre-correction system. In chapter 2, we will first review the related studies on this topic in recent years and the corresponding image preprocessing methods in those prior arts. The deep neural network architecture and related model in this thesis will be proposed in the end of chapter 2. In chapter 3, the experimental procedures of this thesis will be given, including the experimental set-up, methodology for dataset collection, and estimation of system parameters from the captured images. Finally, preprocessed images generated by both traditional and proposed methods are captured by the same defocused optical system. In order to evaluate the performance, image quality assessment methods such as root-mean-squared error (RMSE), peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and information entropy are introduced as the figures of merit of pre-corrected system. Finally, we will discuss the feasibility and problems we may encounter with this technology in application.

Abstract (in Chinese) i
Abstract (in English) ii
Acknowledgement iv
Contents v
Lists of Figures vi
Lists of Tables ix

1 Introduction 1
1.1 Image deblurring 2
1.2 Motivation and image pre-correction system 3
2 Related works and the proposed methods 5
2.1 Related works 7
2.2 Traditional deconvolution methods 8
2.3 Deep learning-based deconvolution method 11
3 Experiment of pre-correction system 13
3.1 Experimental setup 15
3.1.1 Characterization of the response curve of sensor
17
3.1.2 Flat field correction 19
3.1.3 Imaging cropping and Hough transform 20
3.2 Estimation of system PSF 24
3.3 Pre-corrected and reconstructed images achievement
26
4 Analysis for pre-correction system 34
4.1 Merit figures 34
4.1.1 Root mean squared error 34
4.1.2 Peak signal-to-noise ratio 35
4.1.3 Structural similarity index measure 35
4.1.4 Information entropy 36
4.2 Results and discussions 38
5 Conclusion and future works 44
5.1 Conclusion 44
5.2 Future works 44

1.Alonso Jr, M., & Barreto, A. (2003). An Image Processing Approach to Pre-compensation for Higher-Order Aberrations in the Eye. Systemics Cybernetics and Informatics, 3(1), 67-85.
2.Bousseau, A. Virtual Glasses: The Myopic Revenge. 1st April 2008 Rodolphe Héliot & Antoine Zimmermann RAFT Editors, 25.
3.Montalto, C., Garcia-Dorado, I., Aliaga, D., Oliveira, M. M., & Meng, F. (2015). A total variation approach for customizing imagery to improve visual acuity. ACM Transactions on Graphics (TOG), 34(3), 1-16.
4.Sekko, E., Thomas, G., & Boukrouche, A. (1999). A deconvolution technique using optimal Wiener filtering and regularization. Signal processing, 72(1), 23-32.
5.Haldorsen, J. B., Miller, D. E., & Walsh, J. J. (1994). Multichannel Wiener deconvolution of vertical seismic profiles. Geophysics, 59(10), 1500-1511.
6.Biggs, D. S., & Andrews, M. (1997). Acceleration of iterative image restoration algorithms. Applied optics, 36(8), 1766-1775.
7.Dey, N., Blanc‐Feraud, L., Zimmer, C., Roux, P., Kam, Z., Olivo‐Marin, J. C., & Zerubia, J. (2006). Richardson–Lucy algorithm with total variation regularization for 3D confocal microscope deconvolution. Microscopy research and technique, 69(4), 260-266.
8.He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).
9.Levin, A. (2007). Blind motion deblurring using image statistics. In Advances in Neural Information Processing Systems (pp. 841-848).
10.Cho, S., & Lee, S. (2009). Fast motion deblurring. In ACM SIGGRAPH Asia 2009 papers (pp. 1-8).
11.Shan, Q., Jia, J., & Agarwala, A. (2008). High-quality motion deblurring from a single image. Acm transactions on graphics (tog), 27(3), 1-10.
12.Nayar, S. K., & Ben-Ezra, M. (2004). Motion-based motion deblurring. IEEE transactions on pattern analysis and machine intelligence, 26(6), 689-698.
13.Yuan, L., Sun, J., Quan, L., & Shum, H. Y. (2008). Progressive inter-scale and intra-scale non-blind image deconvolution. Acm Transactions on Graphics (TOG), 27(3), 1-10.
14.Fortunato, H. E., & Oliveira, M. M. (2014). Fast high-quality non-blind deconvolution using sparse adaptive priors. The Visual Computer, 30(6-8), 661-671.
15.Sun, L., Cho, S., Wang, J., & Hays, J. (2014, September). Good image priors for non-blind deconvolution. In European conference on computer vision (pp. 231-246). Springer, Cham.
16.Rossi, R. J. (2018). Mathematical statistics: an introduction to likelihood based inference. John Wiley & Sons.
17.Sinha, A., Lee, J., Li, S., & Barbastathis, G. (2017). Lensless computational imaging through deep learning. Optica, 4(9), 1117-1125.
18.Debevec, P. E., & Malik, J. (2008). Recovering high dynamic range radiance maps from photographs. In ACM SIGGRAPH 2008 classes (pp. 1-10).
19.Van Nieuwenhove, V., De Beenhouwer, J., De Carlo, F., Mancini, L., Marone, F., & Sijbers, J. (2015). Dynamic intensity normalization using eigen flat fields in X-ray imaging. Optics express, 23(21), 27975-27989.
20.Duda, R. O., & Hart, P. E. (1972). Use of the Hough transformation to detect lines and curves in pictures. Communications of the ACM, 15(1), 11-15.
21.Joshi, N., Szeliski, R., & Kriegman, D. J. (2008, June). PSF estimation using sharp edge prediction. In 2008 IEEE Conference on Computer Vision and Pattern Recognition (pp. 1-8). IEEE.
22.Shannon, C. E. (1948). A mathematical theory of communication. The Bell system technical journal, 27(3), 379-423.
23.Huffman, D. A. (1952). A method for the construction of minimum-redundancy codes. Proceedings of the IRE, 40(9), 1098-1101.
24.Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4), 600-612.