影像去模糊是電腦視覺和影像處理領域中一個基本的問題。影像去模糊的目標是從輸入的模糊影像恢復模糊函數和潛在的原始影像。它是類似於影像去雜訊和影像超解析技術的逆問題，但因為運動模糊函數和自然影像的多樣性而更具挑戰性。近年來解影像去模糊的方法包含兩個主要階段：估計模糊函數和復原影像。模糊函數估計演算法必須依靠輸入影像有足夠的顯著邊緣來推斷一個可靠的模糊函數。大多數的方法僅使用YUV顏色空間中的Y層（亮度層）來估計模糊函數。然而，研究指出RGB到Y的轉換可能會喪失對比度和邊緣的資訊。
在本篇論文中，我們專注於解決由手持相機而產生晃動所拍攝的單張影像去模糊問題。我們提出一種新的去模糊演算法，在估計模糊函數的階段同時考慮RGB顏色通道的資訊。我們也提出了兩個從色彩線模型推導而來的彩色影像先驗機率(color-image priors)。我們的彩色影像先驗機率不僅能恢復影像邊緣的對比度，還可以降低最佳化過程中由不準確的模糊函數所產生之影像雜訊。實驗結果顯示我們的演算法可增強估計模糊函數的穩定性和準確性，精確的模糊函數可以減少反卷積(deconvolution)產生的環效應(ringing artifacts)，並帶來更好的去模糊效果。我們的演算法不需要任何外部的學習資訊，但可以達到和以機器學習為基礎的方法相似的效能並勝過其他單一影像盲反卷積的方法。

Image deblurring in a fundamental problem in computer vision and image processing. The goal of image deblurring is to recover blur kernels and latent images from input blurred images. It is an inverse problem similar to image denoising and image super-resolution, but more challenging due to the diversity of motion blur kernels and natural images. Recent deblurring approaches include two main stages: blur kernel estimation and image restoration. Blur kernel estimation algorithms rely on sufficient significant edges on input images to infer a reliable blur kernel. Most approaches use only Y layer in the YUV color space to estimate blur kernels. However, researches show that RGB-to-Y conversion may lose contrast and edge information.
In this thesis, we focus on solving single-image motion deblurring problem in which blurry images are produced by camera shakes with hand-held shots. A novel deblurring algorithm is proposed, which consider RGB color channels together in the blur kernel estimation stage. We also propose two color-image priors derived from the color-line model. Our color-image priors not only restore image contrasts around edges, but also reduce image noises caused by inaccurate estimated blur kernels during optimization. Experimental results show that our algorithm can enhance the stability and the accuracy of blur kernel estimation. Accurate blur kernels can reduce the ringing artifacts generated from deconvolution, and lead to better deblurred results. Our algorithm do not need any external learning information, but the performance of our algorithm is comparable to the learning-based method and outperforms other single-image blind deconvolution methods.

口試委員會審定書 #
誌謝 i
中文摘要 iii
ABSTRACT v
CONTENTS vii
LIST OF FIGURES xi
LIST OF TABLES xvii
Chapter 1 Introduction 1
1.1 Motivation 1
1.2 Image Deblurring Overview 1
1.3 Main Contribution 4
1.4 Organization 4
Chapter 2 Non-Blind Deconvolution 5
2.1 Direct Inverse Filter 6
2.2 Wiener Deconvolution 8
2.3 Richardson-Lucy Deconvolution 11
2.4 Regularized RL Algorithm 13
2.5 MAP-based algorithm 18
2.5.1 MAP Formulation 18
2.5.2 Fast Deconvolution using Hyper-Laplacian priors 25
2.5.3 Patch-based Method 30
2.6 Summary 31
Chapter 3 Blind Deconvolution 33
3.1 Hardware or Multiple Images Approaches 34
3.2 Single-Image Blind Deconvolution 36
3.2.1 Naive MAP Approaches 37
3.2.2 Extended MAP Approaches 40
3.2.3 Maximum Marginal Distribution Approaches 53
3.3 Summary 64
Chapter 4 Proposed Image Deblurring Method 65
4.1 Observation 65
4.2 Framework 68
4.3 The Color-Line Model 71
4.4 The Proposed Deblurring Algorithm 77
4.4.1 Solving Intermediate Latent Image 78
4.4.2 Solving Blur Kernel 81
4.4.3 Final Non-Blind Deconvolution 82
4.5 Experimental Results 83
4.5.1 Quantitative Comparison on Sun and Hays’ dataset [44] 84
4.5.2 Deblurring on Real-World Photos 98
4.6 Discussion 108
4.7 Summary 113
Chapter 5 Conclusion and Future Work 115
5.1 Conclusion 115
5.2 Future Work 116
Appendix A 117
Appendix B 123
REFERENCE 127

A. Non-Blind Deconvolution
[1] N. Wiener, “Extrapolation, interpolation, and smoothing of stationary time series: with engineering applications,” Vol. 8. MIT press, 1964.
[2] W. H. Richardson, “Bayesian-based iterative method of image restoration,” JOSA, vol. 62, no. 1, pp. 55-59, 1972.
[3] L. B. Lucy, "An iterative technique for the rectification of observed distributions." The astronomical journal, vol. 79, pp. 745-754, 1974.
[4] L. A. Shepp and Y. Vardi, “Maximum likelihood reconstruction for emission tomography,” IEEE Transactions on Medical Imaging, vol. 1, no. 2, pp. 113-122, 1982.
[5] L. Yuan, J. Sun, L. Quan, and H. Y. Shum, “Progressive inter-scale and intra-scale non-blind image deconvolution,” ACM Transactions on Graphics, vol. 27, no. 3, pp. 74:1-74:10.
[6] Y. W. Tai, P. Tan, and M. S. Brown, “Richardson-lucy deblurring for scenes under a projective motion path,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 8, pp. 1603-1618, 2011.
[7] A. Levin, R. Fergus, F. Durand, and W. T. Freeman, “Image and depth from a conventional camera with a coded aperture,” ACM Transactions on Graphics, vol. 26, no. 3, pp. 328–341, 2007.
[8] D. Krishnan, and R. Fergus, “Fast Image Deconvolution using Hyper-Laplacian Priors,” In Proceedings of the Neural Information Processing Systems Conference, pp. 1033-1041, 2009.
[9] Y. Wang, J. Yang, W. Yin, and Y. Zhang, “A new alternating minimization algorithm for total variation image reconstruction,” SIAM Journal on Imaging Sciences, vol. 1, no. 3, pp. 248-272, 2008.
[10] D. Zoran, and Y. Weiss, “From learning models of natural image patches to whole image restoration,” In Proceedings of the IEEE International Conference on Computer Vision, pp. 479-486, 2011.
[11] N. Joshi, C. L. Zitnick, R. Szeliski, and D. Kriegman, "Image deblurring and denoising using color priors," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1550-1557, 2009.

B. Blind Deconvolution
[12] R. Fergus, B. Singh, A. Hertzmann, S. T. Roweis, and W. T. Freeman, “Removing camera shake from a single photograph,” ACM Transactions on Graphics, vol. 25, no. 3, pp. 787-794, 2006.
[13] S. K. Nayar, and M. Ben-Ezra, “Motion-based motion deblurring,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 6, pp. 689-698, 2004.
[14] R. Raskar, A. Agrawal, and J. Tumblin, “Coded exposure photography: motion deblurring using fluttered shutter,” ACM Transactions on Graphics, vol. 25, no. 3, pp. 795-804, 2006.
[15] S. Wang, T. Hou, J. Border, H. Qin, and R. L. Miller, “High-quality image deblurring with panchromatic pixels,” ACM Transactions on Graphics, vol. 31, no. 5, p. 120:1--120:11, 2012.
[16] L. Yuan, J. Sun, L. Quan, and H. Y. Shum, “Image deblurring with blurred/noisy image pairs,” ACM Transactions on Graphics, vol. 26, no. 3, p. 1:1-1:10, 2007.
[17] D. Kundur, and D. Hatzinakos, “Blind image deconvolution,” IEEE Signal Processing Magazine, vol. 13, no. 3, pp. 43-64, 1996.
[18] Y. L. You, and M. Kaveh, “Blind image restoration by anisotropic regularization,” IEEE Transactions on Image Processing, vol. 8, no. 3, pp. 396-407, 1999.
[19] A. Levin, “Blind motion deblurring using image statistics,” In Proceedings of the Neural Information Processing Systems Conference, pp. 841-848, 2006.
[20] R. N. Neelamani, H. Choi, and R. Baraniuk, “ForWaRD: Fourier-wavelet regularized deconvolution for ill-conditioned systems,” IEEE Transactions on Signal Processing, vol. 52, no. 2, pp. 418-433, 2004.
[21] A. Levin, Y. Weiss, F. Durand, and W. T. Freeman, “Understanding and evaluating blind deconvolution algorithms,” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1964-1971, 2009.
[22] J. Jia, “Single image motion deblurring using transparency,” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8, 2007.
[23] Q. Shan, J. Jia, and A. Agarwala, “High-quality motion deblurring from a single image,” ACM Transactions on Graphics, vol. 27, no. 3, p. 73:1-73:10, 2008.
[24] S. Cho, and S. Lee, “Fast motion deblurring,” ACM Transactions on Graphics, vol. 28, no. 5, p. 145:1-145:8, 2009.
[25] L. Xu, and J. Jia, “Two-phase kernel estimation for robust motion deblurring,” In Proceedings of the European Conference on Computer Vision, pp. 157-170, 2010.
[26] A. Levin, Y. Weiss, F. Durand, and W. T. Freeman, “Efficient marginal likelihood optimization in blind deconvolution,” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2657-2664, 2011.
[27] A. Goldstein, and R. Fattal, “Blur-kernel estimation from spectral irregularities,” In Proceedings of the European Conference on Computer Vision ,pp. 622-635, 2012.
[28] D. Krishnan, T. Tay, and R. Fergus, “Blind deconvolution using a normalized sparsity measure,” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pp. 233-240, 2011.
[29] L. Xu, S. Zheng, and J. Jia, "Unnatural l0 sparse representation for natural image deblurring," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1107-1114, 2013.
[30] L. Sun, S. Cho, J. Wang, and J. Hays, "Edge-based blur kernel estimation using patch priors," In Proceedings of the IEEE International Conference on Computational Photography, pp. 1-8, 2013.

C. Computer Vision Topics
[31] D. Martin, C. Fowlkes, D. Tal, and J. Malik, "A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics," In Proceedings of the IEEE International Conference on Computer Vision, pp. 416-423, 2001.
[32] E. P. Simoncelli, “Bayesian denoising of visual images in the wavelet domain,” Bayesian inference in wavelet-based models, Springer New York, pp. 291-308, 1999.
[33] A. Levin, D. Lischinski, and Y. Weiss, “A closed-form solution to natural image matting,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 228-242, 2008.
[34] C. Tomasi, and R. Manduchi, “Bilateral filtering for gray and color images,” In Proceedings of the IEEE International Conference on Computer Vision, pp. 839-846, 1998.
[35] S. Osher, and L. I. Rudin, “Feature-oriented image enhancement using shock filters,” SIAM Journal on Numerical Analysis, vol. 27, no. 4, pp. 919-940, 1990.
[36] P. Arbelaez, M. Maire, C. Fowlkes and J. Malik, "Contour Detection and Hierarchical Image Segmentation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 5, pp. 898-916, 2011.
[37] C. Lu, L. Xu, and J. Jia, “Contrast preserving decolorization,” In Proceedings of the IEEE International Conference on Computational Photography ,pp. 1-7, 2012.
[38] Y. Song, L. Bao, X. Xu, and Q. Yang, “Decolorization: is rgb2gray () out?” In ACM SIGGRAPH Asia Technical Briefs, p. 15:1-15:4, 2013.
[39] I. Omer, and M. Werman, "Color lines: Image specific color representation," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 946-953, 2004.
[40] Y. Bando, B. Y. Chen, and T. Nishita, "Extracting depth and matte using a color-filtered aperture," ACM Transactions on Graphics, vol. 27, no. 5, p. 134:1-134:10, 2008.
[41] E. P. Bennett, M. Uyttendaele, C. L. Zitnick, R. Szeliski, and S. B. Kang, "Video and image bayesian demosaicing with a two color image prior," In Proceedings of the European Conference on Computer Vision, pp. 508-521, 2006.
[42] C. Liu, R. Szeliski, S. B. Kang, C. L. Zitnick, and W. T. Freeman, "Automatic estimation and removal of noise from a single image," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 299-314, 2008.
[43] K. He, J. Sun, and X. Tang, "Guided image filtering," In Proceedings of the European Conference on Computer Vision, pp. 1–14, 2010.
[44] L. Sun, and J. Hays, "Super-resolution from internet-scale scene matching," In Proceedings of the IEEE International Conference on Computational Photography, pp. 1-12, 2012.
[45] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, 2004.
[46] Z. Hu, and M. H. Yang, "Good regions to deblur," In Proceedings of the European Conference on Computer Vision, pp. 59-72, 2012.

D. Theorems and Mathematics
[47] Y. Katznelson, “An introduction to harmonic analysis,” Cambridge University Press, 2004.
[48] R. N. Bracewell, ‘Fourier transform and its applications,” McGraw-Hill Education, 1980.
[49] J. R. Rice, and K. H. Usow, “The Lawson algorithm and extensions,” Mathematics of Computation, pp. 118-127, 1968.
[50] Y. Wang, and W. Yin, W. “Compressed sensing via iterative support detection,” Rice University, CAAM Technical Report, TR09-30, 2009.
[51] A. Beck, and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM Journal on Imaging Sciences, vol. 2, no. 1, pp. 183-202, 2009.
[52] J. M. Bioucas-Dias, and M. A. Figueiredo, "A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration," IEEE Transactions on Image Processing, vol. 16, no. 12, pp. 2992-3004, 2007.