臺灣博碩士論文加值系統

在今日，高解析度的顯示器早已經在市場上普及，然而舊有的圖像或是影片中是以低解析度的格式儲存。若是想要讓其在此類顯示器上撥放，則需要先對其進行放大。為了提高影像的解析度，研究者們先後提出各種超解像方法。最常見的方法是利用插值法放大，這類方法較為簡易但是會有邊緣模糊或是紋理不清晰等缺點。人類視覺對於這些高頻的區域較為敏感，所以應該更小心的處理這些細節。基於相似區塊比較的超解像方法是一個熱門的研究方向，它的目標是進一步改善插值法高頻區域的放大效果。它利用兩張高低不同解析度圖片之間的關聯性，用適當的高解析區塊替換掉模糊的區塊。這些替換用高解析區塊是來自相似區塊搜尋的結果。由此可知，是否能在資料庫中找到合適的高解像區塊對演算法的結果影響很大。區塊搜尋的效果會取決於他們比較的依據，例如說比較普遍採用的搜尋方法是比較低頻的數值，但是此種方法忽略了邊緣以及紋理的重要性。基於梯度變化的區塊搜尋對於這些區域有很好的敏感度，但是在比較平滑的區域比較難判斷相似度。在這篇論文中，我們提出一個結合梯度及低頻的區塊搜尋流程用以提升搜尋方法的效果。我們採用了不同大小區塊會自我相似的假設將搜尋區域限制在一個小窗格內，並且仍可以得到相似的結果。演算法中採用了兩種不同的區塊搜尋方法，我們使用梯度變化來搜尋屬於邊緣的區塊；在平滑的區域則是使用低頻資訊。當兩個區塊的相似程度差距很小的時候，我們用他們的梯度方向做進一步的比較。在實驗結果中，我們的結果在PSNR 與SSIM 的平均數值比起只使用低頻搜尋的超解像方法高了9.5％與3％，比只使用梯度搜尋的超解像方法高了10％與5％。同時我們簡化了計算梯度方向的步驟，使重建高頻的執行時間與基於梯度方向的超解像方法減少了54％。

Nowadays, High-resolution (HR) monitors have been very popular in the market. However, many legacy images/videos are stored in low-resolution (LR) formats These LR files need to be enlarged before playing on HR monitors. Super-Resolution(SR) algorithms have been proposed to enhance the image resolution. The most common way to do up-scaling is using interpolation-based methods. These kind of methods are simple but may suffer from noticeable blurring edges or unclear textures. Human vision is more sensitive to these high-frequency(HF)-rich regions, so these regions should not be processed only by interpolated method. Example-based method is one of the most popular approaches, which aim to enhance the HF of interpolated results. The concept of these methods are to make use of the correspondence between LR and HR image. In Example-based methods, the LR patches are replaced by their HR versions. These HR patches are selected by similar patch search process. Hence, whether suitable HR patch can be found in the patch search have a significant impact on the final result. The effect of search methods depends on what they use in comparison. For example, the common way to search for similar patches is comparing the low-frequency(LF) values. However, edge and texture regions are not suitable since only raw pixel information is used. The radient-based patch search is proposed to enhance the effect on these HF regions. Nevertheless, it’s hard to compare the gradient difference on smooth regions. In this paper, We proposed a hybrid patch search process that combine the gradient and LF-based patch search for further enhance the effect of above methods. We use the assumption of local self-similarity to limit the search area within a small window but get similar results in most cases. In the proposed framework, two different patch search methods are applied. For edge regions, we use the gradient-based patch search; in smooth regions, low frequency-based patch search is adopted. When the difference is close between two patches of the hybrid patch search, we further compare the gradient direction for verification. In the experimental results, comparing with the SR methods which use LF-based patch search only, our proposed method get 9.5% higher at average PSNR values and 3% higher at SSIM average values. The proposed method gets 10% higher at average PSNR values and 5% higher at SSIM average values compared with the SR methods which use gradient-based patch search only. At the same time, we simplify the patch verification process. The computation for HF reconstruction is reduced about 54% compared to the gradient-based SR method.

1 Introduction 1
1.1 Background of image super resolution . . . . . . . . . . . . . . . . . 1
1.2 Motivation and Problem Description . . . . . . . . . . . . . . . . . 3
1.3 Goal and Contribution . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 Related Work 5
2.1 Interpolation-based approaches . . . . . . . . . . . . . . . . . . . . 5
2.2 Multi Image Super-Resolution . . . . . . . . . . . . . . . . . . . . . 6
2.3 Reconstruction-based approaches . . . . . . . . . . . . . . . . . . . 6
2.4 Learning-based approaches . . . . . . . . . . . . . . . . . . . . . . . 7
3 Single Image Super-resolution using hybrid patch search and selfsimilarity
12
3.1 Principles and Flowchart . . . . . . . . . . . . . . . . . . . . . . . . 12
3.2 Pre-processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.3 Hybrid Patch Search Process . . . . . . . . . . . . . . . . . . . . . 17
3.4 Generate HR image . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4 Experimental Results 24
4.1 Evaluating the performance improvements . . . . . . . . . . . . . . 26
4.2 Computation Complexity . . . . . . . . . . . . . . . . . . . . . . . 29
4.3 Visual and Quantitative Comparisons . . . . . . . . . . . . . . . . . 31
5 Conclusion and Future work 43
5.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
5.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
ii

[1] W. T. Freeman, T. R. Jones, and E. C. Pasztor, “Example-based superresolution,” Computer Graphics and Applications, IEEE, vol. 22, no. 2, pp. 56–65, 2002.
[2] D. Glasner, S. Bagon, and M. Irani, “Super-resolution from a single image,” Computer Vision, 2009 IEEE 12th International Conference on, pp. 349–356, 2009.
[3] G. Freedman and R. Fattal, “Image and video upscaling from local selfexamples,” ACM Transactions on Graphics (TOG), vol. 30, no. 2, pp. 1–10, 2011.
[4] L. Kuo, T. H. Wang, and C. T. Chiu, “Gradient-based image up-scaling with local self similarity,” Signal and Information Processing (GlobalSIP), 2014 IEEE Global Conference on, pp. 960–964, Dec 2014.
[5] “Kodak true color images dataset,” http://r0k.us/graphics/kodak/, accessed: 2016-06-26.
[6] R. Fattal, “Image upsampling via imposed edge statistics,” ACM Transactions on Graphics (TOG), vol. 26, no. 3, 2007.
[7] R. G. Keys, “Cubic convolution interpolation for digital image processing,” Acoustics, Speech and Signal Processing, IEEE Transactions on, vol. 29, no. 6, pp. 1153–1160, 1981.
[8] S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: a technical overview,” Signal Processing Magazine, IEEE, vol. 20, no. 3, pp. 21–36, 2003.
[9] R. Tsai and T. S. Huang, “Multiframe image restoration and registration,” Advances in computer vision and Image Processing, vol. 1, no. 2, pp. 317–339, 1984.
[10] M. Irani and S. Peleg, “Improving resolution by image registration,” CVGIP: Graphical models and image processing, vol. 53, no. 3, pp. 231–239, 1991.
[11] J. Sun, Z. Xu, and H.-Y. Shum, “Image super-resolution using gradient profile prior,” Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on, pp. 1–8, 2008.
[12] Y.-W. Tai, S. Liu, M. S. Brown, and S. Lin, “Super resolution using edge prior and single image detail synthesis,” Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, pp. 2400–2407, 2010.
[13] M. E. Tipping and C. M. Bishop, “Bayesian image super resolution,” Sep. 12 2006, uS Patent 7,106,914.
[14] S. Geman and D. Geman, “Stochastic relaxation, gibbs distributions, and the bayesian restoration of images,” IEEE Transactions on pattern analysis and machine intelligence, no. 6, pp. 721–741, 1984.
[15] J. Yang, J. Wright, T. Huang, and Y. Ma, “Image super-resolution as sparse representation of raw image patches,” Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on, pp. 1–8, 2008.
[16] C.-Y. Yang, J.-B. Huang, and M.-H. Yang, “Exploiting self-similarities for single frame super-resolution.” Springer, 2010, pp. 497–510.
[17] M.-C. Yang and Y.-C. F. Wang, “A self-learning approach to single image super-resolution,” Multimedia, IEEE Transactions on, vol. 15, no. 3, pp. 498– 508, 2013.
[18] K. Zhang, X. Gao, D. Tao, and X. Li, “Single image super-resolution with nonlocal means and steering kernel regression,” Image Processing, IEEE Transactions on, vol. 21, no. 11, pp. 4544–4556, 2012.
[19] J. Yang, J. Wright, T. S. Huang, and Y. Ma, “Image super-resolution via sparse representation,” Image Processing, IEEE Transactions on, vol. 19, no. 11, pp. 2861–2873, 2010.
[20] T. Peleg and M. Elad, “A statistical prediction model based on sparse representations for single image super-resolution,” Image Processing, IEEE Transactions on, vol. 23, no. 6, pp. 2569–2582, 2014.
[21] X. Li and M. T. Orchard, “New edge-directed interpolation,” Image Processing, IEEE Transactions on, vol. 10, no. 10, pp. 1521–1527, 2001.
[22] H. S. Hou and H. Andrews, “Cubic splines for image interpolation and digital filtering,” Acoustics, Speech and Signal Processing, IEEE Transactions on, vol. 26, no. 6, pp. 508–517, 1978.
[23] S. Baker and T. Kanade, “Limits on super-resolution and how to break them,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 24, no. 9, pp. 1167–1183, 2002.
[24] Z. Lin and H.-Y. Shum, “Fundamental limits of reconstruction-based superresolution algorithms under local translation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 1, pp. 83–97, 2004.
[25] J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, “Non-local sparse models for image restoration,” 2009 IEEE 12th International Conference on Computer Vision, pp. 2272–2279, 2009.
[26] K. Zhang, X. Gao, D. Tao, and X. Li, “Single image super-resolution with multiscale similarity learning,” IEEE transactions on neural networks and learning systems, vol. 24, no. 10, pp. 1648–1659, 2013.
[27] A. Buades, B. Coll, and J.-M. Morel, “A non-local algorithm for image denoising,” 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 2, pp. 60–65, 2005.
[28] S. T. Roweis and L. K. Saul, “Nonlinear dimensionality reduction by locally linear embedding,” Science, vol. 290, no. 5500, pp. 2323–2326, 2000.
[29] J.-B. Huang, A. Singh, and N. Ahuja, “Single image super-resolution from transformed self-exemplars,” 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5197–5206, 2015.
[30] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE transactions on image processing, vol. 13, no. 4, pp. 600–612, 2004.
[31] Shan, Qi and Li, Zhaorong and Jia, Jiaya and Tang, Chi-Keung, “Fast image/video upsampling,” ACM Transactions on Graphics (TOG), vol. 27, no. 5, pp. 153—157, 2008.