臺灣博碩士論文加值系統

When taking a photograph using digital devices such as digital cameras, usually we are not able to perfectly duplicate the scene we want to capture due to the limits of camera devices and storage space. Instead, we can only to sample the scene and store the color information of discrete space locations in the form of image pixels.

The above fact arose a problem when we want to display an image on a bigger display device or zoom-in the image for checking details: There are not enough information to display an image in any resolution higher than what it was taken originally. Similarly, If we scale down the resolution of an image due to reasons like storage constraints, we will not be able to scale it back easily.

The super-resolution problem is a heavily ill-posed problem, which means that a perfect solution does not exist. Which means, it is impossible to ”enlarge” an image perfectly. However, this also results in an interesting and useful research subject: How can we produce a better enlargement result with only the limited information we have?

In this thesis, we assume that the after the user took a picture of the scene (target image), he/she may also took one or more pictures of that scene from a closer position (reference image), or can obtain such images from other sources (like internet photo databases etc.). In order to enlarge the target image, we first adapt a modified general examplebased algorithm to enlarge the target while trying to reduce noises often seen in results of such algorithm. Then we match the target and reference images in order to find their relative positions. Since reference images are taken closer to the scene, they include more detail information. The detail information can be used to recover the missed details at the same location in the enlarged target image. Finally, we adapt a texture transfer algorithm to synthesize details for textures in the enlarged target image similar to those on the reference images.

Our result is better than traditional interpolation methods not only in the areas covered by the reference images, but also uncovered areas because of the modified general example-based method. It is also a highly flexible method since the number of reference images required is not a fixed number.

致謝 i
摘要 iii
Abstract v
List of Figures x
List of Tables xiii
Chapter 1 Introduction 1
1.1 Motivation 1
1.2 Problem Statement 3
1.3 Super-resolution By Examples And User Input 4
1.4 Thesis Organization 5
Chapter 2 RelatedWork 7
2.1 Super-resolution 7
2.1.1 Interpolation Methods 7
2.1.2 Signal-based Approach 8
2.1.3 Reconstruction-based approach 9
2.1.4 Example-based approach 11
2.2 Texture synthesis 12
2.3 Texture Transfer 13
2.4 Applications Using Photo Matching Technique 14
Chapter 3 System Overview 15
3.1 System workflow 16
3.1.1 General Example-based Method And Image Matching 16
3.1.2 User-guided Texture Transfer 16
3.2 System Specialty 18
Chapter 4 General Example-based Method 19
4.1 Overview Of General Example-based Method 19
4.2 Building The Lookup Table 21
4.3 Search and Synthesize 23
4.4 Improvements Over The Previous Methods 27
Chapter 5 Specific Example-based Method 31
5.1 Image Matching And Pasting 32
5.2 Deciding Correspondence Maps And Patch Aize 35
5.3 User-guided Texture Transfer 38
5.3.1 Similarity Term 39
5.3.2 Coherence Term 40
5.3.3 Structure Term 40
5.4 Minimum Error Boundary Cut 41
Chapter 6 Result 43
Chapter 7 Conclusion and Discussion 53
Bibliography 57

[1] M. Ashikhmin. Synthesizing natural textures. Proceedings of the 2001 symposium on Interactive 3D graphics, pages 217–226, 2001.
[2] M. Ashikhmin. Fast Texture Transfer. IEEE Computer Graphics and Applications, 23(4):38–43, 2003.
[3] S. Baker and T. Kanade. Limits on Super-Resolution and How to Break Them. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, pages 1167– 1183, 2002.
[4] C. Bishop, A. Blake, and B. Marthi. Super-resolution enhancement of video. Proc. Artificial Intelligence and Statistics, 1, 2003.
[5] S. Borman and R. Stevenson. Spatial resolution enhancement of low-resolution image sequences-a comprehensive review with directions for future research. University of Notre Dame, Tech. Rep, 1998.
[6] J. Canny. A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8(6):679–698, 1986.
[7] D. Chekhlov. Super-Resolution of Images. University of Bristol, Tech. Rep, 2005.
[8] S. Dai, M. Han, W. Xu, Y. Wu, and Y. Gong. Soft Edge Smoothness Prior for Alpha Channel Super Resolution. Computer Vision and Pattern Recognition, 2007. CVPR’07. IEEE Conference on, pages 1–8, 2007.
[9] J. De Bonet. Multiresolution sampling procedure for analysis and synthesis of texture images. Proceedings of the 24th annual conference on Computer graphics and interactive techniques (SIGGRAPH 1997), pages 361–368, 1997.
[10] Digitalcamerainfo.com. Infotrends: ’digital ecosystem’ closer than ever, 2008. http://www.digitalcamerainfo.com/content/InfoTrends-Digital-Ecosystem-Closer-Than-Ever-17535.htm.
[11] A. Efros, W. Freeman, and E. Fiume. Image Quilting for Texture Synthesis and Transfer. SIGGRAPH 2001, Computer Graphics Proceedings, pages 341–346, 2001.
[12] A. Efros and T. Leung. Texture synthesis by non-parametric sampling. International Conference on Computer Vision, 2(9):1033–1038, 1999.
[13] M. Elad and A. Feuer. Superresolution restoration of an image sequence: adaptivefiltering approach. Image Processing, IEEE Transactions on, 8(3):387–395, 1999.
[14] S. Farsiu, M. Robinson, M. Elad, and P. Milanfar. Fast and robust multiframe super resolution. Image Processing, IEEE Transactions on, 13(10):1327–1344, 2004.
[15] R. Fattal. Image upsampling via imposed edge statistics. ACM Transactions on Graphics (TOG), 26(3), 2007.
[16] M. Fischler and R. Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381–395, 1981.
[17] W. Freeman, T. Jones, and E. Pasztor. Example-Based Super-Resolution. IEEE Computer Graphics and Applications, 22:56–65, 2002.
[18] W. Freeman, E. Pasztor, and O. Carmichael. Learning Low-Level Vision. International Journal of Computer Vision, 40(1):25–47, 2000.
[19] J. Hays and A. Efros. Scene Completion Using Millions of Photographs. ACM Transactions on Graphics (SIGGRAPH 2007), 26(3), 2007.
[20] D. Heeger and J. Bergen. Pyramid-based texture analysis/synthesis. Proceedings of the 22nd annual conference on Computer graphics and interactive techniques (SIGGRAPH 1995), pages 229–238, 1995.
[21] A. Hertzmann, C. Jacobs, N. Oliver, B. Curless, and D. Salesin. Image analogies. Proceedings of the 28 th annual conference on Computer graphics and interactive techniques (SIGGRAPH 2001), 2001:327–340, 2001.
[22] M. Irani and S. Peleg. Motion analysis for image enhancement: Resolution, occlusion, and transparency. Journal of Visual Communication and Image Representation, 4(4):324–335, 1993.
[23] S. Kim, N. Bose, and H. Valenzuela. Recursive reconstruction of high resolution image from noisyundersampled multiframes. Acoustics, Speech, and Signal Processing [see also IEEE Transactions on Signal Processing], IEEE Transactions on, 38(6):1013–1027, 1990.
[24] S. Kondo, H. Amirshahi, T. Toma, and T. Aoki. Example-Based Super-Resolution Using Internet Photo Collection. SIGGRAPH 2007 posters, 2007.
[25] Z. Lin and H. Shum. Fundamental Limits of Reconstruction-Based Superresolution Algorithms under Local Translation. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, pages 83–97, 2004.
[26] D. Lowe. Object recognition from local scale-invariant features. International Conference on Computer Vision (ICCV), 2:1150–1157, 1999.
[27] D. Lowe. Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision (ICCV), 60(2):91–110, 2004.
[28] D. Mount and S. Arya. Ann: A library for approximate nearest neighbor searching, 2006. http://www.cs.umd.edu/ mount/ANN/.
[29] S. Park, M. Park, and M. Kang. Super-resolution image reconstruction: a technical overview. Signal Processing Magazine, IEEE, 20(3):21–36, 2003.
[30] T. Q. Pham, L. J. van Vliet, and K. Schutte. Resolution enhancement of low quality videos using a high-resolution frame. Visual Communications and Image Processing, vol. 6077 of Proceedings of SPIE, 2006.
[31] R. Schultz and R. Stevenson. Extraction of high-resolution frames from video sequences. Image Processing, IEEE Transactions on, 5(6):996–1011, 1996.
[32] N. Snavely, S. Seitz, and R. Szeliski. Photo tourism: exploring photo collections in 3D. ACM Transactions on Graphics (TOG), 25(3):835–846, 2006.
[33] W. Su and S. Kim. High-resolution restoration of dynamic image sequences. International Journal of Imaging Systems and Technology, 5(4):330–339, 1994.
[34] J. Sun, L. Liang, F. Wen, and H. Shum. Image vectorization using optimized gradient meshes. ACM transactions on graphics, (SIGGRAPH 2007), 26(3), 2007.
[35] A. Tekalp, M. Ozkan, and M. Sezan. High-resolution image reconstruction from lowerresolution imagesequences and space-varying image restoration. Acoustics, Speech, and Signal Processing, 1992. ICASSP-92., 1992 IEEE International Conference on, 3, 1992.
[36] A. M. Tekalp. Digital Video Processing, chapter 17. Prentice-Hall, 1995.
[37] H. Tong, M. Li, H. Zhang, and C. Zhang. Blur detection for digital images using wavelet transform. Multimedia and Expo, 2004. ICME’04. 2004 IEEE International Conference on, 1, 2004.
[38] L. Wei and M. Levoy. Fast texture synthesis using tree-structured vector quantization. Proceedings of the 27th annual conference on Computer graphics and interactive techniques (SIGGRAPH 2000), pages 479–488, 2000.