臺灣博碩士論文加值系統

影像是二維的訊號，然而受限於取像裝置，影像品質與面積大小往往不能盡如人意。故而依靠影像處理的技巧，將解析度不足的影像或影像序列放大(magnification)是近來一項重要的研究課題。
影像放大，亦可說是解析度加強(resolution enhancement)，就是利用可得到的低解析度的影像輸入經由一個放大機制，而機制的輸出就是一張高解析度的影像。這個放大機制的技巧或方法，就是我們要研究的重點，大致上可以分為兩個方向，其一是靜態影像放大(static image enlargement)，其二是動態影像放大(motion image enlargement)。
在靜態方面我們以單張影像資料作為輸入，運用預估高頻資訊(保留edge)來強化放大的品質。我們將兩種既有的方法加以整合和修改，並提出一種新的而且又直覺的方法－以邊線模型為基礎用來保留邊線的高頻資訊，令放大後的影像不會過於模糊(blur)或人工化(artifact or jagged)。而動態影像方面則以一串影像序列(image sequence)或一段video作為輸入。利用影像序列所具有的空間及時間上的資訊，再套用動作預估(motion estimation)的方法來找出影像間的對應關係，進而取其對應資訊以為己用。本論文以Bayesian MAP的方法為主架構，嘗試加入新的基因演算法(Genetic Algorithm)的求解方式，目的就是要從這些影像序列產生出一張高解析度的影像，即一般所謂的超解析度(super-resolution)。

The two-dimensional (2D) images are usually under less or poor image resolution when acquired with limited imaging equipments. Therefore, the image resizing (or image scaling) is one of the most important research problems in image processing applications. It is intended to enlarge (or enhance) the poor quality (i.e. small size) of image or image sequence (a period of video).
Image magnification is similar to the image resolution enhancement. We use an available low-resolution single image as the input to the magnified mechanism. The output of the magnified mechanism is just the desired high-resolution image. The algorithm of the mechanism is the main subject in our investigation. The algorithms of image magnification are categorized into two directions. One is the static image enlargement and the other is the motion image enlargement.
In static image enlargement, we estimate the high-frequency (edge-preserving) information from the input image. It is then used to change the weight of data points based on the linear approximation scheme to suppress the over-burring and to introduce the edge-directed interpolation to suppress the jagged artifacts. We also propose a novel and intuitive technique of edge-preserving interpolation, named edge model based interpolation, for preserving high frequency information. In motion image enlargement, we use an image sequence or a period of video as an input. Because of the inherent correlation of temporal and special information, we can get useful data through the process of motion compensation. In this thesis, we adopt the Bayesian MAP as the main mechanism. And try to employ the genetic algorithm (GA) to obtain the optimal image estimation results efficiently. Our proposed method, namely the super-resolution scheme, is to extract a frame of high-resolution from a set of low-resolution image sequence including the corresponding unprocessed frame.

中文摘要 I
英文摘要 II
致謝 III
目 錄 IV
圖 目 錄 VI
表 目 錄 IX

第一章 緒論 1
1.1 研究動機 1
1.2 文獻回顧 3
1.3 章節提要 5
第二章 靜態影像之放大 6
2.1 相關研究 6
2.1.1 傳統的 interpolation 6
2.1.2 Edge-preserving interpolation 14
2.2 EDGE的特性 21
2.2.1 因低通濾波所產生的邊緣過渡(transition of edge)效應 21
2.2.2 邊線方向的尋找 25
2.3 邊線保留及有方向性的內插 27
2.4 以邊線模型為基礎的影像內插 33
2.4.1 邊線模型(Edge Model) 33
2.4.2 精細的邊線預估(Sub-pixel Edge Estimation) 35
2.4.3 內插的技巧 38
第三章 動態影像之放大 41
3.1 問題描述 41
3.2 影像序列或VIDEO模型 42
3.3 動作補償 47
3.3.1 Block matching 47
3.3.2 多層式block matching(MLBM) 49
3.3.3 階層式block matching(Hierarchical) 51
3.4 BAYESIAN MAP演算法 52
3.5 最佳解 58
第四章 實驗結果及討論 67
4.1 系統執行環境 67
4.2 實驗結果 67
4.2.1 靜態影像放大實驗結果 67
4.2.2 動態影像序列放大實驗結果 75
4.3 討論 81
4.3.1 靜態影像放大之討論 81
4.3.2 動態影像放大之討論 82
第五章 結論及未來展望 84
5.1 結論 84
5.2 未來展望 85
APPENDIX I 86
APPENDIX II 87
參考文獻 89
作者簡歷 94

[1]R. R. Schultz and R. L. Stevenson, ”A Bayesian Approach to Image Expansion for Improved definition”, IEEE Trans. Image Processing, vol. 3, pp. 233-242, May 1994.

[2]R. R. Schultz and R. L. Stevenson, ”Extraction of High-resolution Frame from Video Sequence”, IEEE Trans. Image Processing, vol. 5, pp. 996-1011, June 1996.

[3]Andrew J. Patti, M. Ibrahim Sezan, A. Murat Tekalp, “Superresolution Video Reconstruction with Arbitrary Sampling Lattices and Nonzero Aperture Time”, IEEE Trans. on Image Processing, vol. 6, pp. 1064-1076, Aug. 1997.

[4]Pei-Hwa Chang, Jin-Jang Leou, Hsun-Chang Hsieh, “A Genetic Algorithm Approach to Image Sequence Interpolation”, Signal Processing: Image Commu., pp. 506-520, 2001

[5]R. G. Keys, “Cubic Convolution Interpolation for Digital Image Processing”, IEEE Trans. Acoustics, Speech, and Signal Processing, vol. ASSP-29, no. 6, pp. 1153-1160, Dec. 1981.

[6]H. S. Hou, H. C. Andrews, “Cubic Splines for Image Interpolation and Digital Filtering”, IEEE Trans. Acoustics, Speech, and Signal Processing, vol. ASSP-26, no. 6, pp. 508-517, Dec. 1978.

[7]W. K. Pratt, Digital Image Processing, 2nd ed. New York: Wiley, 1991.

[8]M. Unser, A. Aldroubi, M. Eden, “B-Spline Signal Processing: Part II-Efficient Design and Applications”, IEEE Trans. Signal Processing, vol. 41, pp. 834-848, Feb. 1993.

[9]M. Unser, A. Aldroubi, M. Eden, “Enlargement or Reduction of Digital Images with Minimum Loss of Information”, IEEE Trans. on Image Processing, vol. 4, pp. 247-258, Mar. 1995.

[10]M. Lehmann, Claudia Gonner, Klaus Spitzer, “Addendum: B-spline Interpolation in Medical Image Processing”, IEEE Trans. on Medical Image, vol. 20, pp. 660-665, July 2001.

[11]Erik H. W. Meijering, Karel J. Zuiderveld, Max A. Viergever, “Image Reconstruction by Convolution with Symmetrical Piecewise nth-Order Polynomial Kernels”, IEEE Trans. on Image Processing, vol. 8, pp. 192-201, Feb. 1999.

[12]Chulhee Lee, Murray Eden, M. Unser, “High-Quality Image Resizing Using Oblique Projection Operators”, IEEE Trans. on Image Processing, vol. 7, pp. 679-692, May 1998.

[13]Giovanni Ramponi, “Warped Distance for Space-Variant Linear Image Interpolation”, IEEE Trans. on Image Processing, vol. 8, pp. 629-639, May 1999.

[14]Kris Jensen, Dimitris Anastassiou, “Subpixel Edge Localization and the Interpolation of Still Image”, IEEE Trans. on Image Processing, vol. 8, pp. 285-295, Mar. 1995.

[15]Hou-Chun Ting, Hsueh-Ming Hang, “Edge Preserving Interpolation of Digital Image Using Fuzzy Inference”, JVCIR, vol. 8, pp.338-355, Dec. 1997.

[16]Jia-Guu Leu, “Image Enlargement Based on a Step Edge Model”, Pattern Recognition, pp. 2055-2073, 2000.

[17]Xin Li, M. T. Orchard, “New Edge-Directed Interpolation”, IEEE Trans. on Image Processing, vol. 10, pp. 1521-1527, Oct. 2001.

[18]R. Y. Tsai, T. S. Huang, ”Multiframe Image Restoration and Registration ”, Advances in Computer Vision and Image Processing(R. Y. Tsai, T. S. Huang, Eds.), vol. 1, pp. 317-339, JAI Press, London, 1984.

[19]H. Stark and P. Ozkan, “High-Resolution Image Recovery From Image-Plane Arrays, Using Convex Projections” J. Opt Soc. Am. A, vol. 6, no. 11, pp. 1715-1726, 1989.

[20]A. M. Tekalp, M. K. Ozkan, and M. I. Sezan, “High-resolution Image Reconstruction From Lower-resolution Image Sequences and Space-varying Image Restoration”, in Proceedings, IEEE International Conference on Acoustics, Speech, and Signal Processing, San Francisco, CA, 1992, pp. III169-172.

[21]A. J. Patti, M. I. Sezan, and A. M. Tekalp, ”High-resolution Standards Conversion of Low Resolution Video” , in Proceedings, IEEE International Conference on Acoustics, Speech, and Signal Processing, Detroit, MI, 1995, pp. 2197-2200.

[22]P.Cheesman, B. Kanefsky, R. Kraft, J. Stutz, and R. Hanson, ”Super-Resolved Surface Reconstruction from Multiple Images”, Tech. Rep. FIA-94-12, NASA Ames Research Center, Moffett Field, CA, 1994.

[23]M. Bierling and R. Thoma, “Motion Compensating Field Interpolation Using a Hierarchically Structured Displacement Estimator”, Signal Processing 11(4), pp. 387-404, 1986.

[24]C. H. Lee and L. H. Chen, “A Fast Motion Estimation Algorithm Based on the Block Sum Pyramid”, IEEE Trans. on Image Processing, vol. 6, no. 11, pp. 1587-1591, Nov. 1997.

[25]Jing-Yi Lu, Kuang-Shyr Wu, and Ja-Chen Lin, “Fast Full Search in Motion Estimation by Hierarchical Use of Minkowski’s Inequality”, Pattern Recognition, vol. 31, no. 7, pp. 945-952, 1998.

[26]Byung Cheol Song, Jong Beom Ra, “A fast Multi-resolution Block Matching Algorithm for Motion Estimation”, Signal Processing: Image Commu. 15, pp. 799-810, 2000.

[27]B. K. P. Horn and B.G. Shunck, “Determining Optical Flow”, Artificial Intelligence 17, pp. 185-203, 1981

[28]M. S. Bazaraa, H. D. Sherali, and C. M. Shetty, “Nonlinear Programming: Theory and Algorithm”, 2nd Ed. Wiley.

[29]Zbigniew Michalewicz, “Genetic Algorithms + Data Structures = Evolution Programs”, 3rd Ed., Springer.

[30]M. Gen, R. Cheng, “Genetic Algorithms and Engineering Design”, Wiley, New York, 1997.

[31]K. P. Hong et al., “An Edge-preserving Image Interpolation System for a Digital camcorder”, IEEE Trans. Consumer Electron. 42(3), pp. 279-284, 1996.

[32]王隆仁, ”A fast Cubic-Spline Interpolation and its Application”, 博士論文, 國立中山大學CSIE, Mar. 2001.

[33]M. Unser, ”Spline: A Perfect Fit for Signal and Image Processing”, IEEE Signal Processing Magazine, pp. 22-38, Nov. 1999.

[34]Nathalie Plaziac, “Image Interpolation Using Neural Networks”, IEEE Trans. On Image Processing, vol. 8, no. 11, Nov. 1999.

[35]Hayit Greenspan, Charles H. Anderson, and Sofia Akber, “Image Enhancement by Nonlinear Extrapolation in Frequency Space”, IEEE Trans. On Image Processing, vol. 9, no. 6, June 2000.

[36]Murat Belge, Misha E. Kilmer, and Eric L. Miller, “Wavelet Domain Image Restoration with Adaptive Edge-Preserving Regularization”, IEEE Trans. On Image Processing, vol. 9, no. 4, Apr. 2000.

[37]Qinghuai Gao, Fang-Fang Yin, “Two-Dimensional Direction-Based Interpolation with Local Centered Moments”, Graphical Models and Image Processing 61, pp. 323-339, 1999

[38]G. H. Golub, “Matrix Decomposition and Statistical Calculations”, in Statistical Computation, R. C. Milton and J. A. Nelder, Eds. New York: Academic, 1969, pp. 365-397

[39]F. Yeung, S. F. Levinson, and K. J. Parker, “Mutilevel and Motion Model-Based Ultrasonic Speckle Tracking Algorithm”, Ultrasound in Med. & Biol., vol. 24, no. 3, pp. 427-411, 1998

[40]C. H. Lee and L. H. CHen, “A Fast Motion Estimation Algorithm Based on the Block Sum Pyramid”, IEEE Trans. On Image Processing, vol. 6, no. 11, Nov 1997

[41]P. Moulin, R. Krishnamurthy, and J. W. Woods, “Multiscale Modeling and Estimation of Motion Fields for Video Coding”, IEEE Trans. On Image Processing, vol. 6, no. 12, Dec. 1997

[42]J. Malo, F. J. Ferri, J. Albert, and J. M. Artigas, “Splitting Criterion for Hierarchical Motion Estimation Based on Perceptual Coding”, Electronics Letters, vol. 34, no. 6, 1998

[43]B. C. Song, and J. B. Ra, “A Fast Multi-resolution Block Matching Algorithm for Motion Estimation”, Signal Processing: Image Commu., pp.799-810, 2000

[44]J. Allebach, P. W. Wong, “Edge-Directed Interpolation”, Proceedings of the ICIP 96, vol. III, IEEE Press, Laausanne CH, 1996, pp. 707-710

[45]Levenberg, K., "A Method for the Solution of Certain Problems in Last Squares," Quart. Appl. Math. Vol. 2, pp. 164-168, 1944.

[46]Marquardt, D., "An Algorithm for Least Squares Estimation of Nonlinear Parameters," SIAM J. Appl. Math. Vol. 11, pp. 431-441, 1963.

[47]P. J. M. van Laarhoven and E. H. L. Aarts. Simulated Anneling: Theory and Applications. D. Reidel Publishing Company, 1987.

[48]W. H.; Flannery, B. P.; Teukolsky, S. A.; and Vetterling, W. T. "Downhill Simplex Method in Multidimensions", Cambridge, England: Cambridge University Press, pp. 402-406