臺灣博碩士論文加值系統

在現今生活上，可能會需要利用高解析度影像來提供更多有效、更細膩的資訊，從以往傳統內插法的方式，直到現今數位影像處理的技術越來越進步，許多學者提出關於這領域的研究，透過超解析度(Super Resolution)方法來增加影像的高頻資訊(紋理或邊緣)，以提升影像品質。應用層面來說更為廣泛，如道路監控畫面，監控影像中可能受到光線、角度、距離遠近等外在因素所導致肉眼無法清楚的辨識車牌或人臉等。在以往超級解析度研究裡，傳統內插方法裡由於缺乏高頻資訊，後來有學者提出Learning-based中Example-based的概念，目的要增強影像中高頻資訊，此外近年來在機器學習中較熱門的深度學習(Deep Learning)，對於影像檢索、人臉辨識及超級解析度等應用中，研究上皆有顯著的改善。然而超級解析度在以往多張影像的研究中，是將整張影像或是多個候選影像透過已學習的卷積神經網路重建影像，效果好但非常耗時。本研究透過三步搜尋法，找到變動及非變動區塊，基於深度學習將變動區塊做超解析，經由前處理後的區塊重建出下一張高解析度Frame。實驗中藉由Frame與Frame之間搜尋到的變動區塊，輸入到神經網路，重建出高解析度影像，以利於降低整體運算量。

Nowadays, people might need super resolution to have more effective and clear information. The technology of image processing becomes better and better, and there are more and more people present their research in this field. Super resolution algorithm enhances high frequent information (texture or edges) to improve the image quality. We can do more things with super resolution, such as road surveillance system. The view might be influence by illumination, angle, distance, and other conditions, so these might not be good for us to recognize the number of license plate or human face. Interpolation is a great method for super resolution, but this method does not own high frequent information. Therefore, researcher present the concept of Example-based in Learning-based to solve this problem. Besides, in recent years, deep learning has great result and becomes faster than before. Deep learning has not only significant result but also great speed. Although the speed of deep learning is faster than before, it still needs some time to rebuild. The time might be acceptable for single image. But what if we have to enhance video, it will take a lot of time to rebuild. Therefore, our research is able to solve this problem by Three Steps Search algorithm. We present a faster super resolution for video based on deep learning. We find the different blocks between frame and frame. Add these blocks into neural net and rebuild high resolution image to lower the total compute time.

摘要.....................................................I
ABSTRACT.................................................II
誌 謝..................................................III
目錄.....................................................IV
表目錄...................................................VII
圖目錄...................................................VIII
第一章 導論.............................................1
1.1 研究動機............................................ 1
1.2 研究架構............................................ 1
第二章 相關文獻.......................................... 2
2.1基於單張影像之超解析度演算法.... ........................2
2.1.1 多項式內插法 (Polynomial interpolation).... ........2
2.1.2 沿著邊緣方向內插 (Edge-directed interpolation)......4
2.1.3 Learning-based Method............................ 5
2.1.4 卷積神經網路重建高解析度單張影像..................... 6
2.1.5 保留幾何特性之超級解析度演算......... ................8
2.2 基於多張影像之超級解析度演算法......... ................12
2.2.1 動態超級解析度..................................... 12
2.2.2 非對稱內插法(Non-Uniform Interpolation)............ 14
2.2.3 反投影迭代法(Iterative Back Projection)............ 14
2.2.4 凸集合投影法 (Projection onto Convex Sets)........ 16
2.2.5 MAP (Bayesian Maximum A Posteriori Probability).. 16
2.2.6 Deep Draft-Ensemble Learning..................... 16
2.3 YCbCr 色彩空間..................................... 18
第三章 系統架構........................................ 20
3.1 輸入低解析度影片..................................... 20
3.2 移動區塊偵測......................................... 20
3.3 基於深度學習之區塊超解析.............................. 21
3.4 高解析度影像重建..................................... 21
第四章 系統流程........................................ 22
4.1 SRCNN架構介紹....................................... 22
4.2 變動區塊偵測......................................... 24
4.3 基於深度學習之區塊超解析.............................. 25
4.3.1 變動區塊串接....................................... 25
4.3.2 變動區塊延伸....................................... 26
4.4 高解析度影帶重建..................................... 28
4.4.1 輸入神經網路....................................... 28
4.4.2 區塊重疊.......................................... 28
4.4.3 高解析度重建....................................... 29
第五章 實驗結果........................................ 30
5.1 實驗環境............................................ 30
5.2 實驗Video........................................... 30
5.3 評估方法............................................ 31
5.4 實驗數據............................................ 32
第六章 結論............................................ 42
第七章 參考............................................ 43

[1]X. Li and M.T. Orchard, “New edge-directed interpolation”, IEEE Transactions on Image Processing, Vol. 10, No. 10, pp. 1521-1527, 2001.
[2]M. Bevilacqua, A. Roumy, C. Guillemot, and M.L.A. Morel, “Low-complexity single-image super-resolution based on nonnegative neighbor embedding”, British Machine Vision Conference, 2012.
[3]J. Sun, Z. Xu, and H. Y. Shum, “Image super-resolution using gradient profile prior,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8, 2008.
[4]D. Glasner, S. Bagon, and M. Irani, “Super-resolution from a single image”, IEEE International Conference on Computer Vision, pp. 349-356, 2009.
[5]W.T. Freeman, T.R. Jones, and E.C. Pasztor, “Example-based super-resolution”, IEEE Computer Graphics and Applications, Vol. 22, No. 2, pp. 56-65, 2002.
[6]J. Yang, J. Wright, T.S. Huang, and Y. Ma, “Image super-resolution via sparse representation, ”IEEE Transactions on Image Processing, Vol. 19, pp. 2861-2873, 2010.
[7]R. Zeyde, M. Elad, and M. Protter, “On single image scale-up using sparse-representations”, International Conference on Curves and Surfaces, Vol. 6920, pp. 711-730, 2012.
[8]D. Capel and A. Zisserman, “Super-resolution from multiple views using learnt image models”, IEEE Conference on Computer Vision and Pattern Recognition, Vol. 2, pp. II-627-II-634, 2001.
[9]C.Y. Yang and M.H. Yang, “Fast direct super-resolution by simple functions”, IEEE International Conference on Computer Vision, pp. 561-568, 2013.
[10]C. Dong, C.C. Loy, K. He, and X. Tang, “Learning a deep convolutional network for image super-resolution”, European Conference on Computer Vision, pp. 184-199, 2014.
[11]R. Liao, X. Tao, R. Li, Z. Ma, and J. Jia, “Video super-resolution via deep draft-ensemble learning”, IEEE International Conference on Computer Vision, pp. 531-539, 2015.
[12]J.B. Huang, A. Singh, and N. Ahuja, “Single image super-resolution from transformed self-exemplars”, IEEE Computer Vision and Pattern Recognition, pp. 5197-5206, 2015.
[13]J.B. Huang, S.B. Kang, N. Ahuja, and J. Kopf, “Image completion using planar structure guidance”, ACM Transactions on Graphics, 2014.
[14]R.Y. Tsai and T.S. Huang, “Multiframe image restoration and registration”, Computer Vision and Image Processing, Vol. 1, pp. 317-339, 1984.
[15]Katsaggelos, K. Aggelos, Rafael Molina, and Javier Mateos, “Super resolution of images and video”, Synthesis Lectures on Image, Video, and Multimedia Processing, pp. 1-134, 2007.
[16]S.C. Park, M.K. Park, and M.G. Kang, “Super-resolution image reconstruction: a technical overview”, IEEE Signal Processing Magazine, Vol. 20, No. 3, pp. 21-36, 2003.
[17]M. Irani and S. Peleg, “Improving resolution by image registration”, Graphical Models and Image Processing, Vol. 53, No. 3, pp. 231-239, 1991.
[18]M. Irani and S. Peleg, “Motion analysis for image enhancement: resolution, occlusion and transparency”, Journal of Visual Communications and Image Representation, Vol. 4, pp. 324-335, 1993.
[19]J. Patti, M.I. Sezan, and A.M. Tekalp, “Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time”, IEEE Transactions on Image Processing, Vol. 6, No. 8, pp. 1064-1076, 1997.
[20]R.R. Schultz and R.L. Stevenson, “Extraction of high-resolution frame from video sequence”, IEEE Transactions on Image Processing, Vol. 5, pp. 996-1011, 1996.
[21]R.R. Schultz and R.L. Stevenson, “A Bayesian approach to image expansion for improved definition”, IEEE Transactions on Image Processing, Vol. 3, pp.233-242, 1994.
[22]H. Shen, P. Li, L. Zhang, and Y. Zhao, “A MAP algorithm to super-resolution image reconstruction”, International Conference on Image and Graphics, pp.544-547, 2004.
[23]T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, “High accuracy optical flow estimation based on a theory for warping”, European Conference on Computer Vision, 2004.
[24]C. Liu, “Beyond pixels: exploring new representations and applications for motion analysis”, Massachusetts Institute of Technology, 2009.
[25]L. Xu, J. Jia, and Y. Matsushita, “Motion detail preserving optical flow estimation”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 34, pp. 1744-1757, 2012.
[26]D. Eigen, D. Krishnan, and R. Fergus, “Restoring an image taken through a window covered with dirt or rain”, IEEE International Conference on Computer Vision, 2013.
[27]J. Yangqing, S. Evan, D. Jeff, K. Sergey, L. Jonathan, G. Ross, G. Sergio, and D. Trevor, “Caffe: Convolutional architecture for fast feature embedding”, IEEE International Congress of Mathematicians, pp. 675-678, 2014.