臺灣博碩士論文加值系統

本論文提出一個可應用於嵌入式環境下之低複雜度影像暨影片接合演算法。藉由此演算法，使用者可將兩張具有重疊區塊之影像/影片，合併為一大解析度、平滑、無縫之高品質全景影像/影片。在影像/影片接合技術之現有文獻方面，可依照通用之流程分成數個步驟，包含尋找來源影像之相對關係、影像轉換後之失真修補等等。在影像對齊階段，將找出來源影像之間的線性關係，其實作方法亦是相當多元，包括使用相機參數、邊緣修正、特徵點配對等等，得到線性關係後即以座標轉換的方式接合出初步影像。影像修補階段是用來修補線性轉換所產生的各種影像失真，包含黑洞、明顯接合線，以及在影片應用中，因物件移動造成的斷層現象。現今已發表之相關文獻中，大多是以單張接合影像作為基礎，所耗時間較長，無法適用於即時之影片接合應用。本論文針對這些問題，提出一影片接合演算法，將一般相機攝得之非廣角影片，以低成本之方式接合成一高品質之全景影片，並針對在嵌入式平台的運作上，提出了數種簡化運算，以及減少執行時間之多執行緒實作方法。本論文可應用於市面上多種不同產品之多視角應用，包括行車紀錄器、倒車輔助系統、即時監控系統等等。

This thesis presents a low-complexity panoramic video stitching algorithm. With the proposed algorithm, users can stitch two videos into a high-resolution, smooth, seamless panoramic video. In related works, the processing can be divided into several stages, including image alignment, image repairing, and so on. In image alignment stage, the linear relationship of source videos can be found by various methods, such as using camera parameter, edge-correlation, feature-based matching, etc. After the initial stitched frame is obtained, the distortion occurred in the initial stitched frame would be eliminated in Image Repairing stage. The problems include the holes, clear seam, and artifacts caused by moving objects in video stitching. Most of the relative literatures are focusing on the single image stitching and consume a lot of time so that they are not suitable to be applied to real-time video stitching. To solve these problems, the thesis proposes an algorithm that can stitch several videos captured from normal cameras into one panoramic video with high quality. For realization on an embedded system, this thesis implements several methods such as the complexity reduction and multithreading. This thesis can be applied to various applications and products with multiple views, such as tachograph, car rearview, surveillance system, and so on.

Chapter 1 Introduction 1
1.1 Background: What is Panoramic Image/Video Stitching? 1
1.2 Motivation 3
1.3 Organization of Thesis 4
Chapter 2 Related Works 5
2.1 Overview 5
2.2 Image Alignment 5
2.2.1 Multi-scale Edge Correlation Method [2] 6
2.2.2 FFT-based Image Registration [3] 7
2.2.3 Block Matching and M-estimation Method [4] 7
2.2.4 Scale-Invariant Feature Transform [6] 8
2.2.5 Edge-based Stitching Algorithm [7] 13
2.3 Image Repairing and Blending 15
2.3.1 A Two-phase Image Blending Scheme [8] 15
2.3.2 A Fast-to-complexity Blending Algorithm [9] 17
2.3.3 Fade In-out Blending Image Fusion Approach [11] 18
2.3.4 Foreground Detection and Selection Method [13] 19
Chapter 3 Proposed Algorithm 22
3.1 Overview 22
3.2 Block-based Image Alignment Estimation 23
3.3 Image Projection and Blending 26
3.4 Image Repair and Blending 27
3.4.1 Texture-based Repairing 28
3.4.2 Dynamic Optimal Seam Adjustment 30
3.4.3 Brightness Adjustment 32
3.5 Video Stitching with Multiple Cameras 34
3.6 Vertical and Four-channel Video Stitching 35
Chapter 4 Realization on an Embedded System 38
4.1 Simulation Environment 38
4.2 Optimization of the Proposed Algorithm 40
4.2.1 Dynamic Down-sampling 40
4.2.2 Multithreading of Video Stitching Algorithm 42
Chapter 5 Performance Evaluation and Demonstration 44
5.1 Performance Evaluation 44
5.2 Demonstration 46
Chapter 6 Conclusion and Applications 50
6.1 Conclusion 50
6.2 Applications 51
6.2.1 Panoramic Tachograph 51
6.2.2 Panoramic Surveillance System 52
Reference 53

[1]Yingen Xiong; Pulli, K.; , "Sequential image stitching for mobile panoramas," Information, Communications and Signal Processing, 2009. ICICS 2009. 7th International Conference on , vol., no., pp.1-5, 8-10 Dec. 2009
[2]Chiou-Ting Hsu; Tzu-Hung Cheng; Beuker, R.A.; Jyh-Kuen Horng; , "Feature-based video mosaic," Image Processing, 2000. Proceedings. 2000 International Conference on , vol.2, no., pp.887-890 vol.2, 10-13 Sept. 2000
[3]Reddy, B.S.; Chatterji, B.N.; , "An FFT-based technique for translation, rotation, and scale-invariant image registration," Image Processing, IEEE Transactions on , vol.5, no.8, pp.1266-1271, Aug 1996
[4]Li Zhong-xin; Xu Wan-he; Zhang Yue; Mao Yao-bin; Wang Zhi-quan; , "Video Mosaic with Block Matching and M-Estimation," Computer Science and Information Engineering, 2009 WRI World Congress on , vol.6, no., pp.632-635, March 31 2009-April 2 2009
[5]Szeliski, R.; , "Video mosaics for virtual environments," Computer Graphics and Applications, IEEE , vol.16, no.2, pp.22-30, Mar 1996
[6]David G. Lowe, "Distinctive image features from scale-invariant key points," International Journal of Computer Vision, vol.60, no.2, pp.91-110, 2004
[7]Jun-Wei Hsieh; , "Fast stitching algorithm for moving object detection and mosaic construction," Multimedia and Expo, 2003. ICME '03. Proceedings. 2003 International Conference on , vol.1, no., pp. I- 85-8 vol.1, 6-9 July 2003
[8]Wu-Chih Hu; Chin-Hao Chang; Ya-Fang Liang; Chen-Tsung Wu; , "Creating Wide Field-of-View Video using Multicamera," Innovative Computing, Information and Control, 2007. ICICIC '07. Second International Conference on , vol., no., pp.239, 5-7 Sept. 2007
[9]Pulli, K.; Tico, M.; Yingen Xiong; Xianglin Wang; Chia-Kai Liang; , "Panoramic imaging system for camera phones," Consumer Electronics (ICCE), 2010 Digest of Technical Papers International Conference on , vol., no., pp.45-46, 9-13 Jan. 2010
[10]P. Perez, M. Gangnet, and A. Blake, "Poisson image editing," ACM Trans. Graph., vol. 22, no. 3, pp. 313– 318, 2003.
[11]Wang, Yong; Xu, Lu-ping; , "A Global Optimized Registration Algorithm for Image Stitching," Image and Signal Processing, 2008. CISP '08. Congress on , vol.3, no., pp.525-529, 27-30 May 2008
[12]Mann, S.; Picard, R.W.; , "Video orbits of the projective group a simple approach to featureless estimation of parameters," Image Processing, IEEE Transactions on , vol.6, no.9, pp.1281-1295, Sep 1997
[13]Hongming Zhang; Wei Zeng; Xin Chen; , "Foreground Based Borderline Adjusting for Real Time Multi-camera Video Stitching," Image and Graphics, 2009. ICIG '09. Fifth International Conference on , vol., no., pp.355-360, 20-23 Sept. 2009
[14]M. Brown and D. G. Lowe, "Automatic panoramic image stitching using invariant features," International Journal of Computer Vision, vol.74, no.1, pp.59-73, 2007.
[15]Zhen Hua; Yewei Li; Jinjiang Li; , "Image stitch algorithm based on SIFT and MVSC," Fuzzy Systems and Knowledge Discovery (FSKD), 2010 Seventh International Conference on , vol.6, no., pp.2628-2632, 10-12 Aug. 2010.
[16]Hailin Jin; , "A three-point minimal solution for panoramic stitching with lens distortion," Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on , vol., no., pp.1-8, 23-28 June 2008.
[17]Li Zhong-xin; Xu Wan-he; Wu Zhi-lin; Zhang Yue; , "Video Mosaic with Motion Estimation," Information and Computing Science, 2009. ICIC '09. Second International Conference on , vol.2, no., pp.106-109, 21-22 May 2009.
[18]Jing Xing; Zhenjiang Miao; , "An Improved Algorithm on Image Stitching Based on SIFT features," Innovative Computing, Information and Control, 2007. ICICIC '07. Second International Conference on , vol., no., pp.453, 5-7 Sept. 2007.
[19]Yu-jing Zhao; Zhe-qi Lu; Yang-ke Liu; , "Video image stitching based on moving object detection and motion prediction compensation," Image and Signal Processing (CISP), 2010 3rd International Congress on , vol.6, no., pp.2885-2889, 16-18 Oct. 2010.
[20]Yang Ping; Mao Zheng; Zhang Zhi; Qi Feng; , "Image Mosaics Solution for Video Conference," Multimedia Technology (ICMT), 2010 International Conference on , vol., no., pp.1-4, 29-31 Oct. 2010.
[21]Lin-na Li; Nan Geng; , "Algorithm for Sequence Image Automatic Mosaic Based on SIFT Feature," Information Engineering (ICIE), 2010 WASE International Conference on , vol.1, no., pp.203-206, 14-15 Aug. 2010.
[22]Xiong, Y.; Pulli, K.; , "Fast panorama stitching for high-quality panoramic images on mobile phones," Consumer Electronics, IEEE Transactions on , vol.56, no.2, pp.298-306, May 2010.
[23]Yuan Hong-Zhao; Wang Bang-Ping; Zhang Jun; Li Hui; , "A Novel Method for Geometric Correction of Multi-cameras in Panoramic Video System," Measuring Technology and Mechatronics Automation (ICMTMA), 2010 International Conference on , vol.1, no., pp.248-251, 13-14 March 2010.
[24]Pourazad, M.T.; Nasiopoulos, P.; Ward, R.K.; , "An efficient low random-access delay panorama-based multiview video coding scheme," Image Processing (ICIP), 2009 16th IEEE International Conference on , vol., no., pp.2945-2948, 7-10 Nov. 2009.
[25]Jong-Seung Park; Bum-Jong Lee; , "View Synthesis from Two Unrectified Images," Digital Media and its Application in Museum & Heritages, Second Workshop on , vol., no., pp.91-96, 10-12 Dec. 2007.