臺灣博碩士論文加值系統

本文提出一種可應用於公共廣場或大型倉儲之多攝影機之即時視訊拼接(Stitching)方法，其主要功能是將監控攝影機所拍攝之畫面進行拼接動作，使得監控人員能夠觀察到視野較廣的畫面，且與傳統採用多個子畫面監控相較下，單一寬廣監控畫面會更加直觀清楚，也減少監控人員觀看時容易發生空間上的視覺混亂及畫面中的視覺死角。此外，本文提出之方法處理完成之結果影像，也能夠給予相關電腦視覺處理的移動物偵測系統作為前級輸入，使其畫面視野增廣以獲得較多影像資訊且偵測範圍也更為廣泛。本文主要包含五個部分所完成：(1)影像前處理：因監控攝影機逐漸朝向高解析度發展，使得畫面中影像資訊增加反而導致複雜度較高的特徵點運算耗費過多時間，故使用雙線性插值法(Bilinear Interpolation)進行影像縮減取樣；(2)影像特徵點處理：使用SIFT法進行特徵點(Feature Point)搜尋，並透過RANSAC法剃除異常(Outliers)的特徵點，接著以DBSCAN法求得每一特徵點群聚(Cluster)中的核心點(Core Point)給予後續分析使用；(3)找尋最佳透視變換(Perspective Transform)矩陣：計算出影像的自適應性(Adaptive)場景分群線且依核心點分佈位置分成兩群，並計算此兩群核心點數量之比值，依據閥值判斷篩選出能夠得到較佳透視變換矩陣的特徵點集合並存入候選特徵點集合暫存器中，接著在計算候選特徵點集合中各集合之特徵點數量最大值，在計算其特徵點集合的單應性矩陣(Homography Matrix)並視為最佳透視變換矩陣；(4)場景校正及補償: 取得上一階段所得到的最佳透視變換矩陣，對目標畫面進行透視變換校正動作；(5)視訊拼接(Video Stitching):依特徵點分佈將影像劃分成重疊區域與非重疊區域，使用多頻帶融合(Multi-Band Blending)法對重疊區域進行拼接，而使用線性融合(Linear Blending)法對非重疊區域進行拼接，最後再將拼接後的兩區域進行疊加以產生最後拼接結果。

This paper presents a multi-camera stitching method that can be applied to public plazas or large warehouses. The method operates by video Stitching for Surveillance Frames, So that monitoring personnel can monitor the screen from the monitoring center to see a larger view of the screen, And a single screen monitoring is simpler than the observation from multiple screen, As well as to reduce the monitoring personnel to watch when the space is prone to chaos can also reduce the production of dead ends in the screen. Through the system after processing the results of the video, but also to the relevant computer visual processing of the detection system as a pre-image input, As the screen field of view can be increased to obtain more information and more extensive detection range, it can get better detection results; In this paper, the method has five modules: (1) Image preprocessing: The increase in image information in the picture makes it more time consuming for feature points with higher complexity, So the use of bilinear interpolation method for image reduction sampling. (2) Feature point detection: Feature search by SIFT + RANSAC Then use DBSCAN to filter feature points, (3) Find the best perspective transformation matrix: Select the two groups of the number of feature points in the ratio of the closest parameters for the best perspective transformation matrix. (4) Scene correction and compensation: Obtain the best matrix of storage and then carry on the perspective transformation to carry on the correction action to the target picture.(5)Video Stitching : We divide the image into overlapping and non-overlapping areas, And the overlapping area is subjected to multi-band blending splicing instead of overlapping area. Then, the two regions are merged to obtain the image result after splicing is completed.

摘要 III
ABSTRACT V
致謝 VII
目錄 VIII
圖目錄 XI
表目錄 XIV
第一章、 緒論 1
1.1 研究動機 1
1.2 系統架構與流程 3
1.3 論文大綱 5
第二章、 相關習知技術與背景 6
2.1 特徵點偵測 6
2.1.1 尺度不變特徵轉換SIFT(Scale-invariant feature transform) 6
2.1.2 加速穩健特徵SURF(Speeded Up Robust Features) 7
2.1.3 ORB特徵點偵測(Oriented FAST and rotated BRIEF) 8
2.1.4 特徵點偵測總結 9
2.2 影像校正 (Image Distortion Correction) 9
2.2.1 仿射變換 (Affine Transform) 9
2.2.2 透視變換(Perspective Transform) 10
2.2.3 單應性矩陣(Homography Matrix) 11
2.2.4 影像校正總結 12
2.3 圖像縮減 12
2.3.1 最近相鄰內插法(Nearest Neighbor Interpolation) 12
2.3.2 雙線性內插法(Bilinear Interpolation) 13
2.3.3 雙立方內插法 (Bicubic Interpolation) 15
2.4 分群演算法(Clustering Analysis) 15
2.4.1 DBSCAN分群演算法 15
2.4.2 K-Means分群演算法 17
2.4.3 分群總結 17
2.5 拼接演算法 18
2.5.1 線性融合演算法(Linear Blending) 18
2.5.2 多頻帶融合演算法(Multi-band Blending) 18
2.5.3 拼接方法總評 19
第三章、 相關方法探討 20
3.1 圖像拼接相關方法 20
3.1.1 Automatic Panoramic Image Stitching using Invariant Features 20
3.1.2 Smoothly Varying Affine Stitching 26
3.1.3 As-Projective-As-Possible Image Stitching with Moving DLT 30
3.1.4 Seam-Driven Image Stitching 34
3.2 視訊拼接相關方法 37
3.2.1 Panoramic Video Stitching in Multi-Camera Surveillance System 37
3.2.2 An Effective Video Stitching Method 39
第四章、 本系統與方法 41
4.1 圖像拼接與視訊拼接差異 44
4.2 影像前處理 45
4.3 影像特徵點處理 47
4.4 找尋最佳透視變換矩陣 54
4.4.1 自適應場景分群線 54
4.4.2 候選矩陣篩選 56
4.4.3 挑選最佳透視變換矩陣 58
4.5 場景校正及補償 59
4.6 視訊拼接 61
4.6.1 重疊區域 62
4.6.2 非重疊區域 65
第五章、 實驗結果 69
5.1 架設環境與測試影片 69
5.2 輸出結果畫面 75
5.3 實驗結果與分析 85
5.3.1 主觀評估 85
5.3.2 客觀評估 95
第六章、 結論與未來方向 102
6.1 結論 102
6.2 問題與討論 103
6.3 未來方向 103
參考文獻 105

[1]Lowe, David G. “Distinctive Image Features from Scale-Invariant Keypoints”, International journal of computer vision, Vol.60, No. 2, 2004, pp.91-110.
[2]Bay, Herbert, Tinne Tuytelaars, and Luc Van Gool. “Surf: Speeded up Robust Features. ” Computer vision–ECCV, Graz, Austria, pp.404-417, May 2006.
[3]Rublee, E., Rabaud, V., Konolige, K., and Bradski, G. “ORB: An Efficient Alternative to SIFT or SURF”, Computer Vision (ICCV), 2011 IEEE International Conference, Barcelona, Spain, Nov 2011.
[4]M. Ester, H. P. Kriegel, J. Sander and X. Xu, “A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise”, Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD-96), pp. 226–231, 1996.
[5]J. B. MacQueen, “Some Methods for Classification and Analysis of Multivariate Observations”, Proceedings of 5th Berkeley Symposium on Mathematical Statistics and Probability, Volume 1 (University of California Press), pp. 281–297, 1967.
[6]Brown, Matthew, and David G. Lowe. “Automatic panoramic image stitching using invariant features”, International journal of computer vision, Vol.74, No. 1, 2007, pp.59-73.
[7]Lin, Wen-Yan, et al. “Smoothly varying affine stitching”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, American, pp.345-352, June 2011.
[8]Zaragoza, Julio, et al. “As-projective-as-possible image stitching with moving DLT”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, Oregon, pp.2339-2346, Jun 2013.
[9]Junhong Gao, Yu Li, Tat-Jun Chin, and Michael S. Brown “Seam-Driven Image Stitching”, Proceedings of The Eurographics Association ,Short Papers Imaging, Girona, Spain, pp.45-48, May 2013.
[10]Xu, Wei, and Jane Mulligan. “Performance evaluation of color correction approaches for automatic multi-view image and video stitching”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, American, pp.263-270, Jun 2010.
[11]Zhang, Fan, and Feng Liu. “Parallax-tolerant image stitching”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, American, pp.3262-3269, Jun 2014.
[12]Szeliski, Richard, and Heung-Yeung Shum. “Creating full view panoramic image mosaics and environment maps”, Proceedings of the 24th annual conference on Computer graphics and interactive techniques. ACM Press/Addison-Wesley Publishing Co, pp.251-258, 1997.
[13]Rav-Acha, Alex, Giora Engel, and Shmuel Peleg. “Minimal aspect distortion (MAD) mosaicing of long scenes”, International Journal of Computer Vision, Vol.78, No. 2-3, 2008, pp.187-206.
[14]Traka, M., and Georgios Tziritas. “Panoramic view construction”, Signal Processing: Image Communication, Vol.18, No. 6, 2003, pp.465-481.
[15]Yu, Jingyi, and Leonard McMillan. “A framework for multiperspective rendering”, Rendering Techniques, 2004, pp.61-68.
[16]Wang, Yu-Shuen, et al. “Optimized scale-and-stretch for image resizing”, ACM Transactions on Graphics (TOG), Vol. 27, No. 5, 2008, pp.118.
[17]He, Bin, Gang Zhao, and Qifang Liu. “Panoramic video stitching in multi-camera surveillance system”, Image and Vision Computing New Zealand (IVCNZ), 25th International Conference of IEEE, Queenstown, New Zealand, pp.1-6, Nov 2010.
[18]Chen, Linqiang, Xiaoqiang Wang, and Xu Liang. “An effective video stitching method”, Computer Design and Applications (ICCDA), 2010 International Conference on. Vol. 1. IEEE, Qinhuangdao, China, pp.297-301, Jun 2010.
[19]Jing Li, Wei Xu, Jianguo Zhang, Maojun Zhang, Zhengming Wang and Xuelong Li. “Efficient Video Stitching Based on Fast Structure Deformation” IEEE Transactions on Cybernetics, Vol 45, No. 12, Oct. 2015, pp. 2707-2719.
[20]Motaz El-Saban, Mostafa Izz, Ayman Kaheel and Mahmoud Refaat. “Improved optimal seam selection blending for fast video stitching of videos captured from freely moving devices”, Proceedings of The 18th IEEE International Conference on Image Processing, Brussels, Belgium, pp.1481-1484, Sep 2011.
[21]Šegvić, S., Ševrović, M., Kos, G., Stanisavljević, V. and Dadić, I. “Preliminary experiments in multi-view video stitching”, Proceedings of The 34th International Convention MIPRO, Opatija, Croatia, pp.892-896, May 2011.
[22]Doutre, Colin, and Panos Nasiopoulos. “Fast vignetting correction and color matching for panoramic image stitching”, Proceedings of The 16th IEEE International Conference on Image Processing, Cairo, Egypt, pp.709-712, Nov 2009.
[23]El-Saban, Motaz, Mostafa Izz, and Ayman Kaheel. “Fast stitching of videos captured from freely moving devices by exploiting temporal redundancy.”, Proceedings of The 17th IEEE International Conference on Image Processing, Hong Kong, Hong Kong, pp.1193-1196, Sep 2010.