臺灣博碩士論文加值系統

本論文研發基於擷取特徵點進行匹配的視訊同步系統，系統架構包含了數個步驟: 視訊前處理、key-frame selection、特徵點擷取匹配和frame index differences之計算。因為蒐集的視訊可能會有不同的frame rates和frame sizes，所以先透過前處理將frame rates與frame sizes進行正規化。選取出關鍵訊框後，基於特徵點匹配和RANSAC隨機抽樣一致演算法，可以匹配不同視訊之相似關鍵畫面。透過分析匹配的訊框，可以得到視訊之間的frame index differences，進而達成視訊地同步。本論文中，我們拍攝了一些測試視訊來進行本系統之性能評估。實驗結果證明了可以有效地偵測key-frames，透過特徵點擷取匹配將訊框進行配對，也達到不錯的正確匹配率，由客觀與主觀評估的分數也都證明了本系統在視訊同步方面皆能達到不錯的性能。

In this thesis, a video synchronization system was developed based on feature point matching. The proposed system contains several parts: pre-processing, key frame selection, feature point matching, and frame index difference computation. Since videos may have different frame rates and frame sizes, the pre-processing procedure is used to normalize the frame rate and frame size of videos. After key frame extraction, matching similar key frames from difference videos is conducted based on feature point matching and RANSAC. The frame index differences among videos can be obtained by analyzing these matched key frames and then synchronizing videos can be achieved.To evaluate the performance of the proposed system, some videos are captured for testing. Experimental results show that key frames can be selected effectively. Frame matching can be well achieved based on feature point matching and RANSAC. The higher score of subjective testing demonstrates that the proposed system can synchronize videos well.

摘要 i
ABSTRACT ii
誌謝 iii
目錄 iv
表目錄 vi
圖目錄 vii
第一章緒論 1
1.1 研究背景 1
1.2 研究動機與目標 1
1.3 論文可能遭遇困難 2
1.4 章節概述 3
1.5 實驗平台規格 3
第二章 背景簡介與文獻回顧 4
2.1 視訊閃光偵測研究文獻 4
2.2 尺度不變特徵轉換(SIFT)研究文獻 4
2.3 加速穩健特徵(SURF)研究文獻 7
2.4 視訊同步(VIDEO SYNCHRONIZATION)研究文獻 10
2.5 結論 12
第三章 整體系統架構介紹 13
第四章 前處裡與閃光偵測 16
4.1 視訊前處理(PRE-PROCESSING) 16
4.1.1視訊來源之FPS與訊框大小正規化 16
4.2 閃光偵測(FLASH DETECTION) 17
4.2.1 各環境之閃光自動化偵測 17
4.3 本章結論 19
第五章 特徵點提取與匹配 20
5.1分析閃光群各自相似性 23
5.2 SURF特徵擷取匹配演算法 25
5.2.1 特徵擷取 25
5.2.2 初始特徵匹配 26
5.2.3 選定候選特徵匹配組 27
5.2.4 RANSAC除錯法 29
5.2.5 相似度門檻值選定之分析 31
5.3 本章結論 33
第六章 視訊同步(video synchronization) 34
6.1 KEY-FRAME-LEVEL FRAME INDEX DIFFERENCE計算 34
6.2 VIDEO-LEVEL FRAME INDEX DIFFERENCE計算 35
6.3 多視訊同步分析 37
6.3.1 兩部視訊分析 37
6.3.2 三部視訊分析 39
6.4 分析時視訊之時間軸排序關係 41
6.4.1 時間軸排序關係之步驟流程推導 42
6.5 本章結論 45
第七章 實驗結果與性能評估 46
7.1 KEY-FRAME相似度比對法正確率比較 46
7.1.1 Bhattacharyya(巴氏)常態分布比對法 47
7.1.2 Cross-correlation 交叉相關比對法 50
7.1.3 SIFT特徵擷取比對法 52
7.1.4 SURF特徵擷取比對法 55
7.2 SURF相似度門檻值選定之分析 57
7.2.1選定候選特徵匹配組相似度門檻值之分析 57
7.2.2 RANSAC相似度門檻值之分析 59
7.3 每道閃光最中間之訊框進行正確匹配率分析 61
7.4 系統可容忍之角度架設分析 64
7.5 各環境與事件模擬之視訊同步撥放實驗結果 65
7.6 KEY-FRAME相似度比對法客觀評估 72
7.6.1 正確配對率客觀評估 72
7.6.2 比對執行速度客觀評估 73
7.7 視訊同步之觀看滿意度 74
7.7.1 主觀評估方式 74
7.8 本章節論 75
第八章 結論與未來展望 76
8.1 結論 76
8.2 未來研究方向 76
參考文獻 77

[1]行車記錄器-超車後急煞: https://www.youtube.com/watch?v=niauRlWLiTs
[2]運動視界-林智勝加盟記者會:http://www.sportsv.net/articles/24660
[3]中時報:http://www.chinatimes.com/realtimenews/20160104002767-260403
[4]Bi-Cheng Hsu, “Flashing Light Detection in Video”, 2006 輔仁大學資訊工程學系碩士論文
[5]Lin, Guo-Shiang, Min-Kuan Chang, and Shien-Tang Chiu. "A feature-based Scheme for detecting and classifying video-shot transitions based on spatio-temporal analysis and fuzzy classification." International Journal of Pattern Recognition and Artificial Intelligence 23.06 (2009): 1179-1200.
[6]Ciou, Sian-Tang. "建立於模糊分類器與轉換期間改良之鏡頭轉換偵測." 中興大學電機工程學系所學位論文 (2007): 1-60.
[7]Petersohn, Christian. "Fraunhofer HHI at TRECVID 2004: Shot boundary detection system." TREC Video Retrieval Evaluation Online Proceedings, TRECVID. 2004.
[8]Zhai, You, and Luan Zeng. "A SIFT matching algorithm based on adaptive contrast threshold." Consumer Electronics, Communications and Networks (CECNet), 2011 International Conference on. IEEE, 2011.
[9]李瑞文.郭忠民.楊乃中, “以 SIFT 建立有效的巨觀及微觀特徵描述在影像檢索及分類上的應用”, 2013 義守大學資訊工程學系碩士論文
[10]Huang, Wei-Ting. "Affinity propagation based image clustering with SIFT and color features." 清華大學資訊工程學系學位論文 (2009): 1-47.
[11]維基百科:
https://zh.wikipedia.org/wiki/%E5%B0%BA%E5%BA%A6%E4%B8%8D%E8%AE%8A%E7%89%B9%E5%BE%B5%E8%BD%89%E6%8F%9B
[12]360doc個人圖書館:
http://www.360doc.com/content/11/1129/15/3054335_168366315.shtml
[13]Baidu文庫:
http://wenku.baidu.com/view/d156036f7e21af45b307a81d.html?re=view
[14]阿洲的程式教學: http://monkeycoding.com/?p=803
[15]360doc個人圖書館:
http://www.360doc.com/content/11/1129/15/3054335_168366315.shtml
[16]博客: http://blog.csdn.net/zddblog/article/details/7521424
[17]范有臣, 李迎春, and 都琳. "基于 SURF 算法的微小位移测量." 激光與紅外 42.6 (2012): 699-704.
[18]Chen, Chi-Cheng, Liang-Hwei Lee. “The Study of Image Matching Based on Feature Point Selection and Description Method”, 2013 國立高雄應用科技大學土木工程與防災科技研究所碩士論文
[19]壹讀: https://read01.com/AP7mM.html
[20]維基百科:
https://zh.wikipedia.org/wiki/%E5%8A%A0%E9%80%9F%E7%A8%B3%E5%81%A5%E7%89%B9%E5%BE%81
[21]壹讀: https://read01.com/QLayA.html
[22]博客: http://blog.csdn.net/luoshixian099/article/details/47778143
[23]Yi-Chong Zeng and Wen-Tsung Chang, “Fast Seriation of Multiple Homogeneous-content Videos Using Audio-visual Features,” in Proc. of the International Computer Symposium, pp. 1157 – 1166, 2014.
[24]Prarthana Shrestha, Mauro Barbieri, Hans Weda, and Dragan Sekulovski, “Synchronization of Multiple Camera Videos Using Audio-Visual Features,” IEEE Transactions on Image Processing, pp. 79-92, Jan. 2010.
[25]Caspi, Yaron, Denis Simakov, and Michal Irani. "Feature-based sequence-to-sequence matching." International Journal of Computer Vision 68.1 (2006): 53-64.
[26]Yan, Jingyu, and Marc Pollefeys. "Video synchronization via space-time interest point distribution." Advanced Concepts for Intelligent Vision Systems. Vol. 1. 2004.
[27]Ushizaki, Manabu, Takayuki Okatani, and Koichiro Deguchi. "Video synchronization based on co-occurrence of appearance changes in video sequences." 18th International Conference on Pattern Recognition (ICPR'06). Vol. 3. IEEE, 2006.
[28]Dmitry Pundik,Yael Moses, “Video Synchronization Using Temporal Signals from Epipolar Lines,” 11th European Conference on Computer Vision, pp. 15-28, 2010.
[29]Dai, Yu-Chia. "以事件為基礎之多重視角影片同步系統." 臺灣大學資訊工程學研究所學位論文 (2011): 1-27.
[30]Dexter, Emilie, Patrick Pérez, and Ivan Laptev. "Multi-view Synchronization of Human Actions and Dynamic Scenes." BMVC. 2009.
[31]Rao, Cen, et al. "View-invariant alignment and matching of video sequences."Computer Vision, 2003. Proceedings. Ninth IEEE International Conference on. IEEE, 2003.
[32]Shrestha, Prarthana, et al. "Synchronization of multiple video recordings based on still camera flashes." Proceedings of the 14th ACM international conference on Multimedia. ACM, 2006.
[33]昨日: http://yester-place.blogspot.tw/2009/01/opencv.html
[34]維基百科:
https://zh.wikipedia.org/wiki/%E9%9A%A8%E6%A9%9F%E6%8A%BD%E6%A8%A3%E4%B8%80%E8%87%B4
[35]維基百科: https://zh.wikipedia.org/wiki/%E7%9B%B8%E5%85%B3
[36]Choi, Euisun, and Chulhee Lee. "Feature extraction based on the Bhattacharyya distance." Pattern Recognition 36.8 (2003): 1703-1709.
[37]Shrestha, Prarthana, et al. "Automatic mashup generation from multiple-camera concert recordings." Proceedings of the 18th ACM international conference on Multimedia. ACM, 2010.
[38]Casanovas, Anna Llagostera, and Andrea Cavallaro. "Audio-visual events for multi-camera synchronization." Multimedia Tools and Applications 74.4 (2015): 1317-1340.
[39]Lin, Guo-Shiang, and Jie-Fan Chang. "Detection of frame duplication forgery in videos based on spatial and temporal analysis." International Journal of Pattern Recognition and Artificial Intelligence 26.07 (2012): 1250017.