關鍵畫面的萃取在智慧型監視系統中佔有舉足輕重的作用。使用者透過關鍵畫面以得知監控事件之概要，節省觀看事件的時間、提升監控效率。此外萃取關鍵畫面也有利於後續之高階特徵分析與影片檢索查詢。本論文提出一個以物件為基礎之關鍵畫面萃取方法達到取得具代表性畫面的目的，並且提出以物件高階語義特徵為基礎結合權重以及卡式濾波器之關鍵畫面萃取方法。本論文所提出的方法可即時於事件發生時萃取出關鍵畫面，關鍵畫面中的物件不限單一物件，亦可應用於多物件的複雜環境中。
本論文提出之方法經大量實驗驗證其可行性,除了以二十段實驗影片分析實作外，也與IEEE期刊論文已發表的方法比較，並實作於real-time手機監控系統。實驗影片取自MPEG-4測試影片等，影片具單人及多人情境且在具光源變化的真實複雜環境。經實驗證明本論文所提的方法不僅可以取得清晰且具有代表性的關鍵畫面外，也可以減少多餘的關鍵畫面產生。

Key frame extraction is an important step for video surveillance. Key frames are able to inform users about the concept of an alarm event and guard environment more efficiently. Key frames can also be used for analysis of feature extraction, indexing and video retrieval. This paper proposes an object-based key frame extraction method for extracting representative frames of an alarm event. The method combines semantic features and weighted importance to extract key frames and devises an object features based formula to obtain better key frames that have clear object image. We also adopt Kalman filter to help predict objects’ situations and extract key frames during the events. Our method not only processes single object but also deals with multiple objects in on-line procedure.
The proposed method has been verified by large amounts experiments that include testing 20 clips, comparing with other proposed key frame extraction method and implementing in a real-time mobile surveillance system. The experimental videos consist of single objects and multi-objects that are from MPEG-4 test video and so on. Our method proved by experiments not only can get clear and representative key frames but also reduce redundant key frames.

Abstract (in Chinese) i
Abstract ii
Acknowledge iii
Contents iv
List of Figures v
Chapter 1 Introduction 1
1.1 Background 1
1.2 Motivation 2
1.3 Organization of this thesis 3
Chapter 2 Related Work 5
Chapter 3 Object-Based Key Frame Extraction 9
3.1 Formulation of the Key Frame Extraction 9
3.2 Online Extraction by Kalman Filter 10
3.3 Path analyses 17
Chapter 4 Semantic Feature Extraction 23
4.1 Moving Object Detection and Tracking 23
4.2 Skin Color Extraction 26
4.3 Face Detection 27
Chapter 5 Experimental Results 29
5.1 Experimental Data 29
5.2 Weighted Importance Analysis 30
5.3 Noise Covariance Experiments 33
5.4 Multiple-Object Experiments 36
5.5 Comparison Experiments 44
Chapter 6 An Application to A Mobile Video Surveillance System 48
Chapter 7 Conclusions 53
References 54

[1]M. Shah, O. Javed, and K. Shafique, “Automated visual surveillance in realistic scenarios,” IEEE Transactions on Multimedia, Vol. 14, No. 1, pp. 30-39, 2007.
[2] Hao Jiang, S. Fels and J. J. Little, “Optimizing multiple object tracking and best view video synthesis,” IEEE Transactions on Multimedia, Vol. 10, No. 6, pp. 997-1012, 2008.
[3]Y.-H. Ho, C.-W. Lin, J.-F. Chen, and H.-Y. M. Liao, "Fast coarse-to-fine video retrieval using shot-level spatio-temporal statistics," IEEE Transactions on Citcuits and System for Video Technology, Vol. 16, No. 5, pp. 642-648, 2006.
[4]P. M. Fonseca and F. Pereira, "Automatic video summarization based on MPEG-7 description," Signal Processing: Image Communication, Vol. 19, No. 8, pp. 685-699, 2004.
[5]C.-Y. Chen, J.-C. Wang, and J.-F. Wang, "Efficient news video querying and browsing based on distributed news video servers," IEEE Transactions on Multimedia, Vol. 8, No. 2, pp. 257-269, 2006.
[6]C. Choudary and T. Liu, "Summarization of visual content in instructional videos," IEEE Transactions on Multimedia, Vol. 9, No. 7, pp. 1443-1455, 2007.
[7]X. Zhu, A. K. Elmagarmid, X. Xue, L. Wu, and A. C. Catlin, "Insight video: toward hierarchical video content organization for efficient browsing, summarization and retrieval," IEEE Transactions on Multimedia, Vol. 7, No. 4, pp. 648-666, 2005.
[8]A. M. Ferman and A. M. Tekalp, "Two-stage hierarchical video summary extraction to match low-level user browsering preference," IEEE Transactions on Multimedia, Vol. 5, No. 2, pp. 244-256, 2003.
[9]G. Ciocca and R. Schettini, "Supervised and unsupervised classification post-preprocessing for visual video summaries," IEEE Transactions on Consumer Electronics, Vol. 52, No. 2, pp. 630-638, 2006.
[10]N. D. Doulamis, A. D. Doulamis, Y. S. Avrithis, K. S. Ntalianis, and S. D. Kollias, "Efficient summarization of stereoscopic video sequence," IEEE Transactions on Circuits and System for Video Technology, Vol. 10, No. 4, pp. 501-517, 2000.
[11]T. Liu, H.-J. Zhang and Feihu Qi, "A novel video key-frame-extraction algorithm based on perceived motion energy model," IEEE Transactions on Circuits and System for Video Technology, Vol. 13, No. 10, pp. 1006-1013, 2003.
[12]C. Kim and J.-N. Hwang, "Object-based video abstraction for video surveillance systems," IEEE Transactions on Circuits and Systems for Video Technology, Vol. 12, No.12, pp. 1128-1138, 2002.
[13]D. P. Mukherjee, S. K. Das, and S. Saha, "Key frame estimation in video using randomness measure of feature point pattern," IEEE Transactions on Circuits and Systems for Video Technology, Vol. 17, No. 5, pp. 612-620, 2007.
[14]L. Liu and G. Fan, "Combined key frame extraction and object-based video segmentation," IEEE Transactions on Circuits and Systems for Video Technology, Vol. 15, No.7, pp. 869-884, 2005.
[15]X. Song and G. Fan, "Joint key-frame extraction and object segmentation for content-based video analysis," IEEE Transactions on Circuits and Systems for Video Technology, Vol. 16, No. 7, pp. 905-914, 2006.
[16]K. Ito and K. Xiong, "Gaussian Filter for Non-linear Problems," IEEE Transactions on Automatic Control, Vol. 45, No. 5, pp. 910-927, 2000.
[17]Zhuang, Y., Rui, Y., Huang T. S., and Mehrotra, S., "Adaptive key frame extraction using unsupervised clustering," Proceedings, IEEE Image Processing Conference, Chicago, IL,USA, pp. 886-870, 1998.
[18]B. Erol and F. Kossentini, "Automatic key video object plane selection using the shapeinformation in the MPEF-4 compressed domain," IEEE Transactions on Multimedia, Vol. 2, No. 2, pp. 129-138, 2000.
[19]M. K. Hu, "Visual pattern recognition by moment invariants," IRE Transaction on Information Theory, Vol. 8, No. 2, pp. 179-187, 1962.
[20]R. E. Kalman, "A new approach to linear filtering and prediction problem," Transaction of the ASME-Journal of Basic Engineering, No. 82, pp. 35-45, 1960 .
[21]P. Viola and M. J. Jonse, “Robust real-time face detection,” International Journal of Computer Vision, Vol. 57, No. 2, pp. 137-154, 2004.
[22]L. M. Bergasa, M. Mazo, A. Gardel, M. A. Sotelo, L. Boquete, “Unsupervised and adaptive Gaussian skin color model,” Image and Vision Computing, Vol. 18, No. 12, pp. 987-1003, 2000.
[23]Wang, Y. K. and Su, C. H., “Illuminant-invariant Bayesian Detection of Moving Video Objects,” Proceedings, International Conference on Signal and Image Processing Conference, Hawaii, USA, pp. 57-62, 2006.
[24]K. Nummiaro, E. Koller-Meier, L. V. Gool, “An adaptive color-based particle filter,” International Image and Vision Computing, Vol. 21, No. 1, pp. 1-12, 2002.
[25]Wang, Y. K., Wang, L. Y., Hu, Y. H., “A mobile surveillance system with intelligent analysis,” Proceedings, International SPIE on Electronic Imaging Conference , San Jose, USA, pp. 68210I-01-08, 2008.