臺灣博碩士論文加值系統

在這篇論文中，我們提出了一個新的影片簡介系統架構，這個系統可以針對使用者所輸入的影片來選取他們感興趣的精采片段並做為整段影片的簡介。在本系統中會先將影片作整體內容的分析以及場景變換的偵測藉以把影片分割成許多段不同的鏡頭(shot)，再選取每一個鏡頭中的關鍵畫面(key-frame)作為鏡頭分類(shot clustering)的依據。在分類的時候是根據每一個關鍵畫面的場景(scene)以及動作(motion type)作為根據，如果兩兩的鏡頭擁有相似的場景以及動作，那麼它們將會被分類到同一群(cluster)裡面。使用我們所提出的分類方法之後，使用者可以得到許多由不同場景以及動作所構成的群。之後，本系統會根據每一個鏡頭的顏色變化程度(color variation)以及物體的動作大小(motion energy)選出每一個群裡面的精采畫面(highlight)來重新建構這段影片的簡介(summary)，並根據精彩畫面的重要性來決定這些鏡頭在最後的簡介中可以播放的長度，這麼一來可以讓使用者所得到的影片簡介更加的簡潔。

The paper presents a new framework of video summarization which can extract and summarize video shots that a user interests in from a long and complicated video, according to their similarity of motion type and scene. Firstly, the shot detection adopts the color and edge information to make shot boundaries accurate. Then the clustering process classifies the shots according to their similarities in scenes and motion types. Finally, we select the important shots of each cluster by estimating their priority value. The priority value determines the importance of each shot by measuring the motion energy and color variation.
The proposed method can produce a classified video summary, which allows users to review and search the video more easily. Experiment results illustrate that the proposed method can successfully classify a video into several clusters of different motion types and scenes, and extract the specific shots according to their importance.

摘要.....................................................ii
Abstract................................................iii
Chapter 1. Introduction...................................1
1.1. Motivation.......................................1
1.2. Related Works....................................1
1.3. System Framework.................................3
1.4. Thesis Overview..................................5
Chapter 2. Video Decomposition............................6
2.1. Color Difference using Mutual Information........8
2.2. Edge Covering Ratio.............................10
Chapter 3. Video Shot Classification.....................13
3.1. Key-Frame Extraction............................13
3.1.1. Motion Attention Model..........................15
3.1.2. Color Variation.................................17
3.2. Clustering Algorithm............................18
Chapter 4. Video Composition.............................26
4.1. Priority Value..................................26
4.2. Video Frames Skimming...........................29
Chapter 5. Experimental Results..........................31
5.1. Shot Detection..................................31
5.2. Video Summarization.............................32
Chapter 6. Conclusions...................................48
Reference ................................................49

[1] C. H. Ngo, Y. F. Ma, and H. J. Zhang, “Video summarization and scene detection by graph modeling,” IEEE Transactions on Circuits and System for Video Technology, vol. 15, no. 2, pp. 296-305, Feb. 2005.
[2] Y. F. Ma, X. S. Hua, L. Lu, and H.J. Zhang, “A Generic Framework of User Attention Model and Its Application in Video Summarization,” IEEE Transactions on Multimedia, vol. 7, no. 5, pp. 907-919, Oct. 2005.
[3] S. Lu, M. R. Lyu and I. King, “Semantic Video Summarization Using Mutual Reinforcement Principle and Shot Arrangement Patterns,” In Proceedings of the International Multimedia Modelling Conference, 2005.
[4] Z Cerneková, I Pitas, and C Nikou, “Information Theory-Based Shot Cut/Fade Detection and Video Summarization,” IEEE Transactions on Circuits and System for Video Technology, vol. 16, no. 1, pp. 82-91, Jan. 2006.
[5] S. Lu, I. King and M. R. Lyu, “Video Summarization by Video Structure Analysis and Graph Optimization,” In IEEE International Conference on Multimedia and Expo, 2004.
[6] Z. Rasheed and M. Shah, “Detection and Representation of Scenes in Videos,” IEEE Transactions on Multimedia, vol. 7, no. 6, pp. 1097-1105, Dec. 2005.
[7] L. Itti, C. Koch, and E. Niebur, “A Model of Saliency-Based Visual Attention for Rapid Scene Analysis,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp. 1254-1259, Nov. 1998.
[8] C. M. Privitera and L. W. Stark, “Algorithms for Defining Visual Regions-of-Interest: Comparison with Eye Fixations,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 9, pp. 970-982, Sep. 2000.
[9] C. L. Huang and B. Y. Liao, “A Robust Scene-Change Detection Method for Video Segmentation,” IEEE Transactions on Circuits and System for Video Technology, vol. 11, no. 12, pp. 1281-1288, Dec. 2001.
[10] B. V. Funt, and G. D. Finlayson, “Color Constant Color Indexing,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 17, no. 5, pp. 522-529, May. 1995.
[11] L. Kotoulas, and I. Andreadis, “Colour histogram content-based image retrieval and hardware implementation,” IEE Proceedings on Circuits, Devices, and Systems, vol.150, no. 5, pp. 387-393, Oct. 2003.
[12] M. J. Swain and D. H. Ballard, “Color indexing,” International Journal of Computer vision, 7:11-32, 1991.
[13] H. J. Zhang, A. Kankanhalli, and S. W. Smoliar, “Automatic partitioning of full-motion video,” Multimedia System, vol. 1, pp. 10–28, 1993.
[14] I. K. Sethi and N. Patel, “A statistical approach to scene change detection,” SPIE, vol. 2420, pp. 329-338, 1995.
[15] G. Boccignone, A. Chianese, V. Moscato, and A. Picariello, “Foveated Shot Detection for Video Segmentation,” IEEE Transactions on Circuits and System for Video Technology, vol. 15, no. 3, pp. 365-377, Mar. 2005.
[16] J. Bescós, G. Cisneros, J.M. Martínez, J.M. Menéndez, and J. Cabrera, “A Unified Model for Techniques on Video-Shot Transition Detection,” IEEE Transactions on Multimedia, vol. 7, no. 2, pp. 293-307, Apr. 2005.
[17] M. Yeung, B. L. Yeo, and B. Liu, “Extracting story units from long programs for video browsing and navigation,” In Proceedings of the Third IEEE International Conference on Multimedia Computing and Systems, pp. 296-305, 1996.
[18] Y. F. Ma, and H. J. Zhang, “A model of motion attention for video skimming,” In IEEE ICIP, 2002.

[19] C. M. Privitera and L. W. Stark, “Algorithms for Defining Visual Regions-of-Interest: Comparison with Eye Fixations,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 9, pp. 970-982, Sep. 2000.
[20] C. W. Ngo, T. C. Pong, H. J. Zhang, and R. T. Chin, “Motion-based Video Representation for Scene Change Detection,” In Proceedings of 15th International Conference on Pattern Recognition, vol. 1, pp. 827-830, Sep. 2000.
[21] R. C. Gonzales, and R. E. Woods, “Digital Image Processing 2nd Edition,” In Prentice Hall, 2002.