臺灣博碩士論文加值系統

在這論文中，我們發展基於分析和描述視頻時空之特性來對視訊資料庫做視訊樣例檢索。我們的系統以視訊索引(video index)為基礎對視訊短片(video shot)中所具有的特徵來檢索出示訊資料庫中相似的視訊短片，而且我們對視訊短片做處理用以產生顏色、運動、物體三個不同階層的訊息。首先，我們利用 Schunck的方法估計出運動向量場(motion field)。在這裡，我們對視訊短片中的每一畫面(frame)分析得到顏色方面的資訊，每15張取一張得到運動方面的資訊。
在顏色層次(level)的分析中，我們對視訊短片中的的每一畫面從RGB顏色空間轉換到HSV顏色空間。然後使用累積的顏色直方圖把視訊短片切割成連續的片段(clip)。然後我們分析該片段顏色直方圖的資訊作為視訊索引值。在顏色層次的分析中，我們利用空間上的關係和具有相似的運動向量來把像素(pixel)聚集成區域，並產生可以用來獲得局度運動直方圖和整體運動直方圖的 MotionImage。在物體層次的分析中，我們尋找視頻中移動的物體當作視訊物體。我們首先使用畫面像素差值以及運動向量場的資訊來找出物體的區域。然後我們在緊鄰的取樣畫面之間做區域軌跡以找出屬於物體的區域。使用這些追蹤後的區域，我們可以計算出物體的質心、大小、和平均運動向量。

In this thesis, we will develop a video query by examples (QBE) operation for video database system based on the analysis and description of the spatio-temporal characteristics of the video sequences. Our system uses the characteristic of the video shot to retrieve similar shots in the database based on the video indices, and we processes the video shots to generate three hierarchical information: color, and motion, and object. First. We use Schunck method to estimate the motion field. Here, we analyze each frame to obtain the color level information, and one of every fifteen frame to obtain the motion information.
In the color level analysis, we transform the RGB color space to the HSV color space for each frame in the video shot. Then we use the accumulated color histogram to separate the video shot into a series of clips. Then we analyze the clip color histograms for video indexing. In the motion level analysis, we use the spatial-relationship of pixels and the similarity of their motion vectors to cluster pixels into region, and generate a so-called motion image, which can be used to obtain the local motion histogram and global motion histogram. In the object level analysis, we search for the video objects, which are moving objects in the video sequence. We first use the difference image and the data of the motion field to find out the regions of object. Then we take a region tracing between the sub-sampled frame to find the object regions. Using these traced regions, we can calculate the centroid, and the size, and the average motion vector of the moving objects.

Chapter 1 Introduction ………………………... 1
Chapter 2 Video Property Analysis …………... 7
2.1 Color Level Measurement …………………………………..… 7
2.1.1 HSV Color Space ………………………………………………………… 7
2.1.2 Video Clip Color Histogram ……………………………………………… 8
2.2 Motion Level Measurement ……………………………..…… 12
2.2.1 Motion Field Analysis ……………………………………………...…… 12
2.2.2 Motion Histogram …………………………………………….………… 14
2.3 Object Level Measurement ………………………………...… 17
2.3.1 Preprocessing …………………………………………………………… 17
2.3.2 The Difference Map …………………………………………………….. 20
2.3.3 Region Extraction ……………………………………………………..… 22
2.3.4 Region Tracing in Image Sequence ……………………………...……… 26
2.3.5 Object Trajectory Characteristic ……………………..…………..……… 30
Chapter 3 Video Retrieval …………………… 34
Chapter 4 Experimental Results …………..… 38
4.1 Color level similarity ……………………………………….... 38
4.2 Motion level similarity ………………………………………. 38
4.3 Motion Parameters Estimation ………………………………. 39
Chapter 5 Conclusion ………………………... 48
References …………………………………….. 49

1. F. Arman et al., “Content-based Browsing of Video Sequence,” ACM Int. Conf. On Multimedia, pp. 97-103, Oct. 1994.
2. H. Aoki et al., “A Shot Classification Method of Selecting Effective Key-Frames for Video Browsing,” ACM Multimedia, pp.1-10, 1996.
3. N. Dimitrova et al., “Content-based Video Retrieval by Example Video Clip,” Storage and Retrieval for Still Image and Video Database V, Vol. SPIE-3022, pp.59-70, Feb 1997.
4. M Yeung and B. Yeo, “Time constrained Clustering for Segmentation of Video into Story Unit,” Proceedings of ICPR pp.375-380.
5. B. Yeo and M. Yeung, “Retrieving and Visualizing Video,” Communication of the ACM, Vol. 40, No. 12, pp.43-52, Dec. 1997.
6. Y. P. Tan, S. R. Kulkarni, P. J. Ramadge, ”A Framework for Measuring Video Similarity and Its Application to Video Query by Example,” IEEE ICIP 99. Proceedings. 1999 International Conference, pp. 106 -110 vol.2.
7. M. Kurokawa, et al., “Representation and Retrieval of Video Scene by using Object Actions and Their Spatio-Temporal Relationships,” IEEE ICIP 99. Proceedings. 1999 International Conference, pp. 86 -90 vol.2.
8. K. Akira and K. Hirotsugu, “An Image Retrieving Method using the Object Index and the Motion” IEEE ICIP 99. Proceedings. 1999 International Conference Volume: 3 , 1999 , pp. 269 —273, vol.3.
9. P. Salembier, et al., “Hierarchical Visual Description Schemes for Still Images and Video Sequences,” IEEE ICIP 1999.Proceedings. 1999 International Conference, pp. 121 -125 vol.2.
10. N. Dimitrova et al., “Color Super-Histograms for Video Representation,” IEEE ICIP 1999. Proceedings. 1999 International Conference, pp. 314 —318 vol.3.
11. R. Zabih, J. Miller, and K. Mai, “Video Browsing using Edges and Motion,” IEEE Computer Society Conference ,1996, pp. 439 —446.
12. J. Motsch and H. Nicolas, “Object-Based Motion Classification,” IEEE ICIP 1999.Proceedings. 1999 International Conference, pp. 101 -105 vol.2.
13. K. D. Vo, I. Nishihara, T. Yoshida, and Y. Sakai, “Precise Estimation of Motion Vectors and Its application to MPEG Video retrieval,” IEEE ICIP Proceedings. 1999 International Conference, pp. 279 -283 vol.3
14. K. Y. Wong et al., “Motion Segmentation and Indexing for Video Database,” 12th Conf. on Vision Interface, Trios-Rivieres, Canada, 1999.
15. H. L. Eng and K. K. Ma, “Motion trajectory Extraction based on macroblock Motion Vectors for Video Indexing,” IEEE ICIP 1999.Proceedings.1999 International Conference, pp.284-288.
16. B. K. P. Horn, B. G. Schunck, “Determinig optical flow,” Artificial Intelligence 17, 1980, pp.185-203.
17. D. Androutsos, K. N. Plataniotis, and A. N. Venetsanopoulos, “A Novel Vector-Based Approach to Color Image Retrieval Using a Vector Angular-Based Distance Measure,”Computer Vision and Image Understanding,Vol.75, Nos.1/2, July/August, pp.46-58, 1999.
18. R. M. Haralick, L. G. Shapiro, “Computer and Robot Vision Vol. 1,” Addison Wesley, pp.14-23, 1992