臺灣博碩士論文加值系統

本論文以分割畫面技巧(Frame Partitioning Scheme)為基礎，提出場景檢測與影像檢索方法。在場景檢測方面，基於前後分割畫面差之場景轉換檢測方法，稱為SD/PFDS (Shot Detection Based on Partitioned Frame Differencing Scheme)。在SD/PFDS方法中，我們以畫面群為基本單位，以第一張畫面為參考畫面，後續畫面為比較畫面，並將每一張畫面做分割，計算分割畫面間的像素差值來判斷是否發生場景轉換。在實驗範例中，我們所提的方法在大多數影片類型當中，不管是突然式場景變換、漸進式場景變換之檢測正確率都有不錯的表現，這說明了所提的SD/PFDS方法是具體可行的。
另外，我們也應用分割畫面的技巧於影像內容檢索CBIR (Content-Based Image Retrieval)。基於色彩和紋理特徵，本論文提出了IR/PCF (Image Retrieval with Partitioned Color Features)、IR/PTF (Image Retrieval with Partitioned Texture Features)與IR/PCTF(Image Retrieval with Partitioned Color and Texture Features)等檢索方法。在IR/PCF方法中，以分割顏色特徵檢索的步驟如下：首先，將影像分割為數個區塊。第二、基於查詢影像中R、G、B三個成分的能量(Energy)找出各成分之權重，用於相似度評估計算。第三、在分割影像區塊中，計算各成分平均值做為顏色特徵。第四，利用權重計算相似度，得到顏色相似度距離。
在IR/PTF方法中，我們使用灰階共生矩陣GLCM (Gray Level Co-occurrence Matrix)來擷取紋理特徵，其檢索步驟如下：首先，將彩色影像轉為灰階影像並分割為數個區塊。第二、計算各分割影像區塊中的GLCM紋理特徵。第三、計算查詢影像與資料庫影像之紋理特徵距離。
最後，我們結合顏色與紋理二個特徵於影像檢索中，在IR/PCTF方法中，顏色與紋理特徵的距離分別經過正規化，然後依照特徵的重要性取權重，得到相似度距離。實驗結果顯示，在所提的三個方法中，IR/PCTF方法具有最好的檢索，其次是IR/PCF方法，最後是IR/PTF方法。這說明了適當的結合顏色與紋理特徵，確實比起使用單一特徵有更好的檢索效能，並且能明顯提升檢索效能。

This thesis presents approaches to shot detection and image retrieval based on frame partitioning scheme. For the shot detection, the proposed approach is called SD/PFDS (Shot Detection Based on Partitioned Frame Differencing Scheme). In the SD/PFDS, frames are grouped and partitioned into image blocks. The first frame in the group is considered as reference frame and the others compared frames. Then the differences for each image blocks between partitioned reference and compared frames are calculated. By the differences, changes of shots are detected. The proposed SD/PFDS approach is verified by several examples. The results indicate that the overall average accuracy of detection is as high as 0.94 in F1 measure. By the results, the SD/PFDS approach has been justified and shown feasible.
Also, we apply the frame partitioning scheme to image retrieval. With color and texture features, the thesis present three approaches to image retrieval: IR/PCF (Image Retrieval with Partitioned Color Features), IR/PTF (Image Retrieval with Partitioned Texture Features), and IR/PCTF (Image Retrieval with Partitioned Color and Texture Features). Based on partitioned color features, several stages are involved in the IR/PCF. First, images are partitioned. Second, energies in R-, G-, B-components for the partitioned query image are calculated through which weights on the similarity measure are found. Third, find averages of R-, G-, B-components in partitioned image as color features. Finally, calculate the similarity with weights obtained in the second stage.
In the IR/PTF, texture features are acquired by GLCM (Gray Level Co-occurrence Matrix). The IR/PTF approach consists of the following stages. First, convert color images in gray-level images. Second, find texture features by GLCM in the partitioned images. Third, calculate the similarity between query image and images in database.
The IR/PCTF approach uses both partitioned color and texture features. The following stages are involved in the IR/PCTF. First, the similarity measures are obtained by the IR/PCF and the IR/PTF, respectively. Then the similarity measures are normalized and linearly combined with weights proportional to the performance with only one partitioned feature, i.e., color or texture. The resulted similarity is then used in image retrieval.
The three proposed image retrieval approaches are justified by image databases. It shows that the IR/PCTF is of highest retrieval performance and then the IR/PCF, and finally the IR/PCTF. With an appropriate combination of partitioned color and texture features, the IR/PCTF shows better performance than those in the IR/PCF and the IR/PTF.

索引目錄
中文摘要 I
Abstract III
致謝 V
索引目錄 VI
表目錄 IX
圖目錄 X
第一章 簡介 1
1.1研究動機 1
1.2研究資料庫 2
1.3論文架構 3
第二章 相關文獻回顧 4
2.1 場景轉換檢測相關方法 4
2.1.1常見場景轉換方式 5
2.1.1.1突然式場景轉換 5
2.1.1.2漸進式場景轉換 5
2.1.2場景檢測方法回顧 8
(A) 以影像差異為基礎之檢測方法 8
(B) 以顏色直方圖為基礎之檢測方法 9
(C) 以 直方圖結合Twin-Comparison檢測法 9
(D) 以局部關鍵點匹配檢測法 10
(E) 以線性轉換檢測檢測法 11
(F) 以語義為代表檢測法 12
2.2影像內容檢索相關回顧 12
(A) 以顏色直方圖為基礎的方法 12
(B) 以顏色相關圖為基礎的方法 13
(C) 結合顏色與紋理特徵之方法 14
(D) 以形狀特徵為基礎的方法 15
(E) 基於影像區域檢索的方法 16
第三章 以分割畫面為基礎之場景轉換檢測 17
3.1 以分割畫面為基礎之場景檢測方法 17
3.1.1 動機 18
3.1.2 AT檢測方法 20
3.1.3 GT檢測方法 21
3.1.4 GT檢測實行步驟 23
3.2 實驗結果與探討 26
3.2.1 實驗結果 29
3.2.2 結果探討 31
(A) 漸進式場景轉換誤判例子分析 32
(B) 漸溶式場景轉換漏判例子分析 33
3.3 與其他方法比較 37
第四章 以分割畫面為基礎之影像檢索方法 40
4.1 基於分割顏色與紋理特徵之影像檢索方法 40
4.1.1 IR/PCF影像檢索方法 41
4.1.2 IR/PTF影像檢索方法 43
4.1.2.1 擷取GLCM紋理特徵之回顧 43
4.1.2.2 IR/PTF方法之實行步驟 46
4.1.3 IR/PCTF影像檢索方法 47
4.2實驗結果與探討 48
4.2.1 IR/PCF方法之實驗結果 51
4.2.2 IR/PTF方法之實驗結果 52
4.2.3 IR/PCTF方法之實驗結果 53
第五章 結論 61
參考文獻 62

表目錄
表2-1 Bins與Sub-divisions的顏色相關圖 14
表3-1 範例影片場景轉換類型統計表 29
表3-2 所提出SD/PFDS場景轉換方法檢測統計表 30
表3-3 不同參數L對F1影響的統計表 31
表3-4 SD/PFDS方法與其他方法之比較結果 39
表4-1 文獻[14]使用之資料庫影像類別表 49
表4-2 文獻[15]使用之資料庫影像類別/數量表 49

圖目錄
圖2-1 突然式場景轉換說明圖 5
圖2-2 突然式場景轉換畫面 5
圖2-3 淡出、淡入式場景轉換說明圖 6
圖2-4 淡出式場景轉換畫面 6
圖2-5 淡入式場景轉換畫面 6
圖2-6 漸溶式場景轉換說明圖 7
圖2-7 漸溶式場景轉換畫面 7
圖2-8 抹去式場景轉換畫面 7
圖2-9 通過物件8個方向示意圖 15
圖3-1 畫面分割示意圖 18
圖3-2 背景轉動範例 19
圖3-3 畫面間為相異時示意圖 22
圖3-4 所提出之SD/PFDS場景轉換檢測方法流程圖 23
圖3-5 GT場景轉換檢測流程圖 26
圖3-6 範例影片之代表畫面 27
圖3-7 鏡頭左右搖晃以及移動所造成的誤判 32
圖3-8 特寫鏡頭所造成的誤判 33
圖3-9 翻頁式的場景變換 34
圖3-10 畫面取代式的場景變換 35
圖3-11 漸溶式的場景變換漏判情況(一) 36
圖3-12 漸溶式的場景變換漏判情況(二) 36
圖4-1 畫面分割示意圖( ) 43
圖4-2 大小為4×4之I影像 45
圖4-3 計算 所有應使用之座標對 45
圖4-4 大小為4×4之4灰階影像 45
圖4-5 為圖4-4之灰階共生矩陣 46
圖4-6 IR/PCTF影像檢索方法之流程圖 48
圖4-7 資料庫[14]所使用之查詢影像 50
圖4-8 資料庫[15]所使用之查詢影像 51
圖4-9 不同L值對IR/PCF方法影響之 比較圖 52
圖4-10不同 值對IR/PCTF方法影響之 比較圖 53
圖4-11“Buildings”類別前10張相似影像檢索結果 (a)IR/PCF檢索結果 (b)IR/PTF檢索結果 (c)IR/PCTF檢索結果 54
圖4-12“Laptop”類別前10張相似影像檢索結果 (a)IR/PCF檢索結果 (b)IR/PTF檢索結果 (c)IR/PCTF檢索結果 55
圖4-13“Elephants”為查詢影像的前10張相似影像檢索結果 56
圖4-14“Flowers”為查詢影像的前10張相似影像檢索結果 57
圖4-15“Faces”為查詢影像的前10張相似影像檢索結果 57
圖4-16“Ketch”為查詢影像的前10張相似影像檢索結果 58
圖4-17“Food”為查詢影像的前10張相似影像檢索結果 58
圖4-18“Crab”為查詢影像的前10張相似影像檢索結果 59
圖4-19使用資料庫[14] IR/PTF、IR/PCF、IR/PCTF之整體平均準確率比較率.... 60
圖4-20使用資料庫[15] IR/PTF、IR/PCF、IR/PCTF之整體平均準確率比較率.... 60

[1]Hun Cheol Lee, Cheong Woo Lee, Seong Dae Kim, “Abrupt Shot Change Detection Using an Unsupervised Clustering of Multiple Features,” IEEE Int. Conf. on Acoustics, Speech, and Signal Processing, pp. 2015-2018, Vol. 4, 2000.
[2]Alan Hanjalic, Hong-Jiang Zhang, “An Integrated Scheme for Automated Video Abstraction Based on Unsupervised Cluster Validity Analysis,” IEEE Trans. Circuits Sys. Video Tech., Vol. 9, pp. 1280-1289, Dec. 1999.
[3]Jianyu Wang, Wen Luo, “A Self-adapting Dual-threshold Method for Video Shot Transition Detection,” IEEE Int. Conf. on Networking, Sensing and Control. pp. 704-707, Apr. 2008.
[4]Waqas Rasheed, Gwangwon Kang, Jinsuk Kang, Jonghun Chun, Jongan Park, “Sum of Values of Local Histograms for Image Retrieval,” IEEE Int. Conf. on Networked Computing and Advanced Information Management, Vol. 2, pp. 690-694, Sept. 2008.
[5]Waqas Rasheed, Youngeun An, Sungbum Pan, Ilhoe Jeong, Jongan Park, Jinsuk Kang, “Image Retrieval Using Maximum Frequency of Local Histogram Based Color Correlogram,” IEEE Int. Conf. on Modeling & Simulation, pp. 322-326, May 2008.
[6]Chuen-Horng Lin, Rong-Tai Chen, Yung-Kuan Chan, “A Smart Content-based Image Retrieval System Based on Color and Texture Feature,” Image and Vision Computing, Vol. 27, pp. 658-665, May 2009.


[7]Yung-Kuan Chan, Yu-An Ho, Yi-Tung Liu, Rung-Ching Chen, “A ROI Image Retrieval Method Based on CVAAO,” Image and Vision Computing, Vol. 26, Issue 11, pp. 1540-1549, Nov. 2008.
[8]Open-Video [Online] Available: http://www.open-video.org/
[9]TRECVID2001 video by NIST. [Online] Available:http://www-nlpir.nist.
gov/projects/trecvid/revised.html
[10]X Wu, Pong C Yuen, C Liu, J Huang, “Shot Boundary Detection: An Information Saliency Approach,” IEEE Congress on Image and Signal Processing, Vol. 2, pp. 808-812. May 2008.
[11]Chun-Rong Huang, Huai-Ping Lee, and Chu-Song Chen, “Shot Change Detection via Local Keypoint Matching,” IEEE Trans. Multimedia, Vol. 10, pp. 1097-1108, Oct. 2008.
[12]Robert M. Haralick, “Statistical and Structural Approaches to Texture,” Proceedings of the IEEE, Vol. 67, pp. 786-804, 1979.
[13]Andrea Baraldi, Flavio Panniggiani, “An Investigation of the Textural Characteristics Associated with Gray Level Cooccurrence Matrix Statistical Parameters,” IEEE Trans. on Geoscience and Remote Sensing, Vol. 33, pp. 293-304, Mar. 1995.
[14]Jia Li, James Z. Wang, “Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 25, No. 9, pp. 1075-1088, Sep. 2003.
[15]Li Fei-Fei, Rob Fergus, Pietor Perona, “Learning Generative Visual Models from Few Training Examples: an Incremental Bayesian Approach Tested on 101 object Categories, ” Proceedings of IEEE Conf. on Computer Vision and Pattern Recognition, Workshop on Generative-Model Based Vision, pp. 178-178, 2004.
[16]VidoAnnex [Online]Available: www.research.ibm.com/VideoAnnEx/
[17]VCM[Online]Available:http://astral.ced.tuc.gr/delos/cls_resource_descprition.jsp?id=10129
[18]Costantino Grana, Rita Cucchiara, “Linear Transition Detection As a Unified Shot Detection Approach,” IEEE Trans. Circuits Sys. Video Tech., Vol. 17, pp. 483-489, Apr. 2007.
[19]Songhao Zhu, Yuncai Liu, “Automatic Scene Detection for Advanced Story Retrieval,” Expert Systems with Applications, Vol. 36, Part 2, pp. 5976-5986, Apr. 2009.
[20]C.-C. Chiang, Y.-P. Hung, H. Yang, G. C. Lee, “Region-Based Image Retrieval Using Color-Size Features of Watershed Regions,” Journal of Visual Communication and Image Representation, Vol. 20, pp. 167-177, Apr. 2009.