臺灣博碩士論文加值系統

影像分割是在電腦視覺和影像處理領域中的一個基本問題。雖然本主題已經被研究了許多年，它仍然是一項具有挑戰性的任務。近年來關於超像素(superpixel)的研究有很大的進展，並且這種新技術可以使傳統的影像分割演算法具有更高的效率和更好的性能。另一方面，顯著影像偵測的研究是影像處理的一個新的課題，其表現通常與使用的影像分割技術有深度相關。
在這篇論文中，我們提出了兩種演算法分別針對影像分割及顯著影像偵測。對於影像分割，我們基於採用超像素的圖形表示，提出一種有效的影像分割演算法。該演算法採用了包含SLIC 超像素，5-D譜聚類(spectral clustering)和邊界專注(boundary-focused)的區域合併等技術。利用SLIC超像素，原始影像分割問題被轉化為超像素分類問題。這使得該演算法比基於像素的分割演算法能有更佳的執行效率。利用5-D譜聚類和邊界集中的區域合併法，位置資訊可用於集群且區域合併的門檻值可根據影像作自適應調整。這些技術使分割結果更符合人類感知。依據在柏克萊分割數據庫的結果顯示，我們提出的方法優於目前已知最先進的方法。
對於顯著影像偵測，我們提出了一種非常有效的顯著影像偵測演算法。我們的演算法主要是基於以下兩個步驟。首先，離散餘弦變換（discrete cosine transform）用於產生塊單位(block-wise)的顯著影像圖。然後，基於超像素的影像分割演算法被應用以得到像素單位(pixel-wise)的顯著影像圖。由於離散餘弦變換係數可以反映在頻域的每個塊的顏色特徵且超像素可以很好地保留物體的邊界，這兩種技術可以大幅改善顯著影像偵測演算法的性能。根據在具有1000張影像的影像數據庫上進行的實驗結果，我們所提出的方法可以非常精確地提取顯著區域，並優於所有現有的顯著影像偵測方法。

Image segmentation is a fundamental problem in computer vision and image processing. Though this topic has been researched for many years, it is still a challenging task. Recently, the researches of superpixels have great improvement. This new technique makes the traditional segmentation algorithms more efficient and has better performances. On the other hand, the saliency detection is another new topic of image processing and its performance usually closely related to the segmentation techniques we used.
In this thesis, we propose two algorithms for image segmentation and saliency detection, respectively. For image segmentation, an effective graph-based image segmentation algorithm using the superpixel-based graph representation is introduced. The techniques of SLIC superpixels, 5-D spectral clustering, and boundary-focused region merging are adopted in the proposed algorithm. With SLIC superpixels, the original image segmentation problem is transformed into the superpixel labeling problem. It makes the proposed algorithm more efficient than pixel-based segmentation algorithms. With the proposed methods of 5-D spectral clustering and boundary-focused region merging, the position information is considered for clustering and the threshold for region merging can be adaptive. These techniques make the segmentation result more consistent with human perception. The simulations on the Berkeley segmentation database show that our proposed method outperforms state-of-the-art methods.
For saliency detection, a very effective saliency detection algorithm is proposed. Our algorithm is mainly based on two new techniques. First, the discrete cosine transform (DCT) is used for constructing the block-wise saliency map. Then, the superpixel-based segmentation is applied. Since DCT coefficients can reflect the color features of each block in the frequency domain and superpixels can well preserve object boundaries, with these two techniques, the performance of saliency detection can be significantly improved. The simulations performed on a database of 1000 images with human-marked ground truths show that our proposed method can extract the salient region very accurately and outperforms all of the existing saliency detection methods.

口試委員會審定書 #
誌謝 i
中文摘要 iii
ABSTRACT v
CONTENTS vii
LIST OF FIGURES xi
LIST OF TABLES xiv
Chapter 1 Introduction 1
1.1 Motivation 1
1.2 Main Contribution 1
1.3 Organization 2
Chapter 2 Overview of Recent Image Segmentation Methods 3
2.1 Normalized Cut 3
2.1.1 The Concept of Normalized Cut 3
2.1.2 The Algorithm 5
2.1.3 Simulation Results and Discussion 7
2.2 Efficient Graph-based Segmentation 9
2.2.1 The Concept of Efficient Graph-based Segmentation 9
2.2.2 The Algorithm 10
2.2.3 Simulation Results and Discussion 12
2.3 SLIC Superpixels 14
2.3.1 The Concept of SLIC Superpixels 14
2.3.2 The Algorithm 17
2.3.3 Simulation Results and Discussion 18
2.4 Segmentation by Aggregating Superpixels (SAS) 20
2.4.1 The Concept of Segmentation by Aggregating Superpixels 20
2.4.2 The Algorithm 24
2.4.3 Simulation Results and Discussion 26
Chapter 3 Overview of Recent Saliency Detection Methods 27
3.1 Context Aware Saliency Detection 27
3.1.1 The Concept of Context Aware 27
3.1.2 Simulation Results and Discussion 32
3.2 Global Contrast-based Salient Region Detection 34
3.2.1 The Concept of Global Contrast-based Salient Region Detection 34
3.2.2 Simulation Results and Discussion 37
3.3 Saliency Filters: Contrast-based Filtering 39
3.3.1 The Concept of Saliency Filters 39
3.3.2 Simulation Results and Discussion 46
Chapter 4 Proposed Segmentation Methods 49
4.1 Introduction 49
4.2 Superpixel Generation and Graph Representation 51
4.2.1 Superpixel Generation 51
4.2.2 Graph Representation 53
4.3 Clustering and Merging 55
4.3.1 Spectral Clustering in 5D Space 55
4.3.2 Boundary-Focused Region Merging 56
4.4 Experimental Results 61
4.4.1 Database 61
4.4.2 Comparison to Spectral Methods 62
4.4.3 Visual Comparison 65
4.4.4 Discussion of Parameters 67
4.4.5 Analysis of Our Algorithm 69
4.5 Conclusion 70
Chapter 5 Proposed Saliency Detection Methods 71
5.1 Introduction 71
5.2 DCT-based Block-wise Saliency Maps 75
5.2.1 Discrete Cosine Transform for Dimension Reduction 75
5.2.2 Block-based Saliency Detection Using DCT Coefficient 76
5.3 Boundary Scoring and Border Measurement 78
5.3.1 Boundary Scoring 78
5.3.2 Border Measurement 79
5.3.3 Precision-Enhancing Integration 82
5.4 Experimental Results 84
5.4.1 Database 84
5.4.2 Precision/Recall and F-measure 84
5.4.3 MAE and MSE 86
5.4.4 Visual Comparison 90
5.4.5 Analysis of Our Algorithm 91
5.5 Conclusion 91
Chapter 6 Conclusion and Future Work 93
6.1 Conclusion 93
6.2 Future Work 94
REFERENCE 95

A.Image Segmentation
[1]X. Ren and J. Malik, “Learning a classification model for segmentation,” In ICCV, pp. 10-17, 2003.
[2]R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Süsstrunk, “SLIC superpixels compared to state-of-the-art superpixel methods,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 34, no. 11, pp. 2274 - 2282, May 2012.
[3]D. Martin, C. Fowlkes, D. Tal, and J. Malik, “A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics,” In ICCV, pp.416-423, 2001.
[4]T. Kim and K. Lee, “Learning full pairwise affinities for spectral segmentation,” In CVPR, pp. 2101-2108, 2010.
[5]Z. Li, X. M. Wu, and S. F. Chang, “Segmentation Using Superpixels: A Bipartite Graph Partitioning Approach,” In CVPR, pp. 789-796, 2012.
[6]R. Unnikrishnan, C. Pantofaru, and M. Hebert, “Toward objective evaluation of image segmentation algorithms,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 6, pp. 929–944, 2007.
[7]M. Meilǎ, “Comparing clusterings: an axiomatic view,” In ICML, pp. 577-584, 2005.
[8]J. Freixenet, X. Muñoz, D. Raba, J. Martí, and X. Cufí, “Yet another survey on image segmentation: region and boundary information integration,” In ECCV, pp. 408-422, 2002.
[9]J. Shi and J. Malik, “Normalized cuts and image segmentation.” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888–905, 2000.
[10]D. Comaniciu and P. Meer, “Mean shift: a robust approach toward feature space analysis,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 603–619, 2002.
[11]P. F. Felzenszwalb and D. P. Huttenlocher, “Efficient graph-based image segmentation,” International Journal of Computer Vision, vol. 59, no. 2, pp. 167–181, 2004.
[12]Y. Deng and B. S. Manjunath, “Unsupervised segmentation of color-texture regions in images and video,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 8, pp. 800–810, 2001.
[13]T. Cour, F. Benezit, J. Shi, “Spectral segmentation with multiscale graph decomposition,” In CVPR, pp. 1124-1131, 2005.
[14]J. Wang, Y. Jia, X. S. Hua, C. Zhang, and L. Quan, “Normalized tree partitioning for image segmentation,” In CVPR, pp. 1-8, 2008.
[15]M. Donoser, M. Urschler, M. Hirzer, and H. Bischof, “Saliency driven total variation segmentation,” In ICCV, pp. 817-824, 2009.
[16]S. R. Rao, H. Mobahi, A. Y. Yang, S. S. Sastry, and Y. Ma, “Natural image segmentation with adaptive texture and boundary encoding,” In ACCV, pp. 135-146, 2009.
[17]P. Arbelaez, M. Maire, C. Fowlkes and J. Malik, “From contours to regions: an empirical evaluation,” In CVPR, pp. 2294-2301, 2009.
B.Clustering Techniques
[18]L. Zelnik-Manor and P. Perona, “Self-tuning spectral clustering,” In NIPS, pp. 1601–1608. 2004.
[19]W. Y. Chen, Y. Song, H. Bai, C. J. Lin, and E. Y. Chang, “Parallel spectral clustering in distributed systems,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 3, pp. 568-586, March 2011.
[20]A. Y. Ng, M. I. Jordan, and Y. Weiss, “On spectral clustering: Analysis and an algorithm,” Advances in neural information processing systems, vol. 2, pp. 849–856, 2002.
[21]S. X. Yu and J. Shi, “Multiclass spectral clustering,” In ICCV, pp. 313–319, 2003.
[22]I. Dhillon, “Co-clustering documents and words using bipartite spectral graph partitioning,” In ACM SIGKDD, pp. 269-274, 2001
[23]H. Zha, X. He, C. Ding, H. Simon, and M. Gu, “Bipartitegraph partitioning and data clustering,” In CIKM, pp.25-32, 2001.
C.Saliency Detection
[24]U. Rutishauser, D. Walther, C. Koch, and P. Perona, “Is bottom-up attention useful for object recognition?” in CVPR, pp. II–37, 2004.
[25]L. Itti and C. Koch, “Computational modelling of visual attention,” Nature reviews neuroscience, vol. 2, no. 3, pp. 194–203, 2001.
[26]L. Itti, C. Koch, and E. Niebur, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp. 1254–1259, 1998.
[27]Y. F. Ma and H. J. Zhang, “Contrast-based image attention analysis by using fuzzy growing,” in Proceedings of the eleventh ACM international conference on Multimedia. ACM, pp. 374–381, 2003.
[28]J. Harel, C. Koch, and P. Perona, “Graph-based visual saliency,” In NIPS, pp. 545-552, 2006.
[29]X. Hou and L. Zhang, “Saliency detection: a spectral residual approach,” in CVPR, pp. 1–8, 2007.
[30]R. Achanta, F. Estrada, P. Wils, and S. Süsstrunk, “Salient region detection and segmentation,” in Computer Vision Systems. Springer, pp. 66–75, 2008.
[31]S. Goferman, L. Zelnik-Manor, and A. Tal, “Context-aware saliency detection,” IEEE Trans. PAMI, vol. 34, no. 10, pp. 1915–1926, 2012.
[32]R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, “Frequency-tuned salient region detection,” in CVPR, pp. 1597–1604, 2009.
[33]Y. Zhai and M. Shah, “Visual attention detection in video sequences using spatiotemporal cues,” in Proceedings of the 14th annual ACM international conference on Multimedia. ACM, pp. 815–824, 2006.
[34]M. M. Cheng, G. X. Zhang, N. J. Mitra, X. Huang, and S. M. Hu, “Global contrast based salient region detection,” in CVPR, pp. 409–416, 2011.
[35]H. Jiang, J. Wang, Z. Yuan, T. Liu, N. Zheng, and S. Li, “Automatic salient object segmentation based on context and shape prior,” in BMVC, vol. 3, no. 4, , p. 7, 2011.
[36]F. Perazzi, P. Krahenbuhl, Y. Pritch, and A. Hornung, “Saliency filters: contrast based filtering for salient region detection,” in CVPR, pp. 733–740, 2012.
[37]A. Borji, D. N. Sihite, and L. Itti, “Salient object detection: a benchmark,” in ECCV, pp. 414–429, 2012.
[38]W. Einhäuser and P. König, “Does luminance-contrast contribute to a saliency map for overt visual attention?” European Journal of Neuroscience, vol. 17, no. 5, pp. 1089–1097, 2003.
[39]K. Koffka. Principles of Gestalt Psychology. Routledge & Kegan Paul, 1955
[40]T. Judd, K. Ehinger, F. Durand, and A. Torralba, “Learning to predict where humans look,” In ICCV, pp. 2106–2113, 2009.
[41]J. Reynolds and R. Desimone, “Interacting roles of attention and visual salience in v4,” Neuron, vol. 37, no. 5, pp. 853–863, 2003.
[42]T. Kadir and M. Brady, “Saliency, scale and image description,” International Journal of Computer Vision, vol. 45, no. 2, pp. 83–105, 2001.
[43]T. Liu, Z. Yuan, J. Sun, J. Wang, N. Zheng, X. Tang, and H. Y. Shum, “Learning to detect a salient object,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 2, pp. 353–367, 2011.
D.Computer Vision Applications
[44]J. Xue, C. Li, and N. Zheng, “Proto-object based rate control for jpeg2000: an approach to content-based scalability,” IEEE Trans. Image Processing, vol. 20, no. 4, pp. 1177–1184, 2011.
[45]T. Chen, M. M. Cheng, P. Tan, A. Shamir, and S. M. Hu, “Sketch2photo: internet image montage,” in ACM Transactions on Graphics (TOG), vol. 28, no. 5. ACM, p. 124, 2009.
[46]J. Han, K. N. Ngan, M. Li, and H.-J. Zhang, “Unsupervised extraction of visual attention objects in color images,” IEEE Trans. Circuits and Systems for Video Technology, vol. 16, no. 1, pp. 141–145, 2006.
[47]O. C. Chen and C. C. Chen, “Automatically-determined region of interest in jpeg 2000,” IEEE Trans. Multimedia, vol. 9, no. 7, pp. 1333–1345, 2007.
[48]Y. Boykov and V. Kolmogorov, “An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 9, pp. 1124–1137, 2004.
[49]S. Avidan and A. Shamir, “Seam carving for content-aware image resizing,” ACM Transactions on graphics (TOG), vol. 26, no. 3. ACM, p. 10, 2007.
[50]Y. S. Wang, C. L. Tai, O. Sorkine, and T. Y. Lee, “Optimized scale-and-stretch for image resizing,” ACM Transactions on Graphics (TOG), vol. 27, no. 5, p. 118, 2008.
[51]H. Wu, Y. S. Wang, K. C. Feng, T. T. Wong, T. Y. Lee, and P. A. Heng, “Resizing by symmetry-summarization,” ACM Transactions on Graphics (TOG), vol. 29, no. 6, p. 159, 2010.
E.Mathematic Tools
[52]A. Criminisi, T. Sharp, C. Rother, and P. Pérez, “Geodesic image and video editing,” ACM Transactions on Graphics (TOG), vol. 29, no. 5, p.134, 2010.
[53]A. Adams, J. Baek, and M. A. Davis, “Fast high-dimensional filtering using the permutohedral lattice,” In Computer Graphics Forum, vol. 29, no. 2, pp. 753-762, 2010.
[54]J. Dolson, J. Baek, C. Plagemann, and S. Thrun, “Upsampling range data in dynamic environments,” In CVPR, pp. 1141–1148, 2010.
[55]G. Golub and C. Van Loan. Matrix computations. Johns Hopkins University Press, 1996.