臺灣博碩士論文加值系統

立體視覺是電腦視覺中的一個重要組成部分，也一直是幾十年來研究的熱點。雙目立體視覺實際上是模仿人類視覺獲取深度資訊以及三維場景重建的過程，其應用範圍從機器人導航、工業測量到醫療和軍事等方面，獲取密集的深度圖是本文的研究重心。
到目前為止，遮擋區域，深度不連續區域，弱紋理等一系列問題是獲取精確深度圖的主要障礙。在本文中我們提出了一種新的方法，它結合了色彩，空間和圖像分割等資訊來填充遮擋的無效圖元，並用同樣方式作用於整個圖像像素從而保證深度圖一致性和完整性。我們工作的另一個創新是邊緣恢復機制的引入，用來處理深度不一致的區域，隨後的雙邊濾波和平滑處理進一步提高最終的深度圖的品質。我們在Middlebury dataset平臺上測試了我們的演算法並取得了顯著的效果。我們還將演算法對真實世界的室內和室外的圖像進行測試，結果表明我們的演算法在不同的條件下取得了不錯的深度圖也驗證了演算法的魯棒性。

Stereo vision is an important part of computer vision and has been research hotspot for decades. Binocular stereo vision is actually the process of mimicking human vision to obtain depth information and reconstruct 3D scenes, its application ranges from robot navigation, industrial measurement to medical treatment and military affairs, acquiring dense accurate depth maps is the main concern in our paper.
So far, the technical problems of occlusion regions, depth inconsistent and weak texture are the main obstacles in gaining accurate depth maps. Our paper propose an novel method which combines the elements of color, spatial and image segmentation information to fill the occluded pixels and the same principle is applied to the whole image pixels for consistency and integrity. Another innovation of our work is the introduction of an edge restoration mechanism which performs well in dealing with depth inconsistent region, subsequent bilateral filter and smoothing processing further improve the quality of final depth maps. We test our algorithms on Middlebury dataset and have remarkable results. Test on real-world sources of indoor and outdoor images indicates that our algorithm has good robustness, capable of gaining decent dense depth maps under various conditions.

Chapter 1 Introduction 1
1.1 Motivation 1
1.2 category of stereo algorithms 1
1.2.1 Local stereo matching 1
1.2.2 Global stereo matching 3
Chapter 2 Fundamental and research on binocular stereo correspondence 6
2.1 Basic principles of stereo vision 6
2.1.1 Epipolar Geometry 8
2.1.2 Relation between depth and disparity 10
2.2 Constraints of stereo matching algorithm 11
2.3 Adaptive Support-Weight Approach 14
2.4 Cross-Based Local Stereo Matching 16
2.4.1 Cross-Based Local Support Region Construction 16
2.4.2 Fast Cost Aggregation 19
Chapter 3 Stereo Matching 21
3.1 Combined matching cost 21
3.2 Cross-based Cost Aggregation 25
3.3 Image segmentation based on MRF 26
Chapter 4 Post processing Method 31
4.1 Novel occlusion filling method 31
4.2 Edge restoration based on canny detection 36
4.2.1 Canny detection 37
4.2.2 Edge restoration 39
Chapter 5 Experiment Results 45
5.1 Experimental Parameters 45
5.2 Middlebury Dataset 46
5.3 Real-world explorations 48
5.3.1 Experiment Setup 48
5.3.2 Results 49
5.3.3 Discussion 52
Chapter 6 Future works and Conclusion 53
6.1 Conclusion 53
6.2 Future work 54
Reference 56




List of Figures

Figure 1: schematic drawing of Epipolar geometry.. 9
Figure 2: Standard stereo vision system setup. 10
Figure 3: diagram of conversion of depth and disparity. 11
Figure 4: the schematic drawings of sequential consistency constraint. 14
Figure 5: (a) Cross skeleton built for every anchor pixel. (b)Adaptive-shape support region constructed for every anchor pixel. (c)Samples of Support regions, being akin to local image structures appropriately. 16
Figure 6 : Representation of a local upright cross for pixel p.. 18
Figure 7 : framework of the proposed symmetric stereo matching method. 19
Figure 8 : schematic diagram of census transform. 22
Figure 9: original image and image processed by census transform(3×3). 23
Figure 10 : Teddy. 24
Figure 11 : (a) close-up of appointed region (b) results of AD (c) results of AD-census. 25
Figure 12 : (a) first order neighboring system, also called 4-neighborhood system. (b) second order system, also called 8-neigborhood system. 28
Figure 13 : original image from Middelbury dataset. 29
Figure 14 : (a) segmentation image. (b)segmentation image after filtering.. 30
Figure 15: (a) and (b) are left and right images of Teddy from Middelbury dataset.. 32
Figure 16 : Schematic diagram of occlusion problem. 32
Figure 17 : (a) initial left depth map. (b) initial right depth map. (c)occlusion map of Teddy. (d) ground truth. 34
Figure 18 : (a) disparity without occlusion filling (b) after occlusion filling. 36
Figure 19 : (a) Input image. (b) Outcome by means of canny detection. 39
Figure 20 : The flow chart of Our proposed edge restoration mechanism. 40
Figure 21 : representation of match edge and mismatch edge, comparison of results before and after edge restoration. 41
Figure 22 : Comparison of disparity map before and after edge restoration mechanism. 44
Figure 23 : The results of Middlebury dataset.. 48
Figure 24 : Outdoor image and results.. 50
Figure 25 : Indoor image and depth map.. 52








List of Tables

Table 1 : Edge restoration process. 43
Table 2 : Parameters of Stereo Camera. 49

[1] Azuma, Ronald, et al. "Recent advances in augmented reality." Computer Graphics and Applications, IEEE 21.6 (2001): 34-47.
[2] Koren, Yoram, and Johann Borenstein. "Potential field methods and their inherent limitations for mobile robot navigation." Robotics and Automation, 1991. Proceedings., 1991 IEEE International Conference on. IEEE, 1991.
[3] Rheingold, Howard. Virtual Reality: Exploring the Brave New Technologies. Simon & Schuster Adult Publishing Group, 1991.
[4] Yang, Yibing, Alan Yuille, and Jie Lu. "Local, global, and multilevel stereo matching." Computer Vision and Pattern Recognition, 1993. Proceedings CVPR''93., 1993 IEEE Computer Society Conference on. IEEE, 1993.
[5] Scharstein, Daniel, and Richard Szeliski. "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms." International journal of computer vision 47.1-3 (2002): 7-42.
[6] Hirschmüller, Heiko, and Daniel Scharstein. "Evaluation of stereo matching costs on images with radiometric differences." Pattern Analysis and Machine Intelligence, IEEE Transactions on 31.9 (2009): 1582-1599.
[7] Schmidt, A., Kraft, M., & Kasiński, A. (2010). An evaluation of image feature detectors and descriptors for robot navigation. In Computer Vision and Graphics (pp. 251-259). Springer Berlin Heidelberg.
[8] Giachetti, Andrea. "Matching techniques to compute image motion." Image and Vision Computing 18.3 (2000): 247-260.
[9] Hannah, Marsha J. Computer matching of areas in stereo images. No. STAN-CS-74-438. STANFORD UNIV CA DEPT OF COMPUTER SCIENCE, 1974.
[10] Yoon, Kuk-Jin, and In So Kweon. "Adaptive support-weight approach for correspondence search." IEEE Transactions on Pattern Analysis & Machine Intelligence 4 (2006): 650-656.
[11] Tombari, Federico, Stefano Mattoccia, and Luigi Di Stefano. "Segmentation-based adaptive support for accurate stereo correspondence." Advances in Image and Video Technology. Springer Berlin Heidelberg, 2007. 427-438.
[12] Yang, Qingxiong. "Recursive bilateral filtering." Computer Vision–ECCV 2012. Springer Berlin Heidelberg, 2012. 399-413.
[13] Yang, Qingxiong, et al. "Real-time Global Stereo Matching Using Hierarchical Belief Propagation." BMVC. Vol. 6. 2006.
[14] Geiger, Andreas, Martin Roser, and Raquel Urtasun. "Efficient large-scale stereo matching." Computer Vision–ACCV 2010. Springer Berlin Heidelberg, 2010. 25-38.
[15] Hirschmüller, Heiko. "Accurate and efficient stereo processing by semi-global matching and mutual information." Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on. Vol. 2. IEEE, 2005.
[16] Woodford, Oliver, et al. "Global stereo reconstruction under second-order smoothness priors." Pattern Analysis and Machine Intelligence, IEEE Transactions on 31.12 (2009): 2115-2128.
[17] Boykov, Yuri, Olga Veksler, and Ramin Zabih. "Fast approximate energy minimization via graph cuts." Pattern Analysis and Machine Intelligence, IEEE Transactions on 23.11 (2001): 1222-1239.
[18] Lempitsky, Victor, Carsten Rother, and Andrew Blake. "Logcut-efficient graph cut optimization for markov random fields." Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on. IEEE, 2007.
[19] Zhao, Ming, Xiao-bai Li, and Rong-ling Lang. "Improved adaptive stereo matching using double dynamic programming with correlation of row and column." Signal Processing Systems (ICSPS), 2010 2nd International Conference on. Vol. 3. IEEE, 2010.
[20] Xiao, Jun, Linyuan Xia, and Liqun Lin. "Segment-based stereo matching using edge dynamic programming." Image and Signal Processing (CISP), 2010 3rd International Congress on. Vol. 4. IEEE, 2010.
[21] Sun, Jian, Nan-Ning Zheng, and Heung-Yeung Shum. "Stereo matching using belief propagation." Pattern Analysis and Machine Intelligence, IEEE Transactions on 25.7 (2003): 787-800.
[22] Felzenszwalb, Pedro F., and Daniel P. Huttenlocher. "Efficient belief propagation for early vision." International journal of computer vision 70.1 (2006): 41-54.
[23] Yang, Qingxiong, Liang Wang, and Narendra Ahuja. "A constant-space belief propagation algorithm for stereo matching." Computer vision and pattern recognition (CVPR), 2010 IEEE Conference on. IEEE, 2010.
[24] Cochran, Steven D., and Gérard Medioni. "3-D surface description from binocular stereo." IEEE Transactions on Pattern Analysis & Machine Intelligence 10 (1992): 981-994.
[25] Kanade, Takeo, and Masatoshi Okutomi. "A stereo matching algorithm with an adaptive window: Theory and experiment." Pattern Analysis and Machine Intelligence, IEEE Transactions on 16.9 (1994): 920-932.
[26] Jain, Ramesh, Rangachar Kasturi, and Brian G. Schunck. Machine vision. Vol. 5. New York: McGraw-Hill, 1995.
[27] Svoboda, Tomáš, Tomáš Pajdla, and Václav Hlaváč. "Epipolar geometry for panoramic cameras." Computer Vision—ECCV''98. Springer Berlin Heidelberg, 1998. 218-231.
[28] Bolles, Robert C., H. Harlyn Baker, and David H. Marimont. "Epipolar-plane image analysis: An approach to determining structure from motion."International Journal of Computer Vision 1.1 (1987): 7-55.
[29] Yoshida, Sota R. Computer Vision. Hauppauge, N.Y.: Nova Science, 2011. Print.
[30] Solina, Franc and Alešs Leonardis. Computer Analysis Of Images And Patterns. Berlin: Springer-Verlag Berlin Heidelberg, 1999. Print.
[31] Zhang, Ke, Jiangbo Lu, and Gauthier Lafruit. "Cross-based local stereo matching using orthogonal integral images." Circuits and Systems for Video Technology, IEEE Transactions on 19.7 (2009): 1073-1079.
[32] Mei, Xing, et al. "On building an accurate stereo matching system on graphics hardware." Computer Vision Workshops (ICCV Workshops), 2011 IEEE International Conference on. IEEE, 2011.
[33] Zabih, Ramin, and John Woodfill. "Non-parametric local transforms for computing visual correspondence." Computer Vision—ECCV''94. Springer Berlin Heidelberg, 1994. 151-158.
[34] Forney, G. David. "Generalized minimum distance decoding." Information Theory, IEEE Transactions on 12.2 (1966): 125-131.
[35] Hosni, Asmaa, et al. "Local stereo matching using geodesic support weights."Image Processing (ICIP), 2009 16th IEEE International Conference on. IEEE, 2009.
[36] Zhang, Yongyue, Michael Brady, and Stephen Smith. "Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm." Medical Imaging, IEEE Transactions on 20.1 (2001): 45-57.
[37] Geman, Stuart, and Donald Geman. "Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images." Pattern Analysis and Machine Intelligence, IEEE Transactions on 6 (1984): 721-741.
[38] Permuter, Haim, Joseph Francos, and Ian Jermyn. "A study of Gaussian mixture models of color and texture features for image classification and segmentation." Pattern Recognition 39.4 (2006): 695-706.
[39] Wu, Fa-Yueh. "The potts model." Reviews of modern physics 54.1 (1982): 235.
[40] Zitnick, C. Lawrence, and Takeo Kanade. "A cooperative algorithm for stereo matching and occlusion detection." Pattern Analysis and Machine Intelligence, IEEE Transactions on 22.7 (2000): 675-684.
[41] Fusiello, Andrea, Vito Roberto, and Emanuele Trucco. "Efficient stereo with multiple windowing." cvpr. IEEE, 1997.
[42] Canny, John. "A computational approach to edge detection." Pattern Analysis and Machine Intelligence, IEEE Transactions on 6 (1986): 679-698.
[43] Tomasi, Carlo, and Roberto Manduchi. "Bilateral filtering for gray and color images." Computer Vision, 1998. Sixth International Conference on. IEEE, 1998.
[44]D. Scharstein and R. Szeliski, “Middlebury Stereo Website[Online]”, Available: http://vision.middlebury.ed/stereo/
[45] Fusiello, Andrea, and Luca Irsara. "Quasi-Euclidean epipolar rectification of uncalibrated images." Machine Vision and Applications 22.4 (2011): 663-670.
[46] Mukherjee, Dibyendu, Guanghui Wang, and QM Jonathan Wu. "Stereo matching algorithm based on curvelet decomposition and modified support weights." Acoustics Speech and Signal Processing (ICASSP), 2010 IEEE International Conference on. IEEE, 2010.
[47] Graham, Ronald L., and Pavol Hell. "On the history of the minimum spanning tree problem." Annals of the History of Computing 7.1 (1985): 43-57.
[48] Grama, Ananth, Anshul Gupta, and George Karypis. Introduction to parallel computing: design and analysis of algorithms. Vol. 400. Redwood City, CA: Benjamin/Cummings, 1994.
[49] Harish, Pawan, and P. J. Narayanan. "Accelerating large graph algorithms on the GPU using CUDA." High performance computing–HiPC 2007. Springer Berlin Heidelberg, 2007. 197-208.