臺灣博碩士論文加值系統

近年來，為了能提供觀賞者一個更身歷其境的觀賞效果，立體顯示技術的應用也在生活中越來越越普及。其中最被廣泛討論以及研究的主題為基於深度圖像繪圖法(DIBR)，使用某視角的彩色影像與其對應之深度圖，來合成出其他視角的影像，再透過各種顯示器撥放。
此方法會根據像素點的深度值來計算出一個位移量，將圖片中各個像素位移已產生新的視角影像，但同時也會將原本無法看見的地方給顯露出來,我們稱之為空洞。一般來說會先對深度圖進行平滑處理以來減少空洞的大小方便填補，但是此法耗時，由於以上缺點本論文採用一個新的優化機制來取代傳統的平滑化處理，已便將前景和背景正確分離，而水平位移階段加入了新的裂痕空洞填補機制希望藉此達到省時的效果，在最後一級的補洞法採用本篇論文提出的多向性補洞法有別於傳統補洞法僅僅參考水平方向，更參考了垂直以及鄰近八個像素質的資訊。最後實驗的結果顯現出本篇論文所提出的方法在可操作在177MHz頻率下且能相對於其他現行的方法亦擁有較佳的影像品質。

In order to offer immersive view experience for viewer, 3D stereoscopic technology applications are more popular. One of the most widely discussed topics is (depth image-based rendering) DIBR. The DIBR system renders virtual views at different viewpoints using 3D warping process.
The DIBR calculates disparity value by depth value, and then moves the pixel of an object to generate new viewpoint image. During a 3D warping process, some parts of an occluded background region might be visible in a synthesized virtual image, i.e., a disoccluded region. It is referred to as a “hole”. The conventional way to solve this problem is using a smoothing filter to smooth depth map to reduce hole’s size before shifting, but it’s time-consuming. This thesis discards the traditional smoothing depth approach and proposes a novel depth map refinement scheme to remove blur pixels near the object boundary caused by restriction of camera and make the holes sit on the correct position. With the blur pixels removal, the hole-filling will become more reliable. The holes after warping process are filled by the proposed multi-direction hole-filling algorithms based on direction of edge. It can preserve the texture of background. The experimental results show that the visual quality of our proposal is competitive to others contemporary methods.

摘 要 I
Abstract II
誌 謝 III
List of Figures VI
List of Tables VIII



Chapter 1 Introduction 1
1.1 Motivation 1
1.2 Organization of the Thesis 2

Chapter 2 Background 3
2.1 Depth Image-based Rendering System 3
2.1.1 Pre-processing 4
2.1.2 Warping 5
2.1.3 Hole-filling 11
2.2 Smooth-depth-based Depth Image-based Rendering 12
2.2.1 Symmetric and asymmetric Gaussian smoothing filter 14
2.2.2 Parallax-map-based DIBR 15
2.2.3 Adaptive edge-oriented smoothing filter 15



Chapter 3 The proposed multi-view system 18
3.1 Overview of proposed multi-view system 18
3.2 Proposed M1 20
3.2.1 Boundary detection and refinement 20
3.2.2 Hybrid warping and crack hole-filling 24
3.2.3 Multi-direction hole-filling 30
3.3 Proposed M2 38

Chapter 4 Hardware implements 40
4.1 Overview of hardware architecture 40
4.2 Hardware architecture of the proposed method_1 43
4.3 Hardware architecture of the proposed method_2 46

Chapter 5 Experimental results 47
5.1 Objective quantitative results and subjective results 50
5.2 Hardware synthesis 63

Chapter 6 Conclusions 65
Reference 66

[1]C. Fehn, K. Hopf, and B. Quante, Key technologies for an advanced 3D TV system, in Proc. SPIE Three-Dimensional TV, Video, and Display III, pp. 66-80, Oct. 2004,
[2]C. Fehn, “Depth-image-based-rendering (DIBR), compression and transmission for a new approach on 3D-TV, in Proc. International Society for Optical Engineering Conf. Stereoscopic Displays and Virtual Reality Systems XI, San Jose, CA, vol.5291, pp.93-104, Jan. 2004,.
[3]K. J. Yoon and S. Kweon, “Adaptive support-weight approach for correspondence search IEEE Trans. Pattern Anal. Mach. Intell., vol.28, pp.650-656, Apr. 2006,.
[4]M. C. Yang, J. Y. Liu, Y. C. Yang, and K. H. Chen, “A qualty-improved stereo matching by using incrementally-averaging orthogonally-scanned edge detection, in Proc. Int. Conf. 3DSA, pp. 489-491, June 2012.
[5]K. Zhang, J. Lu, and G. Lafruit, “Cross-based local stereo matching using orthogonal integral images, IEEE Trans. Circuits Syst. Video Technol., vol. 19, pp. 1073-1079, July 2009.
[6]D. Kim, D. Min, and K. Sohn, “A stereoscopic video generation method using stereoscopic display characterization and motion analysis, IEEE Trans. Broadcast., vol. 54, pp. 188-197, June 2008.
[7]J. Lee, S. Yoo, M. Kang, C. Chun, and C. Kim, “Depth estimation from a single image using salient objects detection, in Proc. Int. Conf. 3DSA, pp. 264-267, June 2012.
[8]M. T. Pourazad, P. Nasiopoulos, and R. K. Ward, “An H.264-based scheme for 2D to 3D video conversion, IEEE Trans. Consum. Electron., vol. 55, pp. 742-748, May 2009.
[9]Y. Feng, J. Ren, and J. Jiang, “Object-based 2D-to-3D video conversion for effective stereoscopic content generation in 3D-TV applications, IEEE Trans. Broadcast., vol. 57, pp. 500-509, June 2011.
[10]Y. C. Fan and T. C. Chi, “The novel non-hole-filling approach of depth image based rendering, in Proc. IEEE 3DTV, pp. 325-328, May 2008.
[11]C. Fehn, “A 3D-TV approach using depth-image-based rendering (DIBR), in Proc. Vis., Imaging, and Image Process., pp. 482-487, Sept. 2003.
[12]A. Redert, M. O. de Beeck, C. Fehn, W. Ijsselsteijn, M. Pollefeys, L. V. Gool, E. Ofek, I. Sexton, and P. Surman, “ATTEST-advanced three-dimensional television system technologies, in Proc. IEEE 3DPVT, pp. 313-319, Jan. 2002.
[13]W. J. Tam, G. Alain, L. Zhang, T. Martin, and R. Renaud, “Smoothing depth maps for improved stereoscopic image quality, in Proc. SPIE Three-Dimensional TV, Video, and Display Ⅲ, pp. 162-172, Oct. 2004.
[14]A. Woods, T. Docherty, and R. Koch, “Image distortions in stereoscopic video systems, in Proc. SPIE Stereoscopic Displays and Appl., pp. 36-48, Feb. 1993.
[15]W. A. IJsselsteijn, H. de Ridder, and J. Vliegen, “Effects of stereoscopic filming parameters and display duration on the subjective assessment of eye strain, in Proc. SPIE Stereoscopic Displays and Virtual Reality Syst., pp. 12-22, Apr. 2000.
[16]I. Daribo and H. Saito, “A novel inpainting-based layered depth video for 3DTV, IEEE Trans. Broadcast., vol. 57, pp. 533-541, June 2011.
[17]P. Ndjiki-Nya, M. Koppel, D. Doshkov, H. Lakshman, P. Merkle, K. Muller, and T. Wiegand, “Depth image-based rendering with advanced texture synthesis for 3-D video, IEEE Trans. Multimedia, vol. 13, pp. 453-465, June 2011.
[18]M. Solh and G. AlRegib, “Hierarchical hole-filling (HHF): depth image based rendering without depth map filtering for 3D-TV, in Proc. IEEE MMSP, pp. 87-92, Oct. 2010.
[19]M. S. Ko, D. W. Kim, D. L. Jones, J. Yoo, “A new common-hole filling algorithm for arbitrary view synthesis, in Proc. Int. Conf. 3DSA, pp. 242-245, June 2012.
[20]A. Criminisi, P. Perez, and K. Toyama, “Region filling and object removal by exemplar-based image inpainting, IEEE Trans. Image Process., vol. 13, pp. 1200-1212, Sept. 2004.
[21]L. Zhang and W. J. Tam, “Stereoscopic image generation based on depth images for 3DTV, IEEE Trans. Broadcast., vol. 51, pp. 191-199, June 2005.
[22]W. Y. Chen, Y. L. Chang, S. F. Lin, L. F. Ding, and L. G. Chen, “Efficient depth image based rendering with edge dependent depth filter and interpolation, in Proc. IEEE ICME, pp. 1314-1317, July 2005.
[23]T. C. Lin, H. C. Huang, and Y. M. Huang, “Preserving depth resolution of synthesized images using parallax-map-based DIBR for 3D-TV, IEEE Trans. Consum. Electron., vol. 56, pp. 720-727, May 2010.
[24]H. Dong, S. Jianfei, and X. Ping, “Improvement of virtual view rendering based on depth image, in Proc. IEEE ICIG, pp. 254-257, Aug. 2011.
[25]P. J. Lee and Effendi, “Nongeometric distortion smoothing approach for depth map preprocessing, IEEE Trans. Multimedia, vol. 13, pp. 246-254, Apr. 2011.
[26]L. H. Wang, X. J. Huang, M. Xi, D. X. Li, and M. Zhang, “An asymmetric edge adaptive filter for depth generation and hole filling in 3DTV, IEEE Trans. Broadcast., vol. 56, pp. 425-431, Sept. 2010.
[27]T. C. Yang, P. C. Kuo, B. D. Liu and J. F. Yang, “Depth Image-Based Rendering with Edge-oriented Hole Filling for Multiview Synthesis, in Proc. IEEE ICCCAS, pp. 50-53, Nov 2013.
[28]B. E. Bayer, “Color imaging array, U.S. Patent 3 971 065, July 20, 1976.
[29]W.-Y. Lee, “Improved Hole-filling Algorithms and Hardware Implementation for Virtual View Generation, Master’s Thesis, Dept. of Electrical Engineering, National Tsing Hua University, July 2011.
[30]S. J. Yao, P. F. Jin, H. Fu, L. H. Wang, D. X. Li, M. Zhang, “Real-Time 3DTV System for Autostereoscopic Displays, in Proc. IEEE ICALIP, pp. 621-626, July 2012
[31]H.-J. Chen, F.-H Lo, F.-C. Jan, and S.-D. Wu,“Real-Time Multiview Rendering Architecture for Autostereoscopic Displays, in Proc. IEEE Intl, Symp. Circuits and Systems (ISCAS), pp. 1165-1168, May 2010.