臺灣博碩士論文加值系統

隨著近年來3D電影的風行，如阿凡達與玩具總動員3等電影的推出，3D已在全球掀起一股熱潮，傳統的3D影像合成需要經過多支攝影機進行拍攝，不但在拍攝成本上花費昂貴，而且需要大量的頻寬，Depth Image Based Rendering (DIBR)這項技術解決了有線頻寬的問題，但是DIBR仍然存在一些技術上的問題需要被解決。
在DIBR中，影像合成左眼圖與右眼圖時會產生未填補區域(disocclusion area) ，若這些區域未能被妥善填補將會造成影像失真，因此DIBR會將深度圖(Depth Map)做特定的處理(pre-processing)，讓合成後的影像減少失真。目前大多數的方式是將深度圖做平滑處理，但這樣的方式可能造成幾何失真而且不易硬體化，在論文中將會提出新的前處理演算法來改善幾何失真的問題，並且減少運算複雜度利於硬體實現。

The 3D movies become increasingly popular, such as the film of Avatar and Toy Story 3. Traditional 3D image requires multiple cameras. This way not only costs a lot but also requires a lot of bandwidth. Depth-Image-Based Rendering (DIBR) solves the bandwidth problem, but DIBR still exists some technical issues that need to be resolved.

Synthesizing the virtual views by DIBR will generate disocclusion area. If these areas are not properly filled, it results in image distortion. Therefore, DIBR does some specific processing on depth map (pre-processing), and pre-processing reduces distortion of the synthesized images. Traditional way of the pre-processing step smoothen the depth map, but this may result in geometric distortion and difficult on hardware implement. This paper will propose a new pre-processing algorithm to improve the problem of geometric distortion and reduce the computational complexity degree conducive to hardware implementation.

Table of Contents
Abstract
Table of Content
List of Figures
List of Tables

Chapter1 Introduction 1
1.1 Overview of 3D image Generation 1
1.2 Motivation 3
1.3 Chapter Overview 4

Chapter 2 Related Work 5
2.1 Overview of DIBR 5
2.2 Previous work 12
2.2.1 Symmetric Gaussian Low-pass Filter 13
2.2.2 Asymmetric Gaussian Low-pass Filter 14
2.2.3 Edge Dependent Low-pass Filter 15
2.2.4 Gradient Direction-Based Smoothing 16
2.3 Drawbacks of SDB-DIBR 17
2.3.1 Time-consuming 17
2.3.2 Attenuation of Depth-resolution 17
2.3.3 Non-adaptive Parameter 18
2.3.4 Hardware is difficult to implement 18

Chapter 3 Parallax-Map-Based DIBR Algorithm 19
3.1 New Way of Thinking 19
3.2 Investigation and Analysis Parallax-Map 20
3.2.1 Disocclusion Area 21
3.2.2 Smoothing Strength 24
3.2.3 Attenuation of Depth Resolution 26
3.3 Parallax-Map-Based DIBR 27
3.3.1 Functional Blocks 27
3.3.2 Parallax-map Descent 28

Chapter 4 Proposed Video Transform System Architecture Design 35
4.1 Design Ideas 35
4.1.1 Top Level System 35
4.1.2 Controller Design 37
4.2 Proposed Data Path 39
4.2.1 Data Storage 39
4.2.2 Functional Determination 40
4.2.3 PV-rewrite 42
4.2.4 Double SRAM Design 44
4.3 Pipeline Architecture 44

Chapter 5 Experimental Results 47
5.1 The Result of Software Simulation 47
5.2 Synthesis Results 51
5.2.1 Timing Report with tsmc 0.18um Library 51
5.2.2 Area Report with tsmc 0.18um Library 54
5.2.3 Total Report with tsmc 0.18um Library 55
Chapter 6 Conclusion 56
References 57


LIST OF FIGURES
Fig.1.1 The popular 3D-TV 2
Fig.1.2 Color image and its corresponding depth image 3
Fig.2.1 Three video sequences and their corresponding depth maps 6
Fig.2.2 Z-camera 7
Fig.2.3. Block Diagram of DIBR System 9
Fig.2.4 The actual depth map 11
Fig.2.5 (a) Virtual right image with holes colored in green. (b) Rubber-sheet artifacts by hole-filling with average filter. 13
Fig.2.6 (a) Depth map preprocessed by a symmetric smoothing filter with = =30. (b) Virtual left view. (c) and (d) Enlargement of image (b). 14
Fig.2.7 Depth map preprocessed by an asymmetric smoothing filter with =10 and =90. (b) Virtual left view. (c) and (d) Enlargement of image (b). 15
Fig.2.8 (a) Depth map preprocessed by edge dependent depth filter with = =5. (b) Virtual left view. (c) and (d) Enlargement of image (b). 15
Fig.2.9 (a) Depth map is preprocessed by adaptive smoothing method with number of 50 iterations for each 1st and 2nd smoothing filter. (b) Virtual left view. (c) and (d) Enlargement of image (b). 16
Fig 3.1 Schematic of the x-axis 21
Fig 3.2 The associated Parallax coordinates 22
Fig 3.3 The synthesized right-view row without hole-filling 23
Fig 3.4 Different degrees of smoothing strength (a) Unit parallax-value difference (b) Larger parallax-value difference (c) Unit parallax-value difference but larger transition of parallax-value descent 25
Fig 3.5 Attenuation of depth resolution (a) Both data filled in the disocclusion area (b) Less data filled in the disocclusion area (c) Soiled depth information of background object 26
Fig 3.6 Functional block of Parallax-Map-Based DIBR 27
Fig 3.7 The Parallax-Map Descent Case One (diff/dd = 1) (a) A row of a captured image (b) The parallax-map after parallax-map descent (c) The synthesize row with no attenuation of relative distance 30
Fig 3.8 The Parallax-Map Descent Case Two (diff/dd 〉 1) (a) A row of a captured image
(b) The associated parallax-map after Parallax-Map Descent 31
Fig 3.9 The synthesized row with larger holes but no attenuation of relative distance 33
Fig 3.10 Flow chart of Parallax-Map Descent 34
Fig 4.1 Top level system 36
Fig 4.2 Controller flow 37
Fig 4.3 Design of Data Storage 39
Fig 4.4 Design of Function Determination 41
Fig 4.5 Design of PV-rewrite 43
Fig.4.6 Design of Double SRAM 44
Fig.4.7 Design of Pipelined Architecture 45
Fig.4.8 Pipelined Architecture in VTS 46
Fig.5.1 Gray-level image without pre-processing 48
Fig.5.2 Color image without pre-processing 48
Fig.5.3 Gray-level image with Smooth-Depth-Image-Based DIBR 49
Fig.5.4 Color image with Smooth-Depth-Image-Based DIBR 49
Fig.5.5 Gray-level image with Parallax-Map-Based DIBR 50
Fig.5.6 Color image with Parallax-Map-Based DIBR 50
Fig.5.7 Synthesis architecture of VTS 55
LIST OF TABLES
Table 2.1 Comparison between two methods 8
Table 5.2 Information with tsmc 0.18um Library 55

[1] DISPLAYSEARCH, “3D Display Technology and Market Forecast Report, 2010.
[2] G. Hamagishi, M. Sakata, K. Mashitani, M. Inoue, K. Taima, K. Oyamada, and
Sj. Kishimoto, “A Display System with 2-D/3-D Compatibility, SID98 Digest,
pp. 915-918, 1998.
[3] R.-P.M. Berretty, F. J. Peters and G. T. G. Volleberg, “Real Time Rendering for Multiview Autostereoscopic Displays, Procs. Stereoscopic Displays and Applications Conference, SPIE, vol. 6055, p. 208-219, 2006.
[4] C. Fehn, “Depth-image-based-rendering (DIBR), compression and transmission for a new approach on 3D-TV, in Proc. International Society for Optical Engineering
Conf. Stereoscopic Displays and Virtual Reality Systems XI, San Jose, CA, Jan. 2004, vol. 5291, pp. 93–104.
[5] M. Tanimoto, “FTV (Free viewpoint Television) for 3D scene repro-duction and creation, in Proc. Conf. Computer Vision and PatternRecognition Workshop, 2006,
p.172.
[6] G. J. Iddan and G. Yahav, “3D imaging in the studio and elsewhere, in Proc. SPIE Videometrics and Optical Methods for 3D Shape Measurements, San Jose, CA, Jan. 2001, pp. 48–55.
[7] Y. L. Chang, C. Y. Chen, S. F. Lin, and L. G. Chen, “Depth map generation for 2D-to-3D conversion by short-term motion assisted color segmentation, in Proc. Int. Conf. Multimedia and Expo, 2007, pp.1958–1961..
[8] K. C. Liu, Q. Wu, W. C. Chen, C. F. Wu, F. C. Jan, and T. Chen, “Example-based depth generation from single image for 3D content, in Proc. 3DTV Conf.: The True Vision—Capture, Transmission and Display of 3D Video, May 2008, pp. 333–336.
[9] Y. C. Fan and T. C. Chi, “The novel non-hole-filling approach of depth image based rendering, 3DTV, pp. 325–328, May 2008.
[10] C. Fehn, “A 3D-TV approach using depth-image-based-rendering (DIBR), in Proc. Visualization, Imaging, and Image Processing, Benalmadena, Spain, Sep. 2003, pp. 482–487.
[11] A. Woods, T. Docherty, and R. Koch, “Image distortions in stereoscopic video systems, in Proc. SPIE Stereoscopic Displays and Applications, San Jose, CA, Feb. 1993, pp. 36–48.
[12] W. A. IJsselsteijn, H. de Ridder, and J. Vliegen, “Stereoscopic filming parameters and display duration on the subjective assessment of eye strain, in Proc. International Society for Optical Engineering Conf. Stereoscopic Displays and Virtual Reality Systems, San Jose, CA, Apr. 2000, pp. 12–22.
[13] I. P. Howard and B. J. Rogers, Binocular Vision and Stereopsis. New York: Oxford University Press, 1995.
[14] W. J. Tam, G. Alain, L. Zhang, T. Martin, and R. Renaud, “Smoothing depth maps for improved stereoscopic image quality, in Proc. International Society for Optical Engineering Conf. Three-Dimensional TV, Video, and Display III, Philadelphia, USA, Oct. 2004, vol. 5599, pp. 162–172.
[15] L. Zhang and W. J. Tam, “Stereoscopic image generation based on depth images for 3DTV, IEEE Trans. Broadcast., vol. 51, pp. 191–199, Jun. 2005.
[16] W. Y. Chen, Y. L. Chang, S. F. Lin, L. F. Ding, and L. G. Chen, “Efficient depth image based rendering with edge dependent depth filter and interpolation, in Proc. IEEE Int. Conf. Multimedia and Expo, Jul. 2005, pp. 1314–1317.
[17] Y. K. Park, K. Jung, Y. Oh, S. Lee, J. K. Kim, G. Lee, H. Lee, K. Yun, N. Hur, and J. Kim, “Depth-image-based rendering for 3DTV service over T-DMB, Signal Process.: Image Commun., vol. 24, pp. 122–136, Jan. 2009.
[18] Ting-Ching Lin, Hsien-Chao Huang, and Yueh-Min Huang, “Preserving Depth Resolution of Synthesized Images Using Parallax-Map-Based DIBR for 3D-TV IEEE Transactions on Consumer Electronics, Vol. 56, No. 2, May 2010
[19] Pei-Jun Lee, Member, IEEE, and Effendi, “Nongeometric Distortion Smoothing Approach for Depth Map Preprocessing IEEE Transactions on multimedia, Vol. 13, NO. 2, APRIL 2011