臺灣博碩士論文加值系統

隨著立體影像技術的盛行，自由視點視訊(FTV, Free Viewpoint Television)成為熱門的研究主題。任意視點合成為一關鍵技術，其中視點合成的即時輸出和複雜度降低為重要技術議題。由於NVIDIA 公司提出之Compute Unified Device Architecture (CUDA)平台能有效的處理資料密集型的應用程式，因此我們將MPEG提供的視點合成參考軟體 (View Synthesis Reference Software, VSRS)[2]移植至此架構上。為了在CUDA平台上實現VSRS，必須加強VSRS的平行度。在實作過程中，我們兼顧平行度與影像品質。首先，我們提出內部空洞填補(Intra Hole Filling)方法，取代原本中值濾波器的方法。此外，我們為了減少資料的相依性，透過CUDA 執行緒對資料做適宜的分割，並且重新安排資料執行順序，以減少分支指令。綜合上述技巧，我們節省超過94％的計算時間，並且得到類似的影像品質。

With the prosperity of 3D technology, Free Viewpoint Television (FTV) becomes a popular research topic. “View Synthesis” is a key step in FTV. There are some important and to-be-solved issues such as real-time operation and complexity reduction. NVIDIA Compute Unified Device Architecture (CUDA) is an effective platform in handling data-intensive applications. To implement the MPEG view synthesis reference software (VSRS) on CUDA, we parallelize the VSRS structure. In the meanwhile, our proposed parallel scheme improves the picture quality. We first propose an intra hole filling scheme to replace the original median filter. Then, to avoid data dependence we properly partition the data so that they can be processed by the parallel GPU threads. Also, we rearrange the data processing order in the threads to reduce branching instructions. Combining these techniques together, we save more than 94% computing time and achieve a similar image quality.

Chapter 1 緒論 1
1.1 研究背景 1
1.2 研究動機及貢獻 2
1.3 論文大綱 3
Chapter 2 CUDA的概述 5
2.1 使用GPU進行通用運算 5
2.1.1 GPU與CPU 5
2.1.2 CUDA的簡介 7
2.2 CUDA應用程式編譯過程 7
2.3 CUDA硬體架構 10
2.4 SIMT編譯模型 12
2.5 CUDA程式設計範例 13
Chapter 3 簡介MPEG 3D Video Coding 參考軟體 15
3.1 MPEG 3D Video Coding 概述 15
3.2 極平面圖像概述 16
3.2.1 立體影像系統 16
3.2.2 極平面圖像和三維空間的關係 17
3.3 景深估測參考軟體概述 19
3.4 視點合成演算法概述 21
3.4.1 視點合成軟體概述 21
3.4.2 VSRS演算法步驟 22
3.5 視點合成參考軟體介紹 25
3.5.1 軟體平台概述 25
3.5.2 品質測試指標 27
3.5.3 MPEG 測試影像介紹 28
3.5.4 測試影片在VSRS的效能測試 29
Chapter 4 景深修正(Depth Refinement) 31
4.1 景深修正演算法概述 31
4.1.1 景深圖上符號的說明 31
4.1.2 前景填補洞(Forward Hole Filling ) 32
4.1.3 背景填補洞(Background Hole Filling ) 34
4.1.4 不同的填補方法在測試影像上的分析 36
4.1.4.1 前景補償在真實影像數值分析 37
4.1.4.2 背景補償在真實影像數值分析 41
4.1.5 分析與討論 46
4.2 基於景深修正視點合成演算法設計 47
4.2.1 效能的比較 49
4.2.2 不同景深圖修補方式對合成結果的表現 51
Chapter 5 視點合成平行化設計 61
5.1 視點合成平行化概述 61
5.2 資料相依的問題 62
5.2.1 景深映射上資料的相依性 63
5.2.2 紋理映射上資料的相依性 64
5.2.3 資料相依性在真實影像上的影響 65
5.3 利用執行緒的安排解決競逐的問題 68
5.4 景深映射利用極平面特性加速 70
5.5 景深圖內部填補平行化 72
Chapter 6 實驗與結果 75
6.1 測試平台環境描述 75
6.2 Depth Refinement效能分析 76
6.3 整體效能分析 78
6.4 影像相似度百分比 81
6.5 結果與討論: 85
Chapter 7 結論與未來工作 87
7.1 結論 87
7.2 未來工作 88

[1] D. Scharstein and R. Szeliski, “A taxonomy and evaluation of dense two-frame
stereo correspondence algorithms,” International Journal of Computer Vision,
vol. 47, pp. 7–42, Apr. 2002.
[2] M. Tanimoto et al., "Experimental Framework for 3D Video Coding," ISO/IEC/WG11/MPEG2010, W11477, July, 2010.
[3] C. Fehn, "A 3D-TV system based on video plus depth information," in Proceedings of the 37th Asilomar Conference on Signals, Systems and Computers, vol. 2, pp. 1529-1533, Pacific Grove, Calif, USA, November 2003.
[4] D. Tian, P. Lai, P. Lopez, and C. Gomila, “View synthesis techniques for 3D video,”in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, ser. Presented at the Society of Photo-Optical Instrumentation Engineers (SPIE) Conference, vol. 7443, Aug. 2009.
[5] Y. Mori, et al., ''View generation with 3D warping using depth information for FTV," Signal Processing: Image Communication, Vol, 24, No.1, pp.265-272, 2009.
[6] C. Lee and Y. S. Ho, "View synthesis using depth map for 3D video," Proc. 2009 APSIPA Summit and Conference, Sapporo Japan, Oct. 2009.
[7] A. Telea, "An image inpainting technique based on the fast marching method," J. Graphics Tools, Vol.9, No.1, pp.25–36, 2004.
[8] Y. Boykov, O. Veksler and R. Zabih "Fast Approximate Energy Minimization via Graph Cuts". IEEE Trans. Pattern Anal. Mach. Intell. 23(11): 1222-1239 (2001).
[9] M. Tanimoto, T. Fujii, and K. Suzuki, “Depth Estimation Reference Software (DERS)with Image Segmentation and Block Matching,” ISO/IEC JTC1/SC29/WG11, M16092,Lausanne, Switzerland, February 2009.
[10] K. Wegner, O. Stankiewicz and M. Domański, “Stereoscopic depth estimation using fuzzy segment matching,” Picture Coding Symposium, Nagoya, Japan, Dec. 2010.
[11] S.B. Lee and Y.S. Ho, “Multi-view depth map estimation enhancing temporal consistency”,International Technical Conference on Circuits/Systems, Computers and Communications , pp. 29-32, 2008.
[12] M. O. Wildeboer et al., “A semi-automatic multi-view depth estimation method,” in Proc. SPIE Visual Communications and Image Processing, Huangshan, China, Jul. 2010.
[13] M. Tanimoto, "Experimental Framework for 3D Video Coding,"136 ISO/IEC/WG11/MPEG2010, W11477, July, 2010.
[14] H. Baker and R. Bolles, “Generalizing epipolar-plane image analysis on the spatiotemporal surface,” in DARPA Image Understanding Workshop, Cambridge, MA. April 6-8, pp. 1022-1030.
[15] http://thenextwavefutures.wordpress.com/2009/08/02/the-end-of-moores-law/
[16] NVIDIA CUDA C Programming Guide Version 4.0., 2011. {Online}. Available: http://www.nvidia.com/object/cuda_develop.html
[17] C.-T. Lu and H.-M. Hang, “Multiview Encoder Parallelized Fast Search Realization on NVIDIA CUDA,” IEEE Visual Communications and Image Processing '11, Tainan, Taiwan, Nov. 6-9, 2011.
[18] The CUDA Compiler Driver NVCC Version 2.2, NVIDIA. http://moss.csc.ncsu.edu/~mueller/cluster/nvidia/2.0/nvcc_2.0.pdf.
[19] N. Leischner, V. Osipov, and P. Sanders. Fermi Architecture White Paper, 2009.
[20] C.-Y. Hsu, “Free Viewpoint Video Synthesis using 2-D Camera Array Signals,” M.S. Thesis, NCTU, Hsinchu, Taiwan.