臺灣博碩士論文加值系統

現今立體顯示器技術已趨成熟，搭配 3D 數位視訊內容，可以讓觀視者有著身歷其境的視覺享受。在多種立體視訊資訊擷取方式中，具實用性高、成本低且快速的方式即為利用原始二維視訊，搭配深度轉換系統，以產生三維立體視訊資訊。因此，在本論文中提出一種可以將現有的二維視訊轉換為三維視訊的立體視訊轉換系統。
本論文技術建立在相機為緩慢移動拍攝的假設下，對場景中的背景進行視差/深度估測。而對於移動的前景，在偵測出前景區域後，結合各種線索 (cue) 來對其深度進行估測。最後，再將背景與前景估測深度的結果加以整合而為整體深度圖 (depth map)。本系統首先對每一時刻影像切割出移動物體與靜止場景區域。不同時刻兩兩影像間可以利用其空間域上的相關性，計算出因相機移動所造成的移動視差資訊，並對每個切割區域求出最適合且足以代表此切割區域深度分布的視差平面。接著可以利用連續多張深度影像在時間域的相關性，修正相同物件於連續時間深度突兀的現象。被偵測為移動物體的區域則是計算其移動視差、大氣透視和紋理梯度等線索來估測深度。我們又假設移動物體區域底部與靜止場景區域相接，其深度值相似條件下，提出一深度融合準則，避免前景/背景深度的不協調。本系統最後利用深度影像搭配原始影像，合成立體影像對後，輸出至立體螢幕觀看其立體效果。
實驗結果顯示，使用本論文所提出演算法合成之立體視訊，觀看者可感受出明確之立體感，但垂直線區域則有些微不連續。我們的合成立體影像對立體效果與實際雙眼拍攝的立體影像對立體效果相似度頗高。

The stereo display technology has become mature in recent years. The use of stereo display, in combination with 3D image/video contents, really makes the viewers immersed in reality. There are many methods for getting stereo video. A practical, low-cost, and fast way is to use the original 2-D video, in combination with depth conversion, to generate 3-D video. Therefore, in this thesis, we would like to develop a system to convert existing 2-D videos to their 3-D versions for stereo display.
In this thesis, it is assumed that the camera has slow motion, which is used to estimate the disparity/depth information of static backgrounds. For the moving foreground objects, we combine several depth cues from the image to estimate their depth information. Finally, we fuse the background and foreground depth information to get the whole depth map. In our conversion system, we first detect region of moving objects for each image. For two images captured from two different time instants, we explore the spatial correlation to calculate the motion parallax information, and estimate a disparity plane to each segmented region. Based on the similarity of depth information in temporal domain, depth outliers can be corrected. For moving objects, we compute several depth cues, such as motion parallax, atmospheric perspective and texture gradients, by which the depth is assigned. Also we assume that the depth at the bottom of the foreground will be similar to the background depth at the same image position. Based on this, the depths of foreground and background can be fused to get the whole depth map. With the depth map, the stereo image pair can be synthesized for display.
Experimental results show that the synthesized 3D video presents a good depth perception, but with slight depth discontinuity along vertical direction. Subject tests show that our synthesized 3D video and the real captured 3D video have similar depth perception.

摘要 i
Abstract ii
目錄 iii
圖目錄 v
表目錄 ix
第一章 緒論 1
1.1 研究動機 1
1.2 立體視訊相關理論與技術 1
1.2.1. 影像合成法 4
1.2.2. 深度合成法 6
1.3 本論文方法簡介 11
第二章 基於微小相機移動之影像深度估測系統 13
2.1 系統架構說明 13
2.2 影像中移動物體與靜止場景切割 14
2.3 移動物體之深度估測 15
2.4 靜止場景之深度估測 20
2.4.1. 基於 Mean-shift 法則之影像切割 21
2.4.2. 以像素為單位之初步視差估測 25
2.4.3. 以切割區域為單位之初步視差平面估測 26
2.4.4. 切割區域之視差平面修正 28
2.4.5. 遮蔽區域之視差估測 29
2.5 移動物體與靜止場景之深度合併 31
第三章 基於微小相機移動之 2D 至 3D 立體視訊轉換系統 33
3.1 系統架構說明 33
3.2 連續多張影像間之深度影像修正 34
3.2.1. 利用 sliding-window 選取相鄰的多張深度影像 35
3.2.2. 利用 KLT 演算法德原始影像間的特徵點匹配 35
3.2.3. 修正特徵點所屬視差平面參數 35
3.3 基於深度影像與原始影像之立體影像對合成 36
第四章 實驗結果與分析 38
4.1 空間深度估測結果 (靜物) 38
4.2 時間上連續影像深度之修正評估 49
第五章 結論與未來工作 52
參考文獻 53

[1]M. V. Sudhamani and C. R. Venugopal, "Segmentation of Color Images using Mean Shift Algorithm for Feature Extraction," Proc. of IEEE Int’l Conf. on Information Technology (ICIT-2006), Dec 2006.
[2]D. Comaniciu and P. Meer, "Mean Shift: A Robust Approach toward Feature Space Analysis," IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 24, No. 5, pp.603-619, May 2002.
[3]D. Scharstein and R. Szeliski, "A Taxonomy and Evaluation of Dense Two-frame Stereo Correspondence Algorithms," International Journal of Computer Vision, Vol.47 (1/2/3), pp.7-42, April 2002.
[4]葉晟穎，基於單視域來源之立體視訊轉換技術，國立中正大學電機工程學系碩士論文，中華民國九十七年七月
[5]蕭偉權，建構於物體運動軌跡特徵之視訊搜尋研究，國立中正大學電機工程學系碩士論文，中華民國九十年七月
[6]王泓民，基於 H.264/AVC 核心之快速演算法與立體視訊合成之研究，國立成功大學電機工程學系博士論文，中華民國九十八年六月
[7]Sylvie Jeannin, Radu Jasinschi, Alfred She, Thumpudi Naveen, Benoit Mory and Ali Tabatabai, "Motion Descriptor for Content-Based Video Representation," Signal Processing: Image Communication, Vol.16, pp.59-85, Sep. 2000.
[8]M. T. Suzuki, Y. Yaginuma, T. Yamada and Y. Shimizu, "A Shape Feature Extraction Method Based on 3D Convolution Masks," Proc. of IEEE International Symposium on Multimedia (ISM-2006), pp.837-844, Dec. 2006.
[9]P. F. Felzenszwalb and D. P. Huttenlocher, "Efficient Belief Propagation for Early Vision," Proc. of IEEE Int’l Conf. on Computer Vision and Pattern Recognition (CVPR-2004), Vol.1, pp. 261–268, July 2004.
[10]T. Okino, H. Murata, K. Taima, T. Iinuma and K. Oketani, "New Television with 2D/3D Image Conversion Technologies," Proc. of SPIE, Vol. 2653, pp. 96-103, 1996.
[11]S. Forstmann, Y. Kanou, Jun Ohya, S. Thuering and A. Schmitt, "Real-time Stereo by Using Dynamic Programming," Proc. of IEEE Int’l Conf. on Computer Vision and Pattern Recognition Workshop, pp.29-36, June 2004.
[12]Jae Chul Kim. Kyoung Mu Lee. Byoung Tae Choi. and Sang Uk Lee. "A Dense Stereo Matching Using Two-Pass Dynamic Programming with Generalized Ground Control Points," Proc. of IEEE Int’l Conf. on Computer Vision and Pattern Recognition, Vol.2, pp.1075-1082, June 2005.
[13]Liang Wang, Miao Liao, Minglun Gong, Ruigang Yang and D. Nister, "High-Quality Real-Time Stereo Using Adaptive Cost Aggregation and Dynamic Programming," Proc. of IEEE Int’l Symposium on 3D Data Processing, Visualization, and Transmission, Chapel Hill, NC, pp.798-805, June 2006.
[14]M. Bleyer and M. Gelautz, "Graph-Based Surface Reconstruction from Stereo Pairs Using Image Segmentation," Proc. of SPIE Conf., Vol.5665, pp.288-299, 2005.
[15]M. Bleyer and M. Gelautz. "A Layered Stereo Algorithm Using Image Segmentation and Global Visibility Constraints," Proc. of IEEE Int’l Conf. on Image Processing (ICIP-2004), Vol.5, pp. 2997-3000, Oct. 2004.
[16]A. Klaus, M. Sormann and K. Karner, "Segment-Based Stereo Matching Using Belief Propagation and A Self-Adapting Dissimilarity Measure," Proc. of IEEE Int’l Conf. on Pattern Recognition (ICPR-2006), Hong Kong, Vol.3, pp.15-18, 2006.
[17]Jian Sun, Nan-Ning Zheng and Heung-Yeung Shum "Stereo Matching Using Belief Propagation," IEEE Transactions on Pattern Analysis and Machine Intelligence. Vol. 25, issue 7, pp.787-800, July 2003.
[18]L. Hong and G. Chen, "Segment-Based Stereo Matching Using Graph Cuts," Proc. of IEEE Int’l Conf. on Computer Vision and Pattern Recognition, vol. 1, pp. 74-81, June 2004.
[19]Z. Wang and Z. Zheng. "A Region Based Stereo Matching Algorithm Using Cooperative Optimization," Proc. of IEEE Int’l Conf. on Computer Vision and Pattern Recognition, Anchorage, AK, pp. 1-8, June 2008.
[20]C. Cigla, X. Zabulis, A. Aydin Alatan, "Segment-Based Stereo-Matching via Plane and Angle Sweeping," IEEE Conf. 3DTV, Kos Island, pp.1-4, May 2007
[21]Liang Zhang and Wa James Tam, "Stereoscopic Image Generation Based on Depth Images for 3D TV," IEEE Trans. on Broadcasting, Vol.51, Issue 2, pp.191 – 199, June, 2005.