本論文實現了一個創新的多媒體互動系統，我們結合了3D立體顯示以及利用體感方式來操控遊戲。本系統所使用的方法是基於人類視覺的特性，由於人眼在觀看物體時，左眼影像與右眼影像會有些許不同，利用亮度資訊和極線幾何，將左眼與右眼影像進行影像匹配，進而推導出影像中物體的深度資訊。本系統是在用以手勢去做互動，因此，經由攝影機拍攝左右影像後，藉由匹配演算法計算深度資訊求得手與攝影機距離。本系統也對3D立體影像中物體在虛擬空間裡的座標做估測，利用眼距、使用者與螢幕距離、disparity等資訊去求得虛擬物體在空間中座標，最後再將手與虛擬物體經由座標轉換為同一座標系之後達到互動效果。本論文藉由實作一個3D立體互動水族缸系統，讓使用者去體驗創新的體感互動娛樂效果。

This thesis presents a 3D stereoscopic interactive fish tank system that combines the 3D stereoscopy and “controller-free” components. Based on the characteristics of human vision, when seeing the objects, the left eye image and right eye image will be slightly different, one can use the intensity information and the epipolar geometry to proceed matching, and then to generate the 3D depth information. This system allows a user to use gesture to do interaction. It estimates 3D objects depth information, and uses eyes distance, distance between the user and the sensor, disparity map to calculate the virtual objects’ three-dimensional coordinate, and then transforms hand and virtual objects’ coordinate into the same coordinate to allow accurate interaction. The system allows users to experience the innovative multi-media interactive entertainment.

論文審定書 i
誌 謝 ii
摘 要 iv
Abstract v
Contents vi
List of Figures viii
Chapter 1 Introduction 1
Chapter 2 Background Materials and Related Work 3
2.1 Depth Perception 3
2.1.1 Binocular Disparity 3
2.1.2 Monocular Disparity 5
2.1.3 Three Styles of Generating Depth Perception 7
2.2 Principle of 3D Display 8
Chapter 3 The Proposed System 13
3.1 Disparity Map Reconstruction 14
3.2 3D Objects Depth Position Estimation 19
3.3 3D Hand Tracking and Fingertip Detection 22
3.3.1 Disparity Map Segmentation 24
3.3.2 Contour Finding 25
3.3.3 Convex Hull 26
3.3.4 Feature Points Extraction and Fingertip Detection 27
3.4 3D Model Construction 28
3.5 Interaction Mechanism 30
Chapter 4 Implementation 33
4.1 Introduction 3D fish Tank System 33
4.1.1 Apparatus and environment 33
4.1.2 OpenNI and NITE 33
4.1.3 Kinect 35
4.2 3D Scene Construction 36
4.3 Demonstration 39
Chapter 5 Conclusion and Future Works 44
References 45

[1]H. Murata, and Y. Mori, “A real-time 2D to 3D image conversion technique using computed image depth,” SID, DIGEST, pp.919-922, 1998
[2]T. Okino, H. Murata, K. Taima, T. Iinuma, and K. Oketani, “New television with 2D to 3D image conversion technologies,” In Proc. SPIE Stereoscopic Displays and Virtual Reality Systems III, Vol. 2653, pp. 96-103, 1996.
[3]T. S. Jebara, and A. Pentland, “Parametrized structure from motion for 3D adaptive feedback tracking of faces,” In Proc. IEEE International Conf. on Computer Vision and Pattern Recognition, pp. 144-150, 1997.
[4]E. Steinbach, A. Hanhalic, and B. Girod, “3D motion and scene structure estimation with motion dependent distortion of measurement windows,” In Proc. IEEE International Conf. on Computer Vision, Vol. 1, pp. 434-422, 1999.
[5]A. Criminisi, I. Reid, and A. Zisserman, “Single view metrology,” In Proc. of the Seventh IEEE International Conf., Vol. 1, pp.434-441, 1999.
[6]http://www.yorku.ca/eye/disparit.htm
[7]A. Klaus, M. Sormann, and K. Karner, “Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure,” 18th International Conference on Pattern Recognition, pp.15-18, 2006.
[8]D. Scharstein, and R. Szeliski, “A taxonomy and evaluation of dense two-frame stereo correspondence algorithms,” Int. J. Computer Vision, Vol. 47, pp.7–42, 2002.
[9]I. J. Cox, S. L. Hingorani, S. B. Rao, and B. M. Maggs, "A maximum likelihood stereo algorithm," Computer Vision and Image Understanding, vol.63, no.3, pp.542-567, 1996.
[10]A. Redert, E. Hendriks, and J. Biemond, “Correspondence estimation in image pairs,” IEEE Signal Processing Magazine, vol.16, no.3, pp. 29–46, May 1999.
[11]L. Falkenhagen, “Depth estimation from stereoscopic image pairs assuming piecewise continuos surfaces,” Image Processing for Broadcast and Video Production, pp. 115-127, 1994
[12]E. Izquierdo, “Stereo matching for enhanced telepresence in 3D video communications,” IEEE Trans. Circuits Syst. Video Technol., Special Issue on Multimedia Technology, Systems and Applications, vol.7, pp.629-643 , Aug 1997
[13]R. K. Gupta and S. Y. Cho, “A color-based approach for disparity refinement,” Control Automation Robotics & Vision (ICARCV), 11th International Con., pp.664-667, Dec 2010
[14]P. Fua, “A parallel stereo algorithm that produces dense depth maps and preserves image features,” Machine Vision Applications, Vol. 6, No.1, pp. 35-49, 1993.
[15]N. Holliman, “Mapping perceived depth to regions of interest in stereoscopic images,” In Proc. SPIE, Vol. 5291, 2004
[16]P. Howard, J. R. Brian, and A. P. Howard, “Binocular vision and stereopsis,” Oxford University Press, Chapter 7, 1995.
[17]C. M. Kim, M. W. Shin, S. M. Jeong, I. C. Kang, E. S. Kim, and B. K. Kim, “Real-time motion simulation of artificial fish for virtual marine world,” Advanced Language Processing and Web Information Technology, Sixth International Conf., pp253-260, Aug 2007
[18]S. Malik, “Real-time hand tracking and finger tracking for Interaction,” Computer Science, University of Toronto, project report, Dec 2003
[19]S. Suzuki, and K. Be, “Topological structural analysis of digitized binary images by border following,” CVGIP, Vol. 30, No.1, pp.32–46, Apr 1985.
[20]P. M. Hubbard, “Collision detection for interactive graphics applications,” IEEE Transactions on Visualization and Computer Graphics, Vol. 1, pp218-230, Sep 1995
[21]http://www.openni.org/
[22]http://www.primesense.com/?p=515
[23]http://www.ros.org/wiki/kinect_calibration/technical
[24]T. S. Lyes and K. A. Hawick, “Implementing stereo vision of GPU-accelerated scientific simulations using commodity hardware,” Computational Science Technical Note Series (CSTN) – 131, Mar 2011
[25]S. Battiato, A. Capra, S. Curti, and L. M. Cascia, “3D stereoscopic image pairs by depth-map generation,” IEEE International Symposium on 3D Data Processing, Visualization and Transmission, pp. 124-131, 2004.
[26]S. Curti, D. Sirtori, and F. Vella, “3D effect generation from monocular view,” 3D Data Processing Visualization and Transmission, pp. 550-553, 2002.
[27]http://www.csie.ntnu.edu.tw/~u91029/ConvexHull.html