隨著人機互動的技術創新與進步，輸入模式由從鍵盤輸入到與滑鼠的控制，光筆，搖桿，繪圖板，一直演變到人體攜帶設備，如:跳舞機踏墊、無線搖桿、體感控制器…等。研發至今，科技的日新月異講求的是直覺性與自然性的互動方式，近幾年來科技產品的操作演變趨勢逐漸以人類本能動作為中心的直觀控制為概念。基於這個理念，在本研究中即採用影像偵測追蹤結合手勢辨識在即時通訊軟體中提供互動的功能，使網路通訊之間增添趣味性和靈活性。由於手部辨識與追蹤系統在現有的演算法裡已發展的相當純熟，為了更符合本研究以互動為目標，本系統加強了辨識率與追蹤速率，來強化即時通訊系統的互動效率。故本研究著重在手勢辨識之外，還加上互動系統為宗旨的趣味性與運用的可行性，利用已開發的MSNP通訊協議系統為平台，加入定義手勢的操控方式，達到通訊間以人體手勢來控制互動的直覺性操作方式。

With the increasement of technology level, the evolution and innovation of human-computer interaction (HCI) has many input patterns. Currently, the client of traditional input from the keyboard, mouse, light pen, joystick and pen tablet, until user can wear equipments like data glove or sensors. Nowadays, the technology of HCI already has developed to natural visual-based input devices. Following the trend, in this paper that is using camera to recognition and tracking real time vision based hand gesture to execute special functions on the instant messaging software.
Research into HCI (Human-Computer Interaction) has shown dramatic evidences to the shift of interaction manners. To achieve well-designed I/O scenario, the use of multimedia technologies is necessary. In this paper, an interactive gesture annotation framework was proposed to enhance the communication process. The built-in camera was considered as a simple input device. The tracking algorithm was applied to recognize the gestures input by end-users. As a contribution, the developed framework can now be applied in MSNP (Mobile Status and Notification Protocol), and the recognition rate can achieve around 80% in a real-time scenario.

目錄
第一章 緒論 1
1.1 研究動機與目的 1
1.2 論文架構介紹 4
第二章 相關研究 5
2.1 手部追蹤與辨識相關課題 5
2.2 人機互動模式(HCI)模式簡介 11
2.3 即時通訊在社群互動上之應用 15
第三章 研究方法 16
3.1 系統基礎架構說明 16
3.2 即時通訊系統分析 17
3.3 手部判斷與追蹤演算法 23
3.4 互動關聯模型定義 30
第四章 系統實作 34
4.1 開發工具與環境 34
4.1.1 手勢追蹤與辨識之開發工具 34
4.1.2 即時通訊系統之開發環境 36
4.2 系統介面說明與實作成果 41
第五章 結論與未來展望 49
5.1 結論 49
5.2 未來展望 50
參考文獻 51
附錄-英文論文 54

[1]C. Manresa, J. Varona, R. Mas, and F. J. Perales, “Hand Tracking and Gesture Recognition for Human-Computer Interaction”, Electronic letters on computer vision and image analysis, vol. 5, no.3, 2005, pp. 96-104.

[2]M. Donoser and H. Bischof, “Real Time Appearance Based Hand Tracking”, Pattern Recognition, ICPR 2008, Tampa, FL, Dec. 2008.

[3]Q. Chen, N. D. Georganas and E. M. Petriu, “Real-time Vision-based Hand Gesture Recognition Using Haar-like Features”, Instrumentation and Measurement Technology Conference Proceedings, May. 2007, pp.1-6.

[4]S. Malik, “Real-time Hand Tracking and Finger Tracking for Interaction”, Dec. 2003.

[5]A. Wu, M. Shah and N. da Vitoria Lobo, “A Virtual 3D Blackboard: 3D Finger Tracking using a Single Camera”, IEEE international conference on automatic face and gesture recognition, 2000.

[6]S. Rodriguez, A. Picon and A. Villodas, “Robust Vision-Based Hand Tracking Using Single Camera for Ubiquitous 3D Gesture Interaction”, 3D User Interfaces (3DUI), Waltham, MA , Mar. 2010.

[7]C. Schwarz and N. D. V. Lobo, “The Camera-Driven Interactive Table”, Applications of Computer Vision, 2007. WACV '07. 2007, pp. 50-50.

[8]S. M. Dominguez, T. Keaton and A. H. Sayed, “A Robust Finger Tracking Method for Multimodal Wearable Computer Interfacing”, Multimedia, 2006. Vol. 8, No. 5, Oct. 2006, pp. 956 – 972.

[9]O. Portillo-Rodriguez, O. O. Sandoval-Gonzalez, C. A. Avizzano, E. Ruffaldi, D. Vercelli and M. Bergamasco “Development of a 3D Real Time Gesture Recognition Methodology for Virtual Environment Control”, Robot and Human Interactive Communication, 2008. RO-MAN 2008, Aug. 2008, pp. 279-284.

[10]O. S. Goh; C. C. Fung and K. W. Wong, “VisualChat: A Visualization Tool for Human-Machine Interaction”, Web Intelligence and Intelligent Agent Technology, 2008. WI-IAT '08, 2008, pp.131-134.

[11]P. Viola and M. Jones, “Fast Multi-view Face Detection”, Proc. of Computer Vision and Pattern Recognition, Jul. 2003.

[12]P. Viola and M. Jones, “Robust Real-time Object Detection”, International Journal of Computer Vision, Feb. 2001.

[13]“Tracking features using the KLT algorithm”, VGIS 821, 2009.

[14]R. E. Kalman, “A New Approach to Linear Filtering and Prediction Problems”, Transaction of the ASME-Journal of Basic Engineering, no.82, 1960, pp35-45.

[15]P. L. Rosin and T. Ellis, “Image difference threshold strategies and shadow detection”, British conference on Machine vision. BMVC, 1995.

[16]K. G. Derpanis, “Mean Shift Clustering”, Aug. 2005.

[17]C. Tomasi and T. Kanade, “Shape and Motion from Image Streams: a Factorization Method-Part 3 Detection and Tracking of Point Features”, Technical Report CMU-CS-92-104, Carnegie Mellon University, Mar. 1992.

[18]M. Kolsch and M. Turk, “Hand Tracking with Flocks of Features”, CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), Vol. 2, 2005.

[19]M. Kolsch and M. Turk, “Robust Hand Detection”, Automatic Face and Gesture Recognition, 2004. Proceedings. Sixth IEEE International Conference, May. 2004, pp. 614 – 619.

[20]M. Kolsch and M. Turk, “Analysis of Rotational Robustness of Hand Detection with a Viola-Jones Detector”, Pattern Recognition, 2004. ICPR 2004. Proceedings of the 17th International Conference, Vol.3, Aug. 2004, pp.107 – 110.