臺灣博碩士論文加值系統

傳統的人機介面當中，幾乎都是利用接觸式的方式與電腦做溝通，隨著軟硬體技術不斷增進，人機介面的研究領域亦伴隨著重大發展，從早期的命令式介面、圖形化介面、接觸式近距離操作，到現在的非接觸式遠距離操作。在這麼多種操作方式之中，其中以非接觸式遠距離操作最常使用在遊戲操作上，像是：XBOX360 Kinect、Wii...等，這些都是利用手中的感測器和影像識別技術所達成，讓使用者可以在遠處操作電腦，取代了傳統操作上的拘束性。

在本論文中，利用影像識別、影像切割技術並搭配Kinect深度攝影機，改善了傳統接觸式滑鼠的缺點，達到非接觸式遠距離操作的目的。在即時辨識系統下，如何經由影像識別技術來判斷使用者手勢，是本論文的重要議題，所以本論文提出了五種不同程序來解決此議題，並將識別結果來當成滑鼠移動、控制的基準，使用者只需利用簡單手勢就可達到手勢滑鼠的操作，藉由此系統將取代傳統滑鼠的不便性，已達到更友善的操作方式。

Hand gesture recognition has been a popular research in recent years with a major emphasis on tracking, automatic feature detection and matching. Hand gesture recognition was not often applied to real applications. However, with an inexpensive and effective sensor, hand gesture input can become a useful and popular approach for the human-computer interface such as the remote mouse and virtual keyboard.
In this thesis, image segmentation and object recognition techniques are adopted to implement a real-time non-contact mouse using a Kinect. The algorithm consists of five main procedures: hand/arm detection, preprocessing, hand segmentation, hand gesture recognition, and mouse actions. Through simple hand gestures, the user can control the cursor in the windows system to achieve the control and operation performed by the traditional mouse.

目錄
致謝…...…………………………..…………………………………………….I
摘要…...…………………………..…………………………………………….II
Abstract.……………………...………………………………………………...III
目錄…..………………………………………………………………...……..IV
圖目錄..………..………………………………………………………………VI
表目錄..………………………………………………………………..……IX

第一章 緒論…...………………………………………………………………1
1.1 研究動機與目的..…………………………………………………….1
1.2 相關研究…...…………………………………………………………2
1.3 研究方法…...…………………………………………………………3
1.4 論文架構…...…………………………………………………………4

第二章 手勢偵測系統架構….………………………………………………..5
2.1 攝影機系統及硬體介紹….………………………………..…………6
2.2 物體偵測程序…….……………………………………………11
2.3 影像前處理程序….………………………………………………..13
2.4 手腕位置與切割程序 ………………………………………………14
2.5 手勢辨識程序………………………………….………………..15
2.6 手勢滑鼠辨識程序…………………………………….…...…………16

第三章 影像前處理程序..…………………………………………………...17
3.1 光源補償技術…..……………………………………………….…..18
3.2 色彩空間轉換…..………………………………………….………..22
3.3 色彩空間表……...…………………………….…………………….26
3.4 深度特徵…...……………………….……………………………….28
3.5 影像標註 (Labeling)..………………………………………………32
3.6 形態學 (morphological processing).………………………….……34
 
第四章 手腕位置與切割程序……….………………………………………36
4.1 手部質心位置…………...…………………………………………..37
4.2 手部角度校正…...…………………………………………………..38
4.3 手腕位置切割...……………………………………………………..40

第五章 手勢辨識程序...……………………………………………………..44
5.1 高斯濾波器 (Gaussian filter)..……………………………………..44
5.2 手指切割…………………………………………………………….46
5.3 指尖偵測 (Fingertip detection)……………………………………..47

第六章 手勢滑鼠辨識程序.…………………………………………………49
6.1 手勢滑鼠定義...……………………………………………………..49
6.2 螢幕解析度………………………………………………………….50
6.3 定義滑鼠座標...……………………………………………………..51

第七章 實驗結果…………………………………………………………….52
7.1 影像資料庫………………………………………………………….53
7.2 不同角度手部切割結果…………………………………………….54
7.3 手勢辨識率………………………………………………………….55
7.4 手勢與臉部切割結果.………………………………………………57
7.5 手勢滑鼠應用於瀏覽網頁………………………………………….58
7.6 手勢滑鼠應用於小畫家…………………………………………….59
7.7 手勢滑鼠應用於遊戲.………………………………………………60

第八章 未來展望與結論...…………………………………………………..61

參考文獻…..………………………………………………………………….62

參考文獻
[1] 石柏良, ”利用灰關聯分析法建立即時手部動作辨識系統”, 成功大學醫學工程研究所碩博士班, 2008.
[2] 葉政憲, ”手部資料擷取系統之設計與應用”, 國立成功大學電機工程所碩士論文, 2005.
[3] 沈全發, ”機械式手套與虛擬實境之整合研究”, 國立成功大學電機工程所碩士論文, 2002.
[4] D. Kelly, J. McDonald and C. Markham, ” A person independent system for recognition of hand postures used in sign language,” Pattern Recognition Letters, vol.31, pp.1359-1368, 2010.
[5] Microsoft Xbox. Kinect. [Online]
Available: www.xbox.com/zh-TW/kinect.
[6] P. J. M. Aarts and V. Laarhoven, “Simulated annealing and applications: Mathematics and its applications,” D. Reidel Publishing Company, 1987.
[7] M. S. Lew, T. S. Huang and K.W. Wong, “Learning and feature selection in stereo matching,” IEEE Transactions on Pattern Analysis and Machine Intelligence , Vol.16, No.9, pp.869-881, 1994.
[8] R. L. Hsu, M. Abdel-Mottaleb, and A. K. Jain, ”Face detection in color image,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.24(5), pp. 696–706, 2002.
[9] J. Y. Xu, “Face detection and recognition technology research in complexbackground,” M.S. thesis, Shandong University of Technology, China, pp. 22–24, 2007.
[10] E. Y. Lam, ”Combining gray world and Retinex theory for automatic whitebalance in digital photography,” In Proceedings of the ninth international symposium on consumer electronics, Macau pp.134–139, 2005.
[11] D. Chai and A. Bouzerdoum, “A Bayesian approach to skin color classificationin YCbCr color space,” In Proceedings of TENCON, Vol. 2, pp. 421-424, 2000.
[12] MH. Yang and N. Ahuja, “Gaussian mixture model for human skin color and its applications in image and video database,” In Proceedings of the SPIE: Storage and Retrieval for Image and Video Databases VII, no. 3656, 458-466, 1999.
[13] M. Soriano, B. Martinkauppi, S. Huovinen and M. Laaksonen, “Skin detectionin video under changing illumination conditions”. In Proceedings of the 15th international conference on pattern recognition, Vol. 1, pp. 839–842, 2000.
[14] J. P. Serra, “Image Analysis and Mathematical Morphology,” Academic Press, pp. 115-130, 1982.
[15] A. A. Argyros and M. I. A. Lourakis, “Vision-Based Interpretation of Hand Gestures for Remote Control of a Computer Mouse,” Lecture Notes in Computer Science, vol. 3979, pp. 40-51, 2006.
[16] Y. K. Chan and C. Y. Chen, “Image Retrieval System Based on Color-Complexity and Color-Spatial Features,” The Journal of Systems and Software, Vol. 71(1-2), pp. 65-70, 2004.
[17] C. H. Lin, R. T. Chen and Y. K. Chan, “A smart content-based image retrieval system based on color and texture feature,” Image and Vision Computing, Vol. 27, pp. 658–665, 2009.
[18] GH. Liu, ZY Li, L. Zhang and Y. Xu, ” Image retrieval based on micro-structure descriptor,” Pattern Recognition, vol 44, pp.2123-2133, 2011.
[19] GH. Liu, L. Zhang, YK. Hou, ZY. Li and JY. Yang, “Image retrieval based on multi-texton histogram,” Pattern Recognition, vol.43, pp.2380-2389, 2010.
[20] KJ. Hsiao, TW. Chen and SY. Chien, "Fast fingertip positioning by combining particle filtering with particle random diffusion," IEEE International Conference on Multimedia and Expo, pp. 977-980, 2008.
[21] A. Malima, E. Ozgur, and M. Cetin, "A Fast Algorithm for Vision-Based Hand Gesture Recognition for Robot Control," 14th IEEE International Conference on Signal Processing and Communications Applications, pp. 1-4, 2006.
[22] DY. Huang, WC. Hu and SH. Chang, “Gabor filter-based hand-pose angle estimation for hand gesture recognition under varying illumination,” Expert Systems with Applications, vol.38, pp.6031-6042, 2011.
[23] Mariusz, F. and Szymon, M. ,” On the use of graph parsing for recognition of isolated hand postures of Polish Sign Language,” Pattern Recognition, vol.43 , pp.2249-2264, 2010.
[24] J. Alon and Q. Yuan, “A Unified Framework for Gesture Recognition and Spatiotemporal Gesture Segmentation,” IEEE Transactions on pattern analysis and machine intelligence, vol. 31, no.9 , 2009.
[25] J. L. Raheja, R. Shyam, U. Kumar and PB. Prasad, “Real-Time Robotic Hand Control using Hand Gestures,” Second international conference on machine learning and computing, 2010.
[26] H. I. Park and J. W. Lee, “Hand Gesture Recognition for Table-Top Interaction System,” International Symposium on Ubiquitous Virtual Reality, Vol. 260, pp. 34-35, 2007.
[27] A. Licsar and T. Sziranyi, “Hand Gesture Recognition in Camera-Projector,” International Workshop on Human-Computer Interaction, Lecture Notes in Computer Science, Vol. 3058, pp. 83-93, 2004.
[28] N. Kanopoulos, N. Vasanthavada and R. L. Baker, “Design of An ImageEdge Detection Filter Using the Sobel Operator,” IEEE Journal of Solid-State Circuits, Vol. 23, No. 2, pp. 358-367, 1988.
[29] D. H. Ballard and C. M. Brown, Computer Vision, Prentice-Hall, NewYork, pp. 65-78, 1982.
[30] 邱鐘慶, ”應用於三維立體電視之深度影像重建系統晶片設計與實現”, 國立台北科技大學碩士論文, 2010.
[31] M.Sarfraz, S.A. Mahmoud and Z. Rasheed, “On Skew Estimation and Correction of Text,” Proc. Of IEEE on computer Graphics, Imaging and Visualisation, pp.308-313, 2007.