臺灣博碩士論文加值系統

指寫法辨識可提供使用者自然、直覺的人機互動方式，因此近年來被廣泛應用於各種領域中，如電子遊戲、遠距遙控等。本論文提出一個以色彩深度影像為基礎的指寫法數字辨識系統，分為手掌與指尖位置偵測、特徵擷取及指寫法數字辨識三個部分。首先利用膚色偵測配合聯通物件法與距離轉換法，分別找出手掌與指尖位置；接著追蹤指尖書寫數字之動態軌跡，先擷取軌跡移動方向作為數字辨識用之特徵，其中因為0與6無法經由移動方向特徵予以辨識，所以再提出軌跡起始點、終止點與垂直方向移動距離三種特徵，以提高辨識率。最後，利用串接式最鄰近法進行指寫法數字辨識。從實驗結果可知，本論文所提之即時指寫法數字辨識系統確實具有成效，可達九成五以上之辨識率。

The finger writing recognition approach has been introduced to a diversity of fields, like video games and remote control systems, because it provides a natural and intuitive communication for Human-Computer Interaction (HCI). This thesis proposes a real-time finger writing digit recognition system with high accuracy rate based on the RGB-D information. The system is divided into three main parts, including ROI selection, feature extraction, and finger writing digit recognition. For the ROI selection, first detect the skin color regions, then determine the palm and the fingertips based on the connect component labeling (CCL) and the distance transform respectively. Further, track the fingertip to create the trajectory and extract its directional features for digit recognition. However, since it is often confused in recognizing 0 and 6, three extra features are added to increase their recognition rate. Finally, with series k-NN classifiers, the experimental results show that the accuracy rate is higher than 95% in finger writing digit recognition, which implies the proposed real-time recognition system is indeed effective and efficient.

Chinese Abstract i
English Abstract ii
誌謝 iii
Contents iv
List of Figures vii
Index of Tables x
Chapter 1 Introduction
1.1 Preliminary 1
1.2 System Overview 3
1.3 Research Organization 5
Chapter 2 Background
2.1 Skin Color Segmentation 6
2.2 Dynamic Time Warping 8
2.3 k-Nearest Neighbors Classification 11
Chapter 3 Finger Writing Digit Recognition
3.1 Hand Region Detection 15
3.1.1 Skin Color Extraction 15
3.1.2 Finger Region Extraction 16
3.1.3 Depth Cutting 19
3.2 Fingertips positioning 21
3.2.1 Hand Shape Description 21
3.2.2 Fingertips Detection 23
3.3 Feature Extraction 24
3.3.1 Positional Features 24
3.3.2 Directional Features 25
3.4 Finger Writing Digit Recognition 26
3.4.1 Multiple Classifier Based on Directional Features 26
3.4.2 Binary Classifier Based on Positional Features 29
3.4.3 Finger Writing Digit Recognition System 32
Chapter 4 Experimental Result
4.1 Hand Detection 36
4.2 Fingertips Positioning 42
4.3 Feature Extraction 44
4.4 Finger Writing Digit Recognition 47
Chapter 5 Conclusions and Future Works
5.1 Conclusions 53
5.2 Future Works 55
Reference 56

[1] Allen, J.M.; Asselin, P.K.; Foulds, R., "American Sign Language finger spelling recognition system," Bioengineering Conference, 2003 IEEE 29th Annual, Proceedings of , vol., no., pp.285,286, 22-23 March 2003
[2] Bragatto, T. A C; Ruas, G. I S; Lamar, M.V., "Real-time video based finger spelling recognition system using low computational complexity Artificial Neural Networks," Telecommunications Symposium, 2006 International , vol., no., pp.393,397, 3-6 Sept. 2006
[3] Bui, T.D.; Nguyen, L.T., "Recognizing Postures in Vietnamese Sign Language With MEMS Accelerometers," Sensors Journal, IEEE , vol.7, no.5, pp.707,712, May 2007
[4] Wang, J.S.; Chuang, F.C., "An Accelerometer-Based Digital Pen With a Trajectory Recognition Algorithm for Handwritten Digit and Gesture Recognition," Industrial Electronics, Transactions on IEEE , vol. 59, no. 7, pp.2998,3007, July 2012
[5] Lu, Z.; Chen, X.; Li, Q.; Zhang, X.; Zhou, P., "A Hand Gesture Recognition Framework and Wearable Gesture-Based Interaction Prototype for Mobile Devices," Human-Machine Systems, IEEE , vol. 44, no. 2, pp.293,299, April 2014
[6] Frolova, D.; Stern, H.; Berman, S., "Most Probable Longest Common Subsequence for Recognition of Gesture Character Input," Cybernetics, Transactions on IEEE , vol. 43 no. 3, pp.871,880, June 2013
[7] Zhang, X.; Ye, Z.; Jin, L.; Feng, Z.; Xu S., "A New Writing Experience: Finger Writing in the Air Using a Kinect Sensor," MultiMedia, IEEE , vol. 20, no. 4, pp.85,93, Oct.-Dec. 2013
[8] Terrillon, J.-C.; Shirazi, M.N.; Fukamachi, H.; Akamatsu, S., "Comparative performance of different skin chrominance models and chrominance spaces for the automatic detection of human faces in color images," Automatic Face and Gesture Recognition, Fourth IEEE International Conference on, vol., no., pp.54,61, 2000
[9] M.H. Yang, N. Ahuja, "Gaussian Mixture model for human skin color and its application in image and video databases", Proceedings of SPIE: Conference on Storage and Retrieval for Image and Video Databases, vol. 3656, pp. 458–466,1999.
[10] Steven M. Kay, Fundamentals of Statistical Signal Processing Volume II Detection Theory: Pearson/Prentice Hall, 1996.
[11] Sakoe, H.; Chiba, S., "Dynamic programming algorithm optimization for spoken word recognition," Acoustics, Speech, Signal Processing, Transactions on IEEE , vol. 26, pp.43, Feb. 1978
[12] Rabiner, L.; Levinson, S., " Isolated and connected word recognition — theory and selected applications," Communication, Transactions on IEEE , vol. 29, pp.621,659, May 1981
[13] M. A. Al-Mouhamed, O. Toker, and A. Al-Harthy, "A 3-D vision-based man-machine interface for hand-controlled telerobot," Industrial Electronics, IEEE Transactions on, vol.52, pp.306-319, 2005.
[14] P. Viola and M. Jones, “Robust real-time object detection,” Computer Vision, International Journal on, vol. 2, no. 57, pp. 137–154, 2004.
[15] R. Laganière, OpenCV 2 computer vision application programming cookbook: Packt Publ. Limited, 2011.
[16] G. Borgefors, "Distance transformations in digital images," Computer vision, graphics, and image processing, vol. 34, pp. 344-371, 1986.
[17] A. Meijster, J. B. T. M. Roerdink, and W. H. Hesselink, "A general algorithm for computing distance transforms in linear time," Mathematical Morphology and its applications to image and signal processing, pp. 331-340, 2002.
[18] Le Dung; Mizukawa, M., "Fast fingertips positioning based on distance-based feature pixels," Communications and Electronics (ICCE), 2010 Third International Conference on , vol., no., pp.184,189, 11-13 Aug. 2010
[19] Le Dung and Makoto Mizukawa, " Fast Hand Feature Extraction Based on Connected Component Labeling, Distance Transform and Hough Transform," Journal of Robotics and Mechatronics, Vol.21 No.6, 2009
[20] Shi-Cheng Wu, " Real-Time Hand Gesture Recognition System Design Based on Image Feature Points Extraction and Depth Information, " Master thesis, National Chaio Tung University, 2013
[21] BAI, Z. L.; HUO, Q., " A Study On the Use of 8-Directional Features For Online Handwritten Chinese Character Recognition," Document Analysis and Recognition, 2005 Eighth International Conference on, vol.1, no., pp.232,236, Aug. 2005