臺灣博碩士論文加值系統

為提升手語辨識的準確度、可靠度與便利性，以協助喑啞人士能更自如地與外界溝通，本研究發展了一套結合微軟Kinect控制器及支持向量機的阿拉伯數字與英文字母手勢辨識系統。首先，我們透過Kinect獲取的骨架資訊找出手掌位置，接著我們使用兩種方式取得手勢影像，接著使用Otsu二值化、深度閥值、中值濾波器等進行影像前處理，切割出手掌區域，最後抽取出SURF及水平/垂直投影像素累加得到的形狀與面積特徵，以得到所需的訓練特徵資料，進行支持向量機模型訓練與測試實驗。此系統在使用者手掌距離攝影機600mm與1500mm之間及不同背景亮度的環境中，針對10個阿拉伯數字的靜態手勢可達到93.75%的辨識準確度，針對17個英文字母則有92.35%，實驗結果也顯示此系統對於手勢的旋轉、角度及不同的手掌大小所造成的變異性有著不錯的容忍度。

To help the hearing impaired easily communicate with others, this research aims to enhance accuracy, reliability and convenience of sign language recognition by combining Microsoft Kinect system, image processing and support vector machine techniques. In the experiments, we first derived the depth image through Kinect's built-in functionality, then image pre-processing techniques such as median filtering, Otsu binarization and depth threshold are used to further extract the specific palm area of interest, finally SURF and vertically/horizontally projected depth pixel integration shape/area features are extracted for Support Vector Machine model training and testing. In the environment of various ambient lighting conditions and camera-user distance ranging from 600(mm) to 1,500(mm), our recognition system can reach up to 93.75% of accuracy for 10 numeric digit gesture recognition, and 92.35% for 17 letters in the English alphabet. The experimental results also show this system have a good tolerance to variations caused by different palm sizes and orientations.

致謝 I
摘要 II
Abstract III
目錄 IV
圖目錄 VI
表目錄 VIII
第一章 緒論 1
1-1 研究動機 1
1-2 研究目的 2
1-3 論文架構 3
第二章 文獻探討 4
2-1 手語辨識文獻探討 4
2-2 基於Kinect的手語辨識文獻探討 5
2-3 Kinect規格與原理 7
2-4 Otsu二值化演算法 10
2-5 加速穩健特徵 (Speeded Up Robust Features) 12
2-6 雜訊去除與平滑化 16
2-7 LibSVM支持向量機 18
第三章 研究方法 19
3-1 系統流程與架構 19
3-2 深度資料 21
3-3 骨架資料 24
3-4 擷取手掌影像 25
3-4-1 使用Otsu擷取手掌 25
3-4-2 使用深度閥值擷取手掌 27
3-5 平滑化處理 28
3-6 SURF特徵擷取 29
3-7 水平及垂直投影特徵 31
3-8 LibSVM手語辨識 32
第四章 實驗結果及分析 34
4-1 系統環境 34
4-2 輸入資料 35
4-3 數字辨識結果 38
4-4 英文辨識結果 42
4-5 距離與辨識率之關係 45
4-6 實驗系統對於背景與光源的容忍度 47
4-7 討論影像的大小與旋轉是否會影響辨識率 50
第五章 結論與建議 53
5-1 結論 53
5-2 建議與未來展望 54
參考文獻 55

[1] 張光寒，3D台灣手語辨識系統，南台科技大學電機工程研究所碩士論文，2007
[2] X. Y. Cao, H. F. Liu, Y. Y. Zou,“Gesture Segmentation Based on Monocular Vision Using Skin Color and Motion Cues＂IEEE Conference on Image Analysis and Signal Processing (IASP), p.358-362, 2010
[3] 李健銘，即時手語辨識，國立成功大學工程科學系碩士論文，2010
[4] 陳冠廷、謝景棠、張量鈞、郭泰谷，基於Kinect之部偵測與追蹤系統，2011資訊技術應用與管理研討會，淡江大學電機工程學系，2011
[5] 蕭怡涵，基於Kinect之台灣手語單字辨識，國立師範大學資訊工程研究所碩士論文，2013
[6] kinect感應器組成與規格https://msdn.microsoft.com/en-us/library/jj131033.aspx
[7] N. Otsu, "A Threshold Selection Method from gray-level Histograms, " IEEE International Conference on System, vol. 9, pp. 62–66, 1979
[8] H Bay, A. Ess, T. Tuytelaars, L. V. Gool, "SURF: Speeded Up Robust Features", Computer Vision and Image Understanding (CVIU), Vol. 110, No. 3, pp. 346--359, 2008
[9] J. L. Raheja, A. Chaudhary, K. Singal, "Tracking of Fingertips and Centres of Palm using KINECT" , 2011 Third International Conference on Computational Intelligence, Modelling & Simulation, 2011
[10] Simon Lang, Marco Block, Raul Rojas, “Sign Language Recognition Using Kinect” Proceeding of the Artifical Intelligence and Soft Computing, pp.394-402, 2012
[11] T Kuroda, Y Tabata, A Goto, H Ikuta and M Murakami, "Consumer Price Data-Glove for Sign Language Recognition" , Proc. 5th Intl Conf. Disability, Virtual Reality & Assoc. Tech., 2004
[12] C. S. Fahn, H. Sun, "Development of a Data Glove With Reducing Sensors Based on Magnetic Induction" , IEEE Ｔtransactions on Industrial Electronics, VOL. 52, No. 2, 2005
[13] C. C. Chang, C. J. Lin. “LIBSVM : a library for support vector machines”. Journal of Machine Learning Research 9(2008), 1871-1874, 2008
[14] 台灣衛生福利部統計處－身心障礙者統計報表http://www.mohw.gov.tw/cht/DOS/Statistic.aspx?f_list_no=312&fod_list_no=4198
[15] H. D. Yang, “Sign Language Recognition with the Kinect Sensor Based on Conditional Random Fields”, ISSN 1424-8220, 2015
[16] 黃祐翔，機器人之影像導引控制系統研究，大同大學機械工程碩士論文，2011
[17] NAVI http://hci.uni-konstanz.de/blog/2011/03/15/navi/?lang=en
[18] T. Murata and J. Shin, “Hand Gesture and Character Recognition Based on Kinect Sensor,” International Journal of Distributed Sensor Networks, Vol. 2014, pp. 1-7, 2014
[19] 官方Kinect架構介紹https://msdn.microsoft.com/en-us/library/jj131023.aspx
[20] Otsu二值化介紹https://dotblogs.com.tw/dragon229/2012/12/28/86092
[21] D. G. Lowe, “Distinctive image features from scale-invariant keypoints.” International Journal of Computer Vision, Vol. 60,no. 2, pp. 91–110, 2004
[22] Kinect骨架追蹤https://msdn.microsoft.com/en-us/library/hh973074.aspx
[23] Kinect追蹤模式https://msdn.microsoft.com/en-us/library/hh973077.aspx
[24] Z. Feng, S. Xu, X. Zhang, L. Jin, Z. Ye, and W. Yang, "Real-time fingertip tracking and detection using Kinect depth sensor for a new writing-in-the air system," Proceedings of the International Conference on Internet Multimedia Computing and Service, pp. 70-74, 2012
[25] Kinecthttp://www.techbang.com/posts/2936-get-to-know-how-it-works-kinect
[26] SURF原理介紹https://read01.com/QLayA.html
[27] 林上良，運用Kinect於手勢辨識之研究，國立台北護理健康大學資訊管理碩士論文，2014
[28] S. Goyal, I. Sharma S. Sharma, “Sign Language Recognition System For Deaf And Dumb People”, International Journal of Engineering Research & Technology, Vol. 2 Issue 4, 2013
[29] LibSVM 工具使用說明與下載http://www.csie.ntu.edu.tw/%7Ecjlin/libsvm/
[30] SR4500 Time-of-Flight 規格http://hptg.com/industrial/