(3.235.236.13) 您好!臺灣時間:2021/05/15 04:39
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果

詳目顯示:::

我願授權國圖
: 
twitterline
研究生:李哲緯
研究生(外文):Jhe-WeiLi
論文名稱:基於卷積神經網路之手語影像辨識
論文名稱(外文):Sign Language Recognition Based on Convolutional Neural Networks
指導教授:胡敏君
指導教授(外文):Min-Chun Hu
學位類別:碩士
校院名稱:國立成功大學
系所名稱:資訊工程學系
學門:工程學門
學類:電資工程學類
論文種類:學術論文
論文出版年:2016
畢業學年度:104
語文別:英文
論文頁數:35
中文關鍵詞:卷積神經網路深度學習手語辨識手部分割支持向量機
外文關鍵詞:convolutional neural networkdeep learningsign language recognitionhand segmentationsupport vector machine
相關次數:
  • 被引用被引用:0
  • 點閱點閱:593
  • 評分評分:
  • 下載下載:0
  • 收藏至我的研究室書目清單書目收藏:2
在這篇論文中我們提出了一個使用卷積神經網路並基於視覺的手語辨識系統,該系統利用膚色模型對影像進行完整的手部區域分割,在膚色偵測之後,我們套入形態學運算來優化偵測結果,並採用區域成長演算法對每個區域標上編號及統計,最後切出屬於手部的區域。此方法在一定程度上不會受到不同背景環境的影響。接著我們透過一個經過微調的卷積神經網路模型來對手部影像萃取特徵,並利用支持向量機來將手勢進行分類。實驗部分,我們透過自己拍攝的中文手語資料集以及公開的美國手語資料集來對系統辨識的效果進行評估,中文手語資料集是由五個人在三種不同環境下 (包含複雜背景) 比出二十六種手勢,並將其錄製而成;而美國手語資料集包含了同一種手勢在不同燈光條件下拍攝而成。實驗結果得到證實該方法比現今其他傳統方法的辨識能力更好。
In this paper, we propose a vision-based sign language recognition system using convolutional neural networks (CNN). We design an skin color modeling method for hand segmentation so that the hand region can be derived accurately even when different users use our system in various background scenes. A feature descriptor is extracted from the fully-connected layer of the fine-tuned CNN model as the representation of a hand gesture, and Support Vector Machine (SVM) is applied to classify hand gestures.
Our recognition method is evaluated by two datasets: (1) The Chinese Sign Language (CSL) dataset collected by ourselves, in which images of 26 gestures are captured from 5 people in three kinds of environments including complex background scenes. (2) The American Sign Language (ASL) dataset, in which images of the same gesture were captured in different lighting conditions. The proposed recognition method achieves high accuracy rates, and is proved to be greater than the results of traditional methods.
Abstract (Chinese) i
Abstract (English) ii
Acknowledgments iii
Table of Contents v
List of Tables vii
List of Figures viii
Chapter 1. Introduction 1
Chapter 2. Related Work 4
Chapter 3. Convolutional Neural Networks 7
3.1 Introduction of NNs and CNNs . . . . . . . . . . . . . . . . . . . . . . 7
3.2 CNN Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.3 Training of CNNs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Chapter 4. System Framework 11
4.1 Key Frame Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4.2 Hand Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
4.2.1 Skin Color Detection . . . . . . . . . . . . . . . . . . . . . . . . 12
4.2.2 Region Growing Algorithm . . . . . . . . . . . . . . . . . . . . 13
4.3 Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.3.1 Traditional Image Features . . . . . . . . . . . . . . . . . . . . . 14
4.3.2 CNN Image Features . . . . . . . . . . . . . . . . . . . . . . . . 18
4.4 SVM Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.4.1 Binary Classification . . . . . . . . . . . . . . . . . . . . . . . . 19
4.4.2 Multiclass Classification . . . . . . . . . . . . . . . . . . . . . . 21
Chapter 5. Experimental Results 22
5.1 Datasets and Experimental Environments . . . . . . . . . . . . . . . . . 22
5.2 Evaluation of CNN Features from Layers . . . . . . . . . . . . . . . . . 24
5.3 Comparison of Re-training and Fine-tuning CNN Models . . . . . . . . 24
5.4 Comparison of Background Subtraction . . . . . . . . . . . . . . . . . . 25
5.5 Comparison of Traditional and CNN Features . . . . . . . . . . . . . . 27
5.6 Comparison of DSAE and CNN Features . . . . . . . . . . . . . . . . . 29
Chapter 6. Conclusions & Future Work 30
References 32
[1] R. Akmeliawati, M. P. L. Ooi, and Y. C. Kuang. Real-time malaysian sign language translation using colour segmentation and neural network. In 2007 IEEE Instrumentation & Measurement Technology Conference IMTC 2007, pages 1–6, 2007.
[2] A. L. C. Barczak, N. H. Reyes, M. Abastillas, A. Piccio, and T. Susnjak. A new 2d static hand gesture colour image dataset for asl gestures. Research Letters in the Information and Mathematical Sciences, 15:12–20, 2011.
[3] J. Bouvrie. Notes on convolutional neural networks. November 2006.
[4] H. Brashear, T. Starner, P. Lukowicz, and H. Junker. Using multiple sensors for mobile sign language recognition. In Wearable Computers, 2003. Proceedings. Seventh IEEE International Symposium on, pages 45–52, 2003.
[5] C.-C. Chang and C.-J. Lin. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1–27:27, 2011. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
[6] K. Chatfield, K. Simonyan, A. Vedaldi, and A. Zisserman. Return of the devil in the details: Delving deep into convolutional nets. CoRR, abs/1405.3531, 2014.
[7] N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), volume 1, pages 886–893 vol. 1, 2005.
[8] N. H. Dardas and N. D. Georganas. Real-time hand gesture detection and recognition using bag-of-features and support vector machine techniques. IEEE Transactions on Instrumentation and Measurement, 60(11):3592–3607, 2011.
[9] M. Fernando and J. Wijayanayaka. Low cost approach for real time sign language recognition. In 2013 IEEE 8th International Conference on Industrial and Information Systems, pages 637–642, 2013.
[10] H. M. Gamal, H. M. Abdul-Kader, and E. A. Sallam. Hand gesture recognition using fourier descriptors. In Computer Engineering Systems (ICCES), 2013 8th International Conference on, pages 274–279, Nov 2013.
[11] M. Geetha, C. Manjusha, P. Unnikrishnan, and R. Harikrishnan. A vision based dynamic gesture recognition of indian sign language on kinect based depth images. In Emerging Trends in Communication, Control, Signal Processing & Computing Applications (C2SPCA), 2013 International Conference on, pages 1–7, 2013.
[12] Itseez. Open source computer vision library. https://github.com/itseez/opencv, 2015.
[13] Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. Caffe: Convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093, 2014.
[14] F. Kai-ping and Y. Fang. Static hand gesture recognition based on hog characters and support vector machines. In Instrumentation and Measurement, Sensor Network and Automation (IMSNA), 2013 2nd International Symposium on, pages 936–938, 2013.
[15] M. Kolsch and M. Turk. Robust hand detection. In Automatic Face and Gesture Recognition, 2004. Proceedings. Sixth IEEE International Conference on, pages 614–619, 2004.
[16] A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, pages 1106–1114, 2012.
[17] V. Kumar, G. C. Nandi, and R. Kala. Static hand gesture recognition using stacked denoising sparse autoencoders. In Contemporary Computing (IC3), 2014 Seventh International Conference on, pages 99–104, 2014.
[18] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, Nov 1998.
[19] Y. Liu, Y. Yin, and S. Zhang. Hand gesture recognition based on hu moments in interaction of virtual reality. In Intelligent Human-Machine Systems and Cybernetics (IHMSC), 2012 4th International Conference on, volume 1, pages 145–148, Aug 2012.
[20] Y. Madhuri, G. Anitha, and M. Anburajan. Vision-based sign language translation device. In Information Communication and Embedded Systems (ICICES), 2013 International Conference on, pages 565–568, 2013.
[21] H. Ming-Kuei. Visual pattern recognition by moment invariants. IRE Transactions on Information Theory, 8(2):179–187, 1962.
[22] J. S. N. A. Abdul Rahim, C. W. Kit. Rgb-h-cbcr skin colour model for human face detection. In MMU International Symposium on Information & Communications Technologies (M2USIC), Petaling Jaya, Malaysia, 2006.
[23] J. Nagi, F. Ducatelle, G. A. D. Caro, D. Cireşan, U. Meier, A. Giusti, F. Nagi, J. Schmidhuber, and L. M. Gambardella. Max-pooling convolutional neural networks for vision-based hand gesture recognition. In Signal and Image Processing Applications (ICSIPA), 2011 IEEE International Conference on, pages 342–347, Nov 2011.
[24] L. Prasuhn, Y. Oyamada, Y. Mochizuki, and H. Ishikawa. A hog-based hand gesture recognition system on a mobile device. In Image Processing (ICIP), 2014 IEEE International Conference on, pages 3973–3977, 2014.
[25] D. Rumelhart, G. Hintont, and R. Williams. Learning representations by backpropagating errors. Nature, 323(6088):533–536, 1986.
[26] C. Sun, T. Zhang, B. K. Bao, C. Xu, and T. Mei. Discriminative exemplar coding for sign language recognition with kinect. IEEE Transactions on Cybernetics, 43(5):1418–1428, 2013.
[27] M. Tariq, A. Iqbal, A. Zahid, Z. Iqbal, and J. Akhtar. Sign language localization: Learning to eliminate language dialects. In Multitopic Conference (INMIC), 2012 15th International, pages 17–22, 2012.
[28] R. Wang, D. Kong, L. Wang, and B. Yin. Gestures’ smooth motion for chinese sign language synthesis based on video joining. In Computational Intelligence and Software Engineering, 2009. CiSE 2009. International Conference
on, pages 1–5, 2009.
[29] T. Yamashita and T. Watasue. Hand posture recognition based on bottom-up structured deep convolutional neural network with curriculum learning. In 2014 IEEE International Conference on Image Processing (ICIP), pages 853–857, Oct 2014.
[30] Q. Yang. Chinese sign language recognition based on video sequence appearance modeling. In Industrial Electronics and Applications (ICIEA), 2010 the 5th IEEE Conference on, pages 1537–1542, 2010.
[31] C. Yixin, B. Jinbo, and J. Z. Wang. Miles: Multiple-instance learning via embedded instance selection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(12):1931–1947, 2006.
連結至畢業學校之論文網頁點我開啟連結
註: 此連結為研究生畢業學校所提供,不一定有電子全文可供下載,若連結有誤,請點選上方之〝勘誤回報〞功能,我們會盡快修正,謝謝!
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top