臺灣博碩士論文加值系統

在本論文中，我們提出一個可以在單純背景下辨識連續手勢的系統。由於同一個手勢對於不同的人而言，其手形存在很大的變異性，因此我們為了完成一個合理的辨識系統，我們只分析隨時間變化的二維手形及其移動的軌跡，來與未知的輸入手形做比對，而得知此輸入之可能手勢。整個系統包含手形追蹤及特徵萃取、建立手勢模型及以隱藏式馬可夫模型做辨識。
在單純背景下，我們所要的手形是被限制在由兩隻移動的手在不同活動範圍所構成的手勢動作。手的形狀不同及移動軌跡不同皆代表不同的意義。因此要辨識一個手勢，我們必須分析手的形狀及追蹤其運動方向。為了能夠萃取出手形，我們假設連續兩張畫面手勢並非靜止不動的。因此我們可以用運動資訊來萃取出手形，為了去除不必要的背景雜訊以使追蹤更加準確，我們利用雜訊清除器來消除不具手形特徵的雜訊。
萃取出手形後，我們以傅立葉描述器來表示手形特徵。在我們的系統，我們只需用前22項的傅立葉係數來描述各種手形。由於手勢變化太大，因此我們將每一個手勢做單一考慮，將每張畫面得到的傅立葉係數經由 Vector quantitation 分類後，獲得觀察序列，然後我們利用隱藏馬可夫模型來分析：狀態的初始機率分佈、狀態與狀態間的關係，即狀態轉換之機率分佈和狀態與系統間的關係，即觀察機率分佈。統計出每個手勢的最佳化參數，來建構每個手勢的隱藏式馬可夫模型 (HMM)。在本系統中，辨識時我們利用菲得比演算法來求出每個手勢模型的機率，我們取值最高的模型當成是我們的辨識結果。在整個系統，我們收集了10個人，共800組影像手勢序列來測試，其中含有20 種不同的手勢，本系統均能正確地辨識。

In this thesis, we introduce a hand gesture recognition system to recognize continuous gesture in simple background. The system consists of three modules: feature extraction, hidden Markov model (HMM) training, and gesture recognition using the HMMs. First, we apply the motion information to extract the hand-shape and apply the scale and rotation-invariant Fourier descriptor to characterize hand figures. Then we combine Fourier descriptor and motion information of input image sequence as our feature vector. After having extracted the feature vector, we first train our system using HMM approach and then use the trained HMMs to recognize the input gesture. In training phase, we apply hidden Markov Model to describe the gestures properties (generating the initial state probability distribution, the state transition probability distribution and the observation probability distribution) for each gesture. To recognize gesture, the gesture to be recognized in separately scored against different HMMs. The model with the highest score is selected as the recognized gesture. Our system consists of 20 different hand gestures. The experimental results show that the average recognition rate is 88.5%.

CHAPTER 14
INTRODUCTION4
CHAPTER 210
FEATURE EXTRACTION10
2.1 OBJECT EXTRACTION VIA MOTION INFORMATION10
2.2 OSTU THRESHOLDING METHOD15
2.3 FEATURE SELECTION FOR OBJECT DESCRIPTION17
2.3.1 FOURIER DESCRIPTOR18
2.3.2 PROPERTIES:21
2.3.3 MOTION TRAJECTORY23
2.4 FEATURE SELECTION FOR TWO-HAND GESTURE24
CHAPTER 327
HIDDEN MARKOV MODELS (HMMS) FOR GESTURE RECOGNITION27
3.1 VECTOR QUANTIZATION FOR SYMBOL GENERATION28
3.2 HIDDEN MARKOV MODELS31
3.3 ELEMENTS OF HIDDEN MARKOV MODELS34
3.4 THREE BASIC PROBLEMS FOR HMMS36
3.4.1 PROBABILITY EVALUATION USING THE FORWARD-BACKWARD PROCEDURE37
3.4.2 OPTIMAL STATE SEQUENCE USING THE VITERBI ALGORITHM41
3.4.3 PARAMETER ESTIMATION USING THE BAUM-WELCH METHOD43
CHAPTER 448
EXPERIMENTATION48
4.1 EXPERIMENTAL DATA COLLECTION48
4.2 EXPERIMENT RESULTS53
4.2.1 FOURIER DESCRIPTOR (FD) WITHOUT MOTION VECTOR55
4.2.2 FD AND MOTION VECTOR56
4.2.3 TWO-HAND GESTURES63
CHAPTER 565
CONCLUSION AND FURTHER WORK65
REFERENCES66

[1] V.I. Pavlovic, R. Sharma and T.S. Huang, "Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review", IEEE trans. on Pattern analysis and machine intelligence, Vol.19, No.7, pp.677-695, July 1997.
[2] T. Baudel and M. Baudouin-Lafon, "Charade: Remote Control of Objects Using Free-Hand Gestures", Comm. ACM, Vol.36, No.7, pp.28-35, 1993.
[3] D.J. Sturman and D. Zeltzer, "A Survey of Glove-Based Input", IEEE Computer Graphics and Applications, Vol.14, pp.30-39, Jan. 1994.
[4] T. Takahashi and F. Kishino, "A Hand Gesture Recognition Method and Its Application", Systems and Computers in Japan, Vol.23, No.3, pp.38-48, 1992.
[5] A. F. Bobick and A. D. Wilson, "A State-based Technique for the Summarization and Recognition of Gesture", Proc. Fifth International Conference on Computer Vision, pp.382-388, 1995.
[6] C. L. Huang and W. Y. Huang, "Sign language recognition using model-based tracking and a 3D Hopfield neural network", Machine Vision and Applications, Vol. 10, pp.292-307, 1998.
[7] J. Davis and M. Shah, "Visual Gesture Recognition", IEE Proc.-Vis. Image Signal Process, Vol.141, No.2, April 1994.
[8] T. Darrell and A. Pentland, "Recognition of Space-time Gestures using a Distributed Representation", MIT Media Laboratory Perceptual Computing Group Technical Report, No.197, 1992.
[9] T. Starner and A. Pentland, "Visual Recognition of American Sign Language using Hidden Markov Models", Proc. of Int. Workshop on Automatic Face- and Gesture-Recognition, Zurich, Switzerland, 1995.
[10] L. W. Campbell, D. A. Becker, A. Azarbayejani, A.F. Bobick, amd A. Plentland, "Invariant Features for 3-D Gesture Recognition", In Proc. IEEE 2nd Int. Workshop on Automatic Face and Gesture Recognition, 1996.
[11] J. Schlenzig, E. Hunter, and R. Jain, "Recursive identification of gesture inputers using hidden Markov models", Proc. Second Ann. Conf. On Appl. Of Comp. Vision, pp.187-194, 1994.
[12] R.H. Liang and M. Ouhyoung, "A Real-time Continuous Gesture Recognition system for Sign Language", In Proc. IEEE 2nd Int. Conf. on Automatic Face and Gesture Recognition, pp.558-565, 1998.
[13] Y. Cui and J. J. Weng, "Hand sign recognition from intensity image sequences with complex backgrounds", In Proc. IEEE 2nd Int. Conf. on Automatic Face and Gesture Recognition, 1996.
[14] A. Ohknishi, A. Nishikawa, and F. Miyazaki, "Curvature-based Segmentation and Recognition of Hand Gestures", In Proc. Annual Conf. On Robotics Society of Japan, pp. 401-401, 1997.
[15] S. Nagaya and S. Seki and R. Oka, "A Theoretical Consideration of Pattern Space Trajectory for Gesture Spotting Recognition", In Proc. IEEE 2nd Int. Workshop on Automatic Face and Gesture Recognition, 1996.
[16] T. Heap and D. Hogg, "Towards 3D hand tracking using a deformable model", In Proc. IEEE 2nd Int. Conf. on Automatic Face and Gesture Recognition, 1996.
[17] S. C. Zhu and A. L. Yuille, "FORMS: a Flexible Object Recognition and Modelling System", Proc. Fifth International Conference on Computer Vision, pp.465-472, 1995.
[18] N. Ostu, "A Threshold Selection Method from Gray-Level Histograms", IEEE Trans. on Systems, Man, and Cybernetics, Vol.SMC-9, pp.62-66, 1979.
[19] Zahn CT, Roskies RZ, "Fourier Descriptor for Plane Closed Curves", IEEE Trans SMC 21(3), pp.269-281, 1972.
[20] Persoon E, Fu KS, "Shape Discrimination Using Fourier Descriptor", IEEE Trans SMC 7(3), pp.170-179, 1977.
[21] Shridhar D, Badreldin A, "High-Accuracy Character Recognition Algorithm using Fourier and Topology Descriptors", Pattern Recognition 17, pp.515-524, 1984.
[22] B. H. Juang, and L. R. Rabiner, "Mixture Autoregressive Hidden Markov Models for Speech Signals", IEEE Trans. on Acoustics, Speech, and Signal Processing, Vol.ASSP-33, No.6, pp.1404-1413, 1985.
[23] A. Kundu, Y. He and P. Bahl, "Recognition of Handwritten Word: First and Second Order Hidden Markov Model Based Approach", Pattern Recognition, Vol.22, No.3, pp.283-297, 1989.
[24] J. Yamato, J. Ohya, and K. Ishii, "Recognizing Human Action in Time-Sequential Images Using Hidden Markov Model", Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp.379-385, 1992.
[25] J. Schlenzig, E. Hunter, and R. Jain, "Recursive Identification of Gesture Inputs Using Hidden Markov Models", Proc. 2nd. Ann. Conf. on Applications of Computer Vision, pp.187-194, 1994.
[26] Linde, Y., Buzo, A., and Gray, R., "An Algorithm for Vector Quantizer Design" IEEE Transaction on Communications, Vol. COM-28, NO. 1,pp.84-95, January 1980.
[27] L. R. Rabiner, S. E. Levinson, and M. M. Sondhi, "On the Application of Vector Quantization and Hidden Markov Models to Speaker-independent, Isolated Word Recognition", Bell System Tech. J., Vol.62, No.4, pp.1075-1105, 1983.
[28] Forney, G.D., "The Viterbi Algorithm," Proceedings of the IEEE, Vol. 48, pp.268-278, March 1973.
[29] Rabiner, L.R., "A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition," Proceedings of The IEEE, Vol.77, No.2, pp.257-285, February 1989.