臺灣博碩士論文加值系統

手勢是一種最自然且直觀的溝通方式，因此手勢辨識一直是研究領域上重要的一環。手語擁有一組有限且定義清楚的手勢，適合應用在手勢辨識方面。非靜態的手語由於雙手在三維空間任意移動的關係時常伴隨著遮蔽(occlusion)問題的衍生。遮蔽的問題在於當兩物體之位置於二維影像有部分重疊時，造成較後方的物體在視覺上無法完整呈現。台灣手語主要可以分成表情、手形跟軌跡三方面，在手形方面，遮蔽會造成萃取到不完整的手形資料，而不完整的資訊很容易影響到最後辨識的結果。
欲解決遮蔽會帶來的問題，可以嘗試利用形狀先驗模型來解決。在本論文中，我們主要使用到Chan-Vese提出的水平集模型來追蹤手形的變化，將未發生遮蔽的手形儲存起來當作形狀先驗資料。在遮蔽情形發生時，假設先驗資料水平集每一位置值的變化為一高斯模型,加入形狀先驗資料去驅使追蹤的輪廓向先驗模型的形狀靠近，取得在遮蔽時雙手可能的手形。另外，同時也針對此水平集模型當手移動太快會使的追蹤輪廓重疊部分太小而收斂失敗的情形做改善，主要利用已知的膚色資訊去校正此模型追蹤輪廓內的平均亮度。
實驗中選取了１２種會發生遮蔽情形的手語影帶作測試，結果顯示形狀先驗模型的確能使被遮蔽的不完整手形得到一定的恢復程度，並在辨識上有不錯的效能。

Recognition of a sign language, which is completely defined by a set of gestures a typical application of gesture recognition defined by a completely set of gestures. In recognition of dynamic signs, a difficulty arises when two moving hands in the acquired 2D image appear overlapped partially. The occurrence of occlusion yields incomplete contours of hand shapes, leading to poor recognition results.
In this study, we modified the Chan-Vese level set model for tracking and recovering the contours of moving hands. Recovery of incomplete contours was achieved by combining the modified level set model with shape priors, which were obtained primarily from unoccluded hand shapes in the previous images. Each shape was defined by a Gaussian model. As occlusion occurs, the evolving contour was pulled toward the desired shape by updating the parameters of Gaussian model of the shape prior. It is also noteworthy that a tracking process based on the Chan-Vese model may be aborted in response to a poor initial condition. The contour may shrink and vanish eventually if the initial contour does not cover the object of interest to some extent. We address the abortion problem by incorporating skin information into the interior average in the Chan-Vese model.
In the experiment, we chose 12 sign words associated with hand occlusion for test. The results show that the shape prior-based level set method can recover occluded shapes, and improve the performance on recognition.

1. Introduction 　　　　　　　　　　　　　　　　　　1
　　1.1 Motivation　　　　　　　　　　　　　　　　　　1
　　1.2 Objective 　　　　　　　　　　　　　　　　　　4
　　1.3 Organization 　　　　　　　　　　　　　　　　6
2. Related Work 　　　　　　　　　　　　　　　　　　8
　　2.1 Active Contour Model　　　　　　　　　　　　　8
　　2.2 Level Set Methods 　　　　　　　　　　　　　　10
　　　　2.2.1 Definition 　　　　　　　　　　　　10
　　　　2.2.2 Chan-Vese Model 　　　　　　　　　　12
　　　　2.2.3 Level Set Model without Re-initialization　　13
　　　　2.2.4 Narrow Band 　　　　　　　　　　　　14
3. Hand Shape Tracking 　　　　　　　　　　　　　　16
　　3.1 Initial Contour 　　　　　　　　　　　　　　　　16
　　3.2 Tracking Model　　　　　　　　　　　　　　　　18
　　3.3 Shape Recovery 　　　　　　　　　　　　　　21
　　　　3.3.1 Shape Prior Model 　　　　　　　　　　22
　　　　3.3.2 Shape Pre-processing 　　　　　　　　24
　　　　3.3.3 Energy Function 　　　　　　　　　　25
　　3.4 Hand Shape Representation 　　　　　　　　　　28
　　　　3.4.1 Fourier Descriptors 　　　　　　　　　　28
4. Experimental Results 　　　　　　　　　　　　　　31
　　4.1 Dataset Description 　　　　　　　　　　　　　　31
　　4.2 Hand Shape Recovery 　　　　　　　　　　　　32
　　4.3 Hand Shape Recognition 　　　　　　　　　　　　42
　　4.4 Discussion　　　　　　　　　　　　　　　　　　44
5. Conclusions 　　　　　　　　　　　　　　　　　　46
References 　　　　　　　　　　　　　　　　　　　　48

[1]J. A. V. Montero and L. E. S. Sucar, “Feature selection for visual gesture recognition using hidden Markov models,” in Proc. the fifth Mexican Int. Conf. Computer Science, pp. 196–203, Sept. 2004.
[2]L. Gupta and S. Ma, “Gesture-based interaction and communication: automated classification of hand gesture contours,” IEEE Trans. Systems, Man, and Cybernetics, vol. 31, no. 1, pp.114–120, Feb. 2001.
[3]V. I. Pavlovic, R. Sharma, and T. S. Huang, “Visual interpretation of hand gestures for human computer interaction: a review,” IEEE Trans. Pattern Analysis Machine Intelligence, vol. 19, no. 7, pp. 677–695, July 1997.
[4]J. Triesch and C. von der Malsburg, “A system for person-independent hand posture recognition against complex background,” IEEE Trans. Pattern Analysis Machine Intelligence, vol. 23, no. 12, pp.1449–1453, Dec. 1990.
[5]F. S. Chen, C. M. Fu, and C. L. Huang, “Hand gesture recognition using a real-time tracking method and hidden Markov models,” Image and Vision Computing, vol. 21, no. 8, pp. 745–758, Aug. 2003.
[6]K. Arbter, W. E. Snyder, H. Burkhardt, and G. Hirzinger, “Application of affine-invariant Fourier descriptors to recognition of 3-D objects,” IEEE Trans. Pattern Analysis Machine Intelligence, vol. 12, no. 7, pp. 640–647, July 1990.
[7]M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: active contour models,” Int. Journal Computer Vision, vol. 1, no. 4, pp. 321–331, 1987.
[8]V. M. Yedidand, et.al. “Active contours for the movement and motility analysis of biological objects,” in Proc. IEEE Int. Conf. on Image Processing, vol. 1, pp.196–199, 2000.
[9]C. Zimmer, E. Labruyère, V. Meas-Yedid, N. Guillén, and J. C. Olivo-Marin, “Segmentation and tracking of migrating cells in videomicroscopy with parametric active contours: a tool for cell-based drug testing,” IEEE Trans. Medical Imaging, vol. 21, no. 10,pp. 1212–1221, Oct. 2002.
[10]T. F. Chan and L. A. Vese, “Active contours without edges,” IEEE Trans. Image. Processing, vol. 10, no. 2, pp. 266–277, Feb. 2001.
[11]Y. Fu, A. T. Erdem, and A. M. Tekalp, “Tracking visible boundary of objects using occlusion adaptive motion snake,” IEEE Trans. Image Processing, vol. 9, no. 12, pp. 2051–2060, Dec. 2000.
[12]K. H. Tan, R. S. Feris, M. Turk, J. Kobler, J. Yu; and R. Raskar, “Harnessing real-world depth edges with multi-flash imaging,” IEEE Computer Graphics and Applications, vol. 25, no. 1, pp. 32–38, Jan. –Feb. 2005.
[13]A. Ghosh and N. Petkov, “Robustness of shape descriptors to incomplete contour representations,” IEEE Trans. Pattern Analysis Machine Intelligence, vol. 27, no. 11, pp. 1793–1804, Nov. 2005.
[14]H. T. Nguyen and A. W.M. Smeulders, “Fast occluded object tracking by a robust appearance filter,” IEEE Trans. Pattern Analysis Machine Intelligence, vol. 26, no. 8, pp. 1099–1104, Aug. 2004.
[15]T. Chan, W. Zhu, and S. Esedoglu, “Segmentation with depth: a level set approach,” SIAM Journal on Scientific Computing, vol. 28, no. 5, pp.1957–1973, Sept. 2006.
[16]T. Chan and W. Zhu, “Level set based shape prior segmentation,” in Proc. CVPR’05, vol. 2, pp. 1164–1170, June 2005.
[17]A. Yilmaz, X. Li, and M. Shah, “Contour-based object tracking with occlusion handling in video acquired using mobile cameras,” IEEE Trans. Pattern Analysis Machine Intelligence, vol. 26, no. 11, pp. 1531–1536, Nov. 2004.
[18]R. Malladi, J. A. Sethian, and B. C. Vemuri, “Shape modeling with front propagation: a level set approach,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 2, pp. 158–175, Feb. 1995.
[19]C. Xu and J. L. Prince, “Snakes, shapes and gradient vector flow,” IEEE Trans. Image Processing, vol. 7, no. 3, pp. 359–369, March 1998.
[20]A. A. Amini, S. Tehrani, and T. E. Weymouth, “Using dynamic programming for minimizing the energy of active contours in the presence of hard constraints,” in Proc. IEEE Second Int. Conf. Computer Vision, pp. 95–99, Dec. 1988.
[21]A. A. Amini, T. E. Weymouth, and R. C. Jain, “Using dynamic programming for solving variational problems in vision,” IEEE Trans. Pattern Analysis Machine Intelligence, vol. 12, no. 9, pp. 855–867, Sept. 1990.
[22]V. Caselles, R. Kimmel, and G. Sapiro, “Geodesic active contours,” Int. Journal of Computer Vision, vol. 22, no. 1, pp. 61–79, Feb. 1997.
[23]C. Li, C. Xu, C. Gui, and M. D. Fox, “Level set evolution without re-initialization: a new variational formulation,” in Proc. IEEE Int. Conf. Computer and Pattern Recognition, vol. 1, pp. 430–436, June 2005.
[24]D. Adalsteinsson and J. A. Sethian, “A fast level set method for propagating interfaces,” Journal of Computational Physics, vol. 118, no. 2, pp.269–277, May 1995.
[25]R. L. Hsu, M. Abdel-Mottaleb, and A. K. Jain, “Face detection in color image,” IEEE Trans. Pattern Analysis Machine Intelligence, vol. 24, no. 5, pp.696–706, May 2002.
[26]史文漢、丁立芬，手能生橋，第一冊~第二冊，中華民國聾人協會發行，2004.