臺灣博碩士論文加值系統

在桌球的領域中，人們依靠人眼觀察運動員的動作，進而分析其使用的招式與打法風格。然而桌球是一項技巧性相當高的運動，其中所使用的招式千變萬化，需要足夠的背景知識與經驗才能夠做出正確的判斷，這樣的情形使桌球運動的情報蒐集更具有難度，同時讓許多不熟悉桌球運動的觀眾難以分辨這些招式，進而造成觀眾無法體會一場精彩比賽的箇中奧妙。因此如何使用電腦視覺技術辨識桌球招式就變得十分的重要。
過去的動作辨識僅針對人體影像做特徵擷取及辨識，但是桌球招式與一般的人體動作不同，有些招式動作相似而球性不同，有些招式球性相似而動作不同，傳統的動作辨識做法應用在桌球招式辨識時，單從人體動作資訊辨識桌球招式容易發生動作相似的招式難以分辨的情形。
本篇論文提出使用耦合隱藏式馬可夫模型(Coupled Hidden Markov Model)結合人體及球體資訊做桌球的招式辨識。系統分成三個部分：球體動作特徵擷取、人體動作特徵擷取以及桌球招式辨識。
球體動作特徵擷取的部分，為了還原出球體在三維空間下的位置，首先針對兩台DV做校正，取得攝影機的內部參數以及外部參數，接著分別針對兩台DV所拍攝到的畫面做球體追蹤，最後將追蹤結果取得的座標還原成三維座標，並擷取球體動作特徵。人體動作特徵擷取的部分，本論文使用Kinect追蹤人體骨架，取持拍手的四個關節點，擷取其座標位置、速度、加速度以及手臂的夾角做人體動作特徵。桌球招式辨識的部分，本論文使用耦合隱藏式馬可夫模型辨識十種桌球招式。實驗結果顯示，使用耦合隱藏式馬可夫模型結合人體及球體資訊可以達到比僅使用單一資訊之隱藏式馬可夫模型更高的平均辨識率。

Table tennis is a highly skilled sport, and recognizing the players’ skill through human eyes needs sufficient background knowledge and experience. This makes the table tennis intelligence gathering process difficult. Furthermore, the audiences who are unfamiliar with table tennis are hard to distinguish the skills and hard to realize how wonderful the play is. Therefore, how to recognize the table tennis skills is a very important issue.
Most of the previous posture recognition approaches were focusing on detecting and analyzing the human features. In the experimental examples of such approaches, the differences among human postures are large. In contrast to the traditional posture recognition, the recognition of table tennis skills is a far more complex task in which some skills have similar body motions with different ball characteristics, but some skills have similar ball characteristics with different body motion. Using only one of the body motion information and ball motion information is hard to distinguish the table tennis skills. In order to recognize the skills accurately, we consider both the ball motion information and body motion information.
This paper proposes a table tennis skill recognition system based on coupled Hidden Markov Models (coupled HMMs) combining body motion information and ball motion information. The system consists of three parts: ball motion feature extraction, skeleton motion feature extraction and table tennis skill recognition.
In the ball motion feature extraction, we calibrate the two-camera system and the 3D ball trajectory is reconstructed from 2D ball positions in sequential frames through triangulation. The ball motion features are extracted from the 3D trajectory. In the skeleton motion extraction, we track the skeletons of the player by using Kinect. The skeleton motion features of four joints of the arm holding the racket are extracted. In the table tennis skill recognition, we use the coupled HMMs to recognize ten table tennis skills. The experimental results show that the coupled HMMs, which combine the body motion features and ball motion features, have the better average recognition rate than conventional single feature HMMs in table tennis skill recognition.

致謝 i
摘要 ii
Abstract iii
Contents v
List of Figures vi
List of Tables vii
Chapter 1. Introduction 1
1.1 Motivation 1
1.2 Related Works 2
1.3 System Overview 4
Chapter 2. Feature Extraction 7
2.1 Skeleton Motion Feature Extraction 7
2.2 Ball Motion Feature Extraction 8
2.2.1 Camera Calibration 9
2.2.2 Ball Tracking 11
Chapter 3. Table Tennis Skill Recognition 13
3.1 Recognition Based on HMMs 13
3.2 Table Tennis Skill Recognition Based on Coupled-HMMs 17
Chapter 4. Experimental Results 21
4.1 Environment Setting 21
4.2 Table Tennis Skill Database Construction 22
4.3 Features Selection 24
4.4 State Selection 26
4.5 Recognition Result 27
Chapter 5. Conclusions and Future Works 31
References 32

[1] L. Sun, J. Liu, Y. Wang, L. Zhou, Q. Yang, S. He, "Ball's flight trajectory prediction for table-tennis game by humanoid robot", IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 2379-2384, 19-23 Dec. 2009.
[2] Y. Wang, L. Sun, J. Liu, Q. Yang, L. Zhou, S. He, "A novel trajectory prediction approach for table-tennis robot based on nonlinear output feedback observer", IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 1136-1141, 14-18 Dec. 2010.
[3] X. Chen, Y. Tian, Q. Huang, W. Zhang, Z. Yu, "Dynamic model based ball trajectory prediction for a robot ping-pong player", IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 603-608, 14-18 Dec. 2010.
[4] X. Chen, Q. Huang, W. Zhang, Z. Yu, R. Li, P. Lv, "Ping-pong trajectory perception and prediction by a PC based High speed four-camera vision system", World Congress on Intelligent Control and Automation (WCICA), pp. 1087-1092, 21-25 June 2011.
[5] A. Nakashima, Y. Kobayashi, Y. Ogawa, Y. Hayakawa, "Modeling of rebound phenomenon between ball and racket rubber with spinning effect", ICCAS-SICE, pp. 2295-2300, 18-21 Aug. 2009.
[6] A. Nakashima, Y. Ogawa, Y. Kobayashi, Y. Hayakawa, "Modeling of rebound phenomenon of a rigid ball with friction and elastic effects", American Control Conference (ACC), pp. 1410-1415, June 30 2010-July 2 2010.
[7] J. Nonomura, A. Nakashima, Y. Hayakawa, "Analysis of effects of rebounds and aerodynamics for trajectory of table tennis ball", Proceedings of SICE Annual Conference, pp.1567-1572, 18-21 Aug. 2010.
[8] Z. Zhang, D. Xu, P. Yang, "Rebound model of table tennis ball for trajectory prediction", IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 376-380, 14-18 Dec. 2010.
[9] S. Furuno, K. Kobayashi, T. Okubo, Y. Kurihara, "A study on spin-rate measurement using a uniquely marked moving ball", ICCAS-SICE, pp. 3439-3442, 18-21 Aug. 2009.
[10] L.Y. Duan, M. Xu, Q. Tian, C.S. Xu, J.S. Jin, "A unified framework for semantic shot classification in sports video", IEEE Transactions on Multimedia, vol.7, no.6, pp. 1066-1083, Dec. 2005.
[11] B. Zhang, W. Dou, L. Chen, "Audio Content-based Highlight Detection Using Adaptive Hidden Markov Model", Sixth International Conference on Intelligent Systems Design and Applications (ISDA), Vol.1, pp.798-803, 16-18 Oct. 2006.
[12] W. Chen, Y.J. Zhang, "Tracking Ball and Players with Applications to Highlight Ranking of Broadcasting Table Tennis Video", IMACS Multiconference on Computational Engineering in Systems Applications, Vol.2, pp. 1896-1903, 4-6 Oct. 2006.
[13] F.B. Vidal, V.H.C. Alcalde, "Window-Matching Techniques with Kalman Filtering for an Improved Object Visual Tracking", IEEE International Conference on Automation Science and Engineering, CASE 2007, pp. 829-834, 22-25 Sept. 2007.
[14] J. Sun, J. Tian, Y. Du, Y. Tang, "Fast Ball Detection Method for Ping-Pong Playing Robots", Second International Symposium on Information Science and Engineering (ISISE), pp. 339-343, 26-28 Dec. 2009.
[15] Z. Zhang, D. Xu, "Design of high-speed vision system and algorithms based on distributed parallel processing architecture for target tracking", Asian Control Conference, ASCC 2009, pp. 1638-1643, 27-29 Aug, 2009.
[16] J.Y. Kim, T.Y. Kim, "Soccer Ball Tracking Using Dynamic Kalman Filter with Velocity Control", Sixth International Conference on Computer Graphics, Imaging and Visualization, CGIV '09, pp. 367-374, 11-14 Aug. 2009.
[17] A. Poliakov, D. Marraud, L. Reithler, C. Chatain, "Physics based 3D ball tracking for tennis videos", International Workshop on Content-Based Multimedia Indexing (CBMI), pp. 1-6, 23-25 June 2010.
[18] C. Zhao, B. Zhang, J. Lian, J. He, T. Lin, X. Zhang , "Classification of Driving Postures by Support Vector Machines", Sixth International Conference on Image and Graphics (ICIG), pp. 926-930, 12-15 Aug, 2011.
[19] F. Xie, G. Xu, Y. Cheng, Y. Tian, "Human body and posture recognition system based on an improved thinning algorithm", Image Processing, IET , Vol. 5, No. 5, pp. 420-428, August 2011.
[20] N.H. Dardas, N.D. Georganas, "Real-Time Hand Gesture Detection and Recognition Using Bag-of-Features and Support Vector Machine Techniques", IEEE Transactions on Instrumentation and Measurement, Vol. 60, No. 11, pp. 3592-3607, Nov. 2011.
[21] D. Kelly, J. McDonald, C. Markham, "Weakly Supervised Training of a Sign Language Recognition System Using Multiple Instance Learning Density Matrices", IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, Vol. 41, No. 2, pp. 526-541, April 2011.
[22] T. Starner, J. Weaver, A. Pentland, "Real-time American sign language recognition using desk and wearable computer based video", Pattern Analysis and Machine Intelligence, Vol. 20, No. 12, pp. 1371-1375, Dec 1998.
[23] B. Bauer, K.F. Kraiss, “Towards an automatic sign language recognition system using subunits”, Proc. Revised Papers Int. Gesture Workshop Gesture Sign Lang. Human-Comput. Interaction GW, London, U.K., pp. 64-75, 2002.
[24] B. Bauer, H. Hienz, K-L. Kraiss, “Video-based continuous sign language recognition using statistical methods”, Int’l Conf. Pattern Recognition, 2000.
[25] C. Wang, S. Shan, W. Gao, “An approach based on phonemes to large vocabulary Chinese sign language recognition”, Proc. IEEE FG, Washington, DC, pp. 411, 2002.
[26] M. Brand, N. Oliver, A. Pentland, “Coupled hidden markov models for complex action recognition”, IEEE Conf. on Computer Vision and Pattern Recognition, 1997.
[27] A. V. Nefian, L. Liang, X. Pi, X. Liu, C. Mao, and K. Murphy, “A coupled HMM for audio-visual speech recognition”, Proc. 27th Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP’ 02), pp. 2013-2016, 2002.
[28] L. Ma, Q. Zhou, M. Celenk, D. Chelberg, "Facial event mining using coupled hidden Markov models", International Conference on Image Processing, ICIP '04, Vol. 2, pp. 1405-1408, 24-27 Oct. 2004.
[29] J.C. Lin, C.H. Wu, W.L. Wei, "Error Weighted Semi-Coupled Hidden Markov Model for Audio-Visual Emotion Recognition", IEEE Transactions on Multimedia, Vol.14, No.1, pp.142-156, Feb. 2012.
[30] H. Ohya, H. Saito, “Tracking Racket Face in Tennis Serve Motion Using High Speed Multiple Camera”, Proceedings of the IEICE General Conference, 2004.
[31] S. Rusdorf, G. Brunnett, “Real time tracking of high speed movements in the context of a table tennis application”, Proceedings of the ACM symposium on virtual reality software and technology, ACM Press, New York, pp. 192-200, 2005.
[32] T. Susnjak, "Real-time Video Segmentation and Object Tracking in Tennis", http://www.massey.ac.nz/~albarcza/ResearchFiles/TeoSusnjak_HonoursReport07.pdf, 2007.
[33] M. Viager, “Analysis of kinect for mobile robots”, Technical report, Department of Electrical Engineering, Technical University of Denmark, March 2011.
[34] J. Shotton, A. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio, R. Moore, A. Kipman, A. Blake, "Real-time human pose recognition in parts from single depth images", 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1297-1304, 20-25 June 2011.
[35] L. Rabiner, “A tutorial on hidden markov model and selected applications in speech”, Proceedings of the IEEE, Vol. 77, No. 2, pp. 257-285, 1989.