臺灣博碩士論文加值系統

近年來，除了以傳統的上課學習方式，多媒體教材在各種運動或舞
蹈學習上被廣為使用，人們利用多媒體容易取得的特性，在家中可以
隨時隨地自主練習。然而，多媒體教材雖然方便，卻無法達到像真人
教練引導時的互動性，學習者往往要手動調整多媒體的播放進度來反
覆練習，因為多媒體的播放速度往往是單一且沒有彈性的。另一方面，光靠影片的示範也很難確保所習得的動作之細節及正確性。
為了解決上述問題，我們提出了一套利用姿勢調控多媒體播放進度
之系統：「觀自在」播放器。利用搭載三軸加速器之智慧手錶來感測手部動作資訊辨識動作，依據所計算出來的姿勢速度與進度，來調整學習影片播放之速度，以達到影片隨使用者練習狀態變化而產生對應調整之效果。
在實驗及使用者試用回饋中，我們請使用者依據不同的動作速度測
試動作完整度之辨識，並且得到相當高之正確率。不只如此，在受測
者使用即時影片回饋系統時，使用者認為影片正確隨姿勢調控速度的
時間至少佔七成以上的練習時間，我們的實驗結果顯示我們提出的方
法，能夠達到利用姿勢調控影片之效果。

In addition to the traditional way of learning, multimedia learning materi-als are widely used in training of various kinds of exercises and dancing. With the accessibility of these materials, people can do the training any time and any where. Despite the fact that learning by using multimedia is convenient (such as watching videos), the interaction with teacher in training process is hard to be simulated. Learners usually need to manually adjust the playback progress and repeat it again and again since the monotonous and lack of flex-ibility of video. On the other hand, it is difficult to confirm the correctness and details of gestures the user learned.
In order to solve above problems, we proposed the gesture-mediated mul-timedia player application, ”Follow-Me”, to learning Tai Chi Chuan, which built up with accelerometer-enabled smart watches and commercial mobile devices. It provided an interaction between user and multimedia according to progress of user’s hand gesture. We applied an incomplete time series match-ing method to get the progress ,completeness of gestures and fulfil automatic segmentation. The video playback design is based on the automatic segmen-tation to reach the goal of mediating video content with the alteration of gestures.
In experiments and user study, we asked users to perform gestures in var-ious levels of speed to evaluate the relative error time and percentage error of progress prediction. The result demonstrated low percentage error was achieved. Furthermore, users gave us some positive feedback toward our real-time video feedback system. A percentage of 71% was reported when we questioned participants about how much time they felt the adjustment of video according to gestures was correct. Our results show the effectiveness of the gesture-mediated method.

口試委員審定書 i
致謝 ii
中文摘要 iii
Abstract iv
Contents vi
List of Figures viii
List of Tables ix
1 Introduction 1
2 Related work 4
2.1 Tai Chi Chuan training system . . . . . . . . . . . . . . . . . . . . . . . 4
2.2 Wrist-worn accelerometer gesture recognition . . . . . . . . . . . . . . . 5
2.3 Incomplete gesture matching and segmentation . . . . . . . . . . . . . . 6
3 Hand gesture progress recognition and automatic segmentation 9
3.1 Progress recognition pipeline . . . . . . . . . . . . . . . . . . . . . . . . 11
3.1.1 Data acquisition . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.1.2 Data preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.1.3 Open-begin-end DTW . . . . . . . . . . . . . . . . . . . . . . . 11
3.2 Segmentation pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.2.1 Matching length prediction . . . . . . . . . . . . . . . . . . . . . 15
3.2.2 Gesture segmentation . . . . . . . . . . . . . . . . . . . . . . . . 15
4 Gesture-mediated multimedia player 17
4.1 System prototype . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.2 System design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
5 System evaluation 21
5.1 Data collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
5.2 Experiments and results . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
5.2.1 Progress prediction performance . . . . . . . . . . . . . . . . . . 23
5.3 User study and experience feedback . . . . . . . . . . . . . . . . . . . . 24
5.3.1 Wizard-of-Oz study . . . . . . . . . . . . . . . . . . . . . . . . . 24
5.3.2 User interview and feedback . . . . . . . . . . . . . . . . . . . . 26
5.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
6 Conclusion and future work 28
References 29

[1] Tai chi wizard. http://www.tai-chi-wizard.com/.
[2] Philo Tan Chua, Rebecca Crivella, Bo Daly, Ning Hu, Russ Schaaf, David Ventura, Todd Camill, Jessica Hodgins, and Randy Pausch. Training for physical tasks in virtual environments: Tai chi. In Virtual Reality, 2003. Proceedings. IEEE, pages 87–94. IEEE, 2003.
[3] Paolo Tormene, Toni Giorgino, Silvana Quaglini, and Mario Stefanelli. Matching incomplete time series with dynamic time warping: an algorithm and an application
to post-stroke rehabilitation. Artificial Intelligence in Medicine, 45(1):11–34, 2009.
[4] Liqun Deng, Howard Leung, Naijie Gu, and Yang Yang. Real-time mocap dance recognition for an interactive dancing game. Computer Animation and Virtual Worlds, 22(2-3):229–237, 2011.
[5] Otniel Portillo-Rodriguez, Oscar O Sandoval-Gonzalez, Emanuele Ruffaldi, Rosario Leonardi, Carlo Alberto Avizzano, and Massimo Bergamasco. Real-time gesture recognition, evaluation and feed-forward correction of a multimodal tai-chi platform. In Haptic and Audio Interaction Design, pages 30–39. Springer, 2008.
[6] CJ Wilson and SK Datta. Tai chi for the prevention of fractures in a nursing home population: An economic analysis. JCOM-WAYNE PA-, 8(3):19–28, 2001.
[7] Xu Zhang, Xiang Chen, Yun Li, Vuokko Lantz, Kongqiao Wang, and Jihai Yang. A framework for hand gesture recognition based on accelerometer and emg sensors. Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on, 41(6):1064–1076, 2011.
[8] Juha Kela, Panu Korpip&;#228;&;#228;, Jani M&;#228;ntyj&;#228;rvi, Sanna Kallio, Giuseppe Savino, Luca Jozzo, and Di Marca. Accelerometer-based gesture control for a design environment.
Personal and Ubiquitous Computing, 10(5):285–299, 2006.
[9] Holger Junker, Oliver Amft, Paul Lukowicz, and Gerhard Tr&;#246;ster. Gesture spotting with body-worn inertial sensors to detect user activities. Pattern Recognition, 41(6): 2010–2024, 2008.
[10] Jiayang Liu, Lin Zhong, Jehan Wickramasuriya, and Venu Vasudevan. uwave: Accelerometer-based personalized gesture recognition and its applications. Pervasive and Mobile Computing, 5(6):657–675, 2009.
[11] Thomas Schl&;#246;mer, Benjamin Poppinga, Niels Henze, and Susanne Boll. Gesture recognition with a wii controller. In Proceedings of the 2nd international conference on Tangible and embedded interaction, pages 11–14. ACM, 2008.
[12] Ahmad Akl and Shahrokh Valaee. Accelerometer-based gesture recognition via dynamic-time warping, affinity propagation, &; compressive sensing. In Acoustics Speech and Signal Processing (ICASSP), 2010 IEEE International Conference on, pages 2270–2273. IEEE, 2010.
[13] Kent Lyons, Helene Brashear, Tracy Westeyn, Jung Soo Kim, and Thad Starner. Gart: The gesture and activity recognition toolkit. In Human-Computer Interaction. HCI Intelligent Multimodal Interaction Environments, pages 718–727. Springer, 2007.
[14] Joseph B Kruskal and Mark Liberman. The symmetric time-warping problem: from continuous to discrete. Time Warps, String Edits and Macromolecules: The Theory and Practice of Sequence Comparison, pages 125–161, 1983.
[15] Hiroaki Sakoe and Seibi Chiba. Dynamic programming algorithm optimization for spoken word recognition. Acoustics, Speech and Signal Processing, IEEE Transactions
on, 26(1):43–49, 1978.
[16] Samsung galaxy gear. http://www.samsung.com/tw/consumer/
mobile-phones/mobile-phones/galaxy-gear/.