臺灣博碩士論文加值系統

數位家庭的推行已經行之有年，為了讓家庭生活更加的便利、舒適。家用電子產品的發展逐步朝向多模式(multi-modal)與智慧化操控的整合。基於此發展趨勢，本論文的目標在於研發透過攝影機擷取使用者的肢體動作視訊，並利用智慧型的辨識技術進行分析，藉此讓使用者能自然方便地利用肢體動作來直接操控家電，而無須再藉助遙控器或手按式機械開關。
本研究之動作設計的著重在雙手的變化上，手臂的擷取藉由平均位移演算法（Mean Shift Algorithm）找出畫面中的人體軀幹。將軀幹自人體中去除後即可定位出上肢（手臂），然後再利用主成分分析法（Principal Component Analysis）即可找到各手臂的主軸方向，將此主軸方向編碼為平面座標系統中四象限的角度即成為辨識身體姿勢（Postures）的重要特徵。鑑於每一個肢體動作是由少數幾個不同的連續姿勢所組成，若能先將所有可能的身體姿勢做適當分群，將類似的身體姿勢歸於同類，如此就可用更精簡的姿勢表示法來表示各種不同的肢體動作。但是，某些相似姿勢間的混淆性，可能造成姿勢在分類時的誤判而影響肢體動作的辨識。基於此因，本論文提出一個以模糊理論的概念所發展出的強健特徵－姿勢群隸屬度特徵（Posture Cluster Membership）。這個姿勢群隸屬度特徵藉由預先建立各個姿勢群的高斯機率模型，然後再用以計算某姿勢分別屬於各姿勢群的機率值，並將這些機率值結合為一個機率向量以描述該姿勢。
同時本論文提出模糊動態時軸扭曲（Fuzzy Dynamic Time Warping）演算法，在傳統的動態時軸扭曲演算法中引入隸屬度交集（Membership intersection）以及姿勢群轉移機率（Posture Cluster Transition Probability）之新概念來進行肢體動作的辨識分析。經由實驗証明，姿勢群隸屬度特徵配合模糊動態時軸扭曲演算法進行肢體動作的比對，比起使用傳統的動態時軸扭曲演算法明顯能有更好的辨識結果。

The digital home has been promoted for years. To reach more convenience and comfortable daily life in a digital home, the electronic home appliances tend to be developed toward the inclusion of integrated multi-modal and intelligent control interfaces. The research explored in this paper aims to develop an intelligent human-machine interface to capture the video of user body actions from a camera and to recognize the body actions in the video as the issued commands to operate various home appliances. The conventional remote or hand-touch mechanical switches can then be replaced by this natural and friendly vision-based human-machine interface.
In this research, the designed body actions to be recognized concentrate on the motion of user hands. Firstly, the whole human body on each video frame is segmented by background subtraction. Prior to the action recognition, the torso of human body is located by the mean shift algorithm. Then, the limbs are extracted by subtracting the located torso from the human body. Using the principal component analysis, we calculate the directions of the limbs and represent the directions with the angles on the four quadrants of a 2-D plane. As each body action comprises a sequence of body postures, grouping all common body postures into several clusters can lead to a more compressed and effective representation for different body actions. However, some confusion may exist among similar body postures and may cause the body postures misclassified into wrong clusters. Therefore, this paper proposes a novel robust feature called posture cluster membership to represent body postures based on the fuzzy theory. The posture clusters membership is represented by a vector of probability values with each value quantitatively assessing the degree of being a member of a corresponding posture cluster for each given body posture.
Additionally, the paper also proposes a fuzzy dynamic time warping algorithm for matching the feature patterns of body actions. The feature pattern of each body action is composed of a vary-length sequence of posture cluster membership. By introducing the membership intersection and the posture cluster transition probability into the conventional dynamic time warping algorithm, we achieve better recognition performance than other matching algorithms based on dynamic programming.

摘 要 II
ABSTRACT IV
誌 謝 V
圖目錄 VIII
表目錄 X
1. 導論 1
1.1 研究動機 1
1.2 相關研究 2
1.3 系統流程 7
1.4 章節架構 8
2. 姿勢特徵抽取 9
2.1.1. 色彩模型與前景物件擷取 10
2.1.2. 前景物件修補 13
2.2 特徵抽取 14
2.2.1. 人體軀幹及雙手擷取 15
2.2.2. 特徵選定 19
2.2.3. 姿勢群隸屬度特徵 20
3. 植基於動態規劃演算法之行為分析 23
3.1 動態時軸扭曲 24
3.2 模糊動態時軸扭曲 29
3.3 姿勢特徵轉移機率統計 32
4. 實驗結果 33
4.1 實驗環境 33
4.2 姿勢資料庫類別 33
4.2.1. 姿勢類別一 34
4.2.2. 姿勢類別二 34
4.2.3. 姿勢類別三 34
4.2.4. 姿勢類別四 35
4.2.5. 姿勢類別五 35
4.2.6. 姿勢類別六 35
4.2.7. 姿勢類別七 36
4.2.8. 姿勢類別八 36
4.2.9. 姿勢類別九 36
4.3 控制動作設計 37
4.3.1. 開燈 38
4.3.2. 關燈 38
4.3.3. 開冷氣 39
4.3.4. 關冷氣 39
4.3.5. 開電視 40
4.3.6. 關電視 40
4.3.7. 開電扇 41
4.3.8. 關電扇 41
4.4 指令動作分析 42
4.4.1. 使用編輯距離演算法進行行為分析 42
4.4.2. DTW使用L1距離 44
4.4.2.1. 以四象限角度特徵做比對 44
4.4.2.2. 以姿勢隸屬度特徵做比對 45
4.4.3. Fuzzy DTW 46
4.4.3.1. 以固定常數值充當姿勢轉移機率(未做統計) 46
4.4.3.2. 實際統計姿勢轉移機率 47
4.5 實驗討論 48
5. 結論與未來研究方向 49
5.1 結論 49
5.2 未來研究方向 49
參考文獻 51

[1] http://www.xbox.com/zh-TW/community/news/2009/20090602a.htm
[2] I. Haritaoglu, D. Harwood, and L. S. Davis, “W4: real-time surveillance of people and their activities.” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 22, No. 8, pp.809-830, Aug 2000.
[3] M.–C. Roh, B. Christmas, J. Kittler, and S.–W. Lee, “Gesture spotting in low-quality video with features based on curvature scale space.” in Proc. 7th of IEEE International Conference on Automatic Face and Gesture Recognition, pp.375- 380, Apr 2006.
[4] K. Stephan, H. Thomas, and E. Wolfgang, “Shape-based posture and gesture recognition in videos.” in Proc. SPIE, Vol. 5682, pp.114-124, 2004.
[5] R. Cucchiara, C. Grana, A. Prati, and R. Vezzani, “Probabilistic posture classification for human-behavior analysis.” IEEE Trans. on Systems, Man and Cybernetics, Part A, Vol. 35, No. 1, pp.42- 54, 2005.
[6] H.–S.Chen, H.–T.Chen, Y.–W.Chen, and S.–Y.Lee, “Human action recognition using star skeleton.” in Proc. 4th ACM international workshop on Video surveillance and sensor networks, pp.171-178, 2006.
[7] F. Buccolieri, C. Distante, and A. Leone, “Human posture recognition using active contours and radial basis function neural network.” in Proc. IEEE Conference on AVSS, pp.213-218, 2005.
[8] J. K. Aggarwal and Q. Cai, “Human motion analysis: A review.” Computer Vision and Image Understanding, vol. 73, no.3, pp.428 - 440, 1999.
[9] T. Moeslund and E. Granum, “A survey of computer vision-based human motion capture.” Computer Vision and Image Understanding, vol. 81, no.3, pp.231-268, 2001.
[10] H. Weiming, T. Tieniu, W. Liang, and S. Maybank, “A survey on visual surveillance of object motion and behaviors.” IEEE Trans. on Systems, Man, and Cybernetics, Part C, Vol. 34, No. 3, pp.334-352, Aug. 2004.
[11] P. Maja, P. Alex, N. Anton, and S. H. Thomas, “Human Computing and Machine Understanding of Human Behavior: A Survey.” in Proc. 8th ACM
ICMI, pp.239-248, 2006.
[12] C. Cedras and M. Shah, “Motion-based recognition a survey.” Image and Vision Computing, Vol.13, No. 2, pp.129-155, March 1995.
[13] O. Boiman and M. Irani, “Detecting Irregularities in Images and in Video.” International Journal of Computer Vision, Vol. 74, No. 1, pp.17-31 , Aug 2007.
[14] Y.–C.Chen, “Action Recognition Using Multi-class Boosting.” National Taiwan University, 2006.
[15] C.–F.Juang and C.–M.Chang, “Human body posture classification by a neural fuzzy network and home care system application.” IEEE Trans. on Systems, Man and Cybernetics, Part A, Vol. 37, No 6, pp.984-994, 2007.
[16] P. Guo and Z. Maio, “A Home Environment Posture and Behavior Recognition System.” in Proc. International Conference on Convergence Information Technology, pp.175-180, 2007.
[17] A.F. Bobick and J.W. Davis, “The Recognition of Human Movement Using Temporal Templates.” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 23, No.3, pp.257–267, 2001.
[18] J.–W.Hsieh, Y.–T.Hsu, and H.–Y.Mark Liao, “Video-based human movement analysis and its application to surveillance systems,” IEEE Trans. on Multimedia, Vol. 10, No. 3, pp.372-384, April 2008.
[19] Y.–T. Hsu, J.–W. Hsieh, and H.–Y. Liao, “Human Behavior Analysis Using Deformable Triangulations.” in Proc. International Conference on Multimedia Signal Processing, 2005.
[20] J. Wang, Y. Makihara, and Y. Yagi, “Human tracking and segmentation supported by silhouette-based gait recognition.” in Proc. IEEE International Conference on Robotics and Automation, pp.1698-1703, May 2008.
[21] L. Rabiner and B. Juang, “An introduction to hidden Markov models.” IEEE ASSP Magazine, Vol. 3, No. 1, Part 1, pp.4-16, 1986.
[22] N. Oliver, B. Rosario, and A. Pentland, "A Bayesian Computer Vision System for Modeling Human Interactions." IEEE Trans. on PAMI, vol. 22, No. 8, pp.831-843, Aug 2000.
[23] Z.–H. Lee, "Human Behavior Analysis with Shadow Cancellation and Occlusion Detection Techniques." National Chiao Tung University, 2006.
[24] O. Masoud, N. Papanikolopoulos, “A method for human action recognition.” Image and Vision Computing, Vol. 21, No. 8, pp.729-743, Aug 2003.
[25] J. Zhou and J. Hoang, “Real time robust human detection and tracking system.” in Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 3, pp.149–149, June 2005.
[26] W.-S.Hu and T.-M.Su, “Robust Background Subtraction with Shadow and Highlight Removal for Indoor Surveillance.” EURASIP Journal on Advances in Signal Processing, 2007.
[27] S. Indupalli, M.A. Ali, and B. Boufama, “A Novel Clustering-Based Method for Adaptive Background Segmentation.” in Proc. 3rd Canadian Conference on Computer and Robot Vision, June 2006.
[28] O. Javed, K. Shafique, and M. Shah, “A Hierarchical Approach to Robust Background Subtraction Using Color and Gradient Information.” in Proc. IEEE Workshop on Motion Video Computing, pp.22-27, Dec. 2002.
[29] F. Alexandre and M. Gerald, “Adaptive color background modeling for realtime segmentation of video streams.” in Proc. International on Imaging Science, System and Technology, pp.227-232, 1999.
[30] S. E Umbaugh, “Computer vision and image processing.” international ed., Prentice Hall, 1998.
[31] http://www.blackice.com/colorspaceHSV.htm
[32] R. C. Gonzalez and R. E. Woods, “Digital Image Processing.” Second ed., Prentice Hall, 2002.
[33] D. Comaniciu and P. Meer, “Mean Shift: A Robust Approach Toward Feature Space Analysis.” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 24, pp.603-619, 2002.
[34] R. O. Dudac, P. E. Hart, and D. G. Stork, ”Pattern Classification.” Second ed., Wiley Interscience, 2000.
[35] R. Johnsonbaugh and M. Schaefer, “Algorithms.” Second ed., Prentice Hall, 2004.
[36] C. S. Myers and L. R. Rabiner, “A comparative study of several dynamic time-warping algorithms for connected word recognition.” The Bell System Technical Journal, 1981.
[37] J.-C. Lai, “Irregular behavior detection and analyses using dynamic programming algorithm in Indoor environments.” Nation Dong Hwa University, 2008.
[38] http://www.gesturetek.com/newscenter/newsreleases.php
[39] Y. Freund and R. E. Schapire, “A Short Introduction to Boosting.” Journal of Japanese Society for Artificial Intelligence, Vol. 14, pp.771–780, 1999.