臺灣博碩士論文加值系統

嵌入式裝置近期內不管在娛樂、醫療保健和工業應用的地位都日趨重要，使用上也更貼近現實生活中的需求，此外人類與機器人互動(Human Robot Interaction，簡稱HRI)的領域更是因為嵌入式裝置不斷的擴展而在學術界與工業界都獲得良好且快速的成長，致使此領域在各方面都獲得了更好的發揮舞台。本研究基於顏色和深度攝影機來實現一套支援手勢識別的導航移動機器人系統，在技術上，本系統透過ARM的嵌入式裝置來實作手勢識別並進行移動機器人的控制，而在人物追蹤方面是透過（Human Depth of Interest，簡稱HDOI）找出人的深度範圍並與CAM-shift演算法結合來達到追蹤的效果，此外套入虛擬彈簧模型(virtual spring model)以提高嵌入式裝置的使用效率，在導航的控制部分，基於前人所提出的RGB-H-CrCb色彩空間模型以顏色區塊分割來找出手的部位，並計算其雙手偏離的角度來當作移動機器人旋轉的控制參數，系統中手勢識別的分類器是採用韋伯局部描述子(Weber Local Descriptor)結合支持向量機(Support Vector Machine，合稱WLD+SVM)的演算法來控制其線性移動。

This dissertation researches and implements the human following and hand recognition based navigation mobile robot system using color and depth cameras. Embedded devices recently become more crucial in some real-life applications, such as the entertainment, health care, and industrial applications. Furthermore, Human Robot Interaction (HRI) has drawn the great importance in not only academic but also industrial applications. Given the development of embedded devices and the enormousness of HRI, a mobile robot with the human following function and navigation controlling based on human gestures recognition is built up with an embedded ARM board. In the human following function, Human Depth of Interest (HDOI) is proposed to cooperate with CAM-shift method for the human detection and tracking. The virtual spring model is modified to work more effectively in the embedded device. In the navigation control, the hand detection based human’s face is implemented from skin segmentation with the proposed RGB-H-CrCb colors. The deviation angle between left and right hand is chosen to control the rotation of robot, and hand gesture classification from a new proposed Weber Local Descriptor and Support Vector Machine (WLD+SVM) algorithm is issues to control the linear motion.

ACKNOWLEDGMENTS v
ABSTRACT vi
TABLE OF CONTENTS vii
LIST OF TABLES ix
LIST OF FIGURES x
Chapter 1: INTRODUCTION 1
1.1. Background and Motivation 1
1.2. Thesis Contribution 2
1.3. Thesis Organization 2
Chapter 2: RELATED WORKS 4
2.1. Human Following Applications in Mobile Robots 4
2.2. Human Gestures and Behaviors with Mobile Robots 5
2.3. Weber Law Description 6
Chapter 3: RGB-D CAMERAS IN VISION APPLICATIONS 8
3.1. Overview 8
3.1.1. Introduction of RGB-D camera technologies 8
3.1.2. Asus Xtion Pro Live Specification 9
3.2. Calibration in RGB-D camera 10
3.2.1. Generate 3D coordinates 11
3.2.2. Calibration of depth and color images 12
Chapter 4: HUMAN FOLLOWING MOBILE ROBOT 15
4.1. System Overview 15
4.1.1. Mobile Robot Architecture 15
4.1.2. Overview of Human Tracking and Following 16
4.2. Proposed Human Following Method 17
4.2.1. Preprocessing 17
4.2.2. Human Detection and Tracking 19
4.2.3. Human Position Projection with Robot Motion 24
4.2.4. Modified Virtual Spring Control Rule 26
Chapter 5: VISION BASED STEERING WHEEL IN MOBILE ROBOTS 30
5.1. System Overview 30
5.1.1. Working Environment Setup 30
5.1.2. Overview of proposed robot navigation control 30
5.2. Proposed virtual steering wheel method 31
5.2.1. Preprocessing 31
5.2.2. Hand Detection and Segmentation 37
5.2.3. Hand Gesture Recognition with WLD+SVM 41
5.2.4. Vision Control and Robot Control 46
Chapter 6: EXPERIMENTAL RESULTS 51
6.1. Human Following Mobile Robot Application 51
6.2. Vision-based Steering Wheel Robot Navigation Control 56
Chapter 7: CONCLUSIONS AND FUTURE WORKS 64
7.1. Conclusions 64
7.2. Future works 65
REFERENCES 66

[1]AlphaDog: http://www.pcmag.com/article2/0,2817,2409628,00.asp [Online].
[2]Dancing Robot: http://www.dnp.co.jp/news/10104464_2482.html [Online].
[3]M. Q. Do and C. H. Lin, "Embedded Human-Following Mobile-Robot with an RGB-D camera," in Proc. 14th MVA Conf., Tokyo, Japan, 2015.
[4]H. Zhang, C. Reardon, and L. E. Parker, "Real-Time Multiple Human Perception With Color-Depth Cameras on a Mobile Robot," IEEE Trans. Cybern., vol. 43, pp. 1429-1441, 2013.
[5]J. Kovac, P. Peer, and F. Solina, "Human skin color clustering for face detection," in Proc. EUROCON, 2003, pp. 144-148 vol.2.
[6]C. Jie, S. Shiguang, H. Chu, Z. Guoying, M. Pietikainen, C. Xilin, et al., "WLD: A Robust Local Image Descriptor," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, pp. 1705-1720, 2010.
[7]C. Cortes and V. Vapnik, "Support-vector networks," Machine learning, vol. 20, pp. 273-297, 1995.
[8]W. Chung, H. Kim, Y. Yoo, C.-b. Moon, and J. Park, "The Detection and Following of Human Legs Through Inductive Approaches for a Mobile Robot With a Single Laser Range Finder," IEEE Trans. Ind. Electron., vol. 59, pp. 3156-3166, 2012.
[9]N. Bellotto and H. Hu, "Multisensor-Based Human Detection and Tracking for Mobile Service Robots," IEEE Trans. Syst., Man, Cybern, B, Cybern. , vol. 39, pp. 167-181, 2009.
[10]R. C. Luo, N. W. Chang, S. C. Lin, and S. C. Wu, "Human tracking and following using sensor fusion approach for mobile assistive companion robot," in Proc. 35th Annual Conf. of IEEE IECON, 2009, pp. 2235-2240.
[11]E. Petrović, A. Leu, D. Ristić-Durrant, and V. Nikolić, "Stereo Vision-Based Human Tracking for Robotic Follower," Int. J. Adv. Robot Syst., vol. 10:230, 2013.
[12]J. Satake and J. Miura, "Multiple-Person Tracking for a Mobile Robot using Stereo," in Proc. IAPR Conf. on Mach. Vision App., 2009, pp. 273-277.
[13]S. Jia, L. Wang, S. Wang, and C. Bai, "Fuzzy-based intelligent control strategy for a person following robot," in Proc. IEEE Int. Conf. ROBIO, 2013, pp. 2408-2413.
[14]H. Bouraoui, A. Khamis, and F. Krray, "A testbed platform for assessing human-robot verbal interaction," in Proc. Int. Conf. AIS, 2010, pp. 1-6.
[15]F. Faber, M. Bennewitz, C. Eppner, A. Gorog, C. Gonsior, D. Joho, et al., "The humanoid museum tour guide Robotinho," in Proc. 18th IEEE RO-MAN, 2009, pp. 891-896.
[16]A. Punchihewa and Z. M. Arshad, "Voice command interpretation for robot control," in Proc. Int. Conf. ICARA, 2011, pp. 90-95.
[17]M. T. Wolf, C. Assad, M. T. Vernacchia, J. Fromm, and H. L. Jethani, "Gesture-based robot control with variable autonomy from the JPL BioSleeve," in Proc. IEEE ICRA, 2013, pp. 1160-1165.
[18]J. L. Raheja, R. Shyam, U. Kumar, and P. B. Prasad, "Real-Time Robotic Hand Control Using Hand Gestures," in Proc. ICMLC, 2010, pp. 12-16.
[19]M. Hasanuzzaman, V. Ampornaramveth, Z. Tao, M. A. Bhuiyan, Y. Shirai, and H. Ueno, "Real-time Vision-based Gesture Recognition for Human Robot Interaction," in Proc. IEEE Int. Conf. ROBIO, 2004, pp. 413-418.
[20]R. C. Luo and W. Yen-Chang, "Human-robot interaction with multi-sensor fusion based hand sign recognition for service robot," in Proc. 38th IECON, 2012, pp. 2725-2730.
[21]S. Takahashi, Y. Takahashi, Y. Maeda, and T. Nakamura, "Development of body mapping from human demonstrator to inverted-pendulum mobile robot for imitation," in Proc. IEEE Int. Conf. FUZZ, 2011, pp. 1344-1349.
[22]A.K. Jain, Fundamentals of Digital Image Processing: Prentice-Hall, 1989.
[23]J. Das and H. Roy, "Human Face Detection in Color Images Using HSV Color Histogram and WLD," in Proc. Int. Conf. CICN, 2014, pp. 198-202.
[24]L. Shinfend D., L. Yuan-Ming, and J. Yu-Ruei, "A Robust Image Descriptor for Human Detection based on HOG and Weber's Law," Int. J. Innovative Computing, Information and Control, vol. 9-10, pp. 3887-3901, 2013.
[25]G. Dayi, L. Shutao, and X. Yin, "Face recognition using the Weber Local Descriptor," in Proc. 1 st ACPR, 2011, pp. 589-592.
[26]I. Ullah, M. Hussain, G. Muhammad, H. Aboalsamh, G. Bebis, and A. M. Mirza, "Gender recognition from face images with local WLD descriptor," in Proc. 19 th IWSSIP, 2012, pp. 417-420.
[27]G. Muhammad, M. Hussain, F. Alenezy, A. M. Mirza, G. Bebis, and H. Aboalsamh, "Race recognition using local descriptors," in Proc. IEEE ICASSP, 2012, pp. 1525-1528.
[28]Bumblebee camera: http://www.ptgrey.com/bumblebee-xb3-1394b-stereo-vision-camera-systems-2 [Online].
[29]Kinect: http://www.xbox.com/en-US/xbox-360/accessories/kinect [Online].
[30]pmd[vision]® CamCube 3.0: http://www.pmdtec.com/news_media-/video/camcube.php [Online].
[31]Asus Xtion Pro Live website: http://www.asus.com-/Commercial_3D_Sensor/Xtion_PRO_LIVE [Online].
[32]ROS-Introduction: http://wiki.ros.org/ROS/Introduction [Online].
[33]M. A. Fischler and R. C. Bolles, "Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography," Commun. ACM, vol. 24, pp. 381-395, 1981.
[34]D. G. R. Bradski and A. Kaehler, Learning opencv, 1st edition: O'Reilly Media, Inc., 2008.
[35]D. Comaniciu and P. Meer, "Mean shift: a robust approach toward feature space analysis," IEEE Trans. on Pattern Analysis and Machine Intelligent, vol. 24, pp. 603-619, 2002.
[36]G. A. John, Y. D. X. Richard, and S. J. Jesse, "Object Tracking Using CamShift Algorithm and Multiple Quantized Feature Spaces " in the Pan-Sydney Area Workshop on VIP 2003.
[37]The quaternion coordinate system: http://en.wikipedia.org-/wiki/Quaternion [Online].
[38]Hooke's law: http://scripts.mit.edu/~srayyan/PERwiki/index.php?-title=Module_2_--_Hooke%27s_Law_for_Elastic_Restoring_-Force [Online].
[39]K. Morioka, J.-H. Lee, and H. Hashimoto, "Human-following mobile robot in a distributed intelligent sensor network," IEEE Trans. Ind. Electron., vol. 51, pp. 229-237, 2004.
[40]Gold-white or blue-black dress: http://www.dailymail.co.uk/-news/article-2971409/What-color-dress-White-gold-blue-black.html [Online].
[41]Z. Yanmin and Y. Bo, "Real-time hand gesture recognition with Kinect for playing racing video games," in Proc. IJCNN, 2014, pp. 3240-3246.
[42]J. R. d. Solar and R. Verschae, "Skin Detection using Neighborhood Information," in Proc. of the 6th Int. Conf. on Auto. Face and Gest. Recog. , Seoul, Korea, 2004, pp. 463-468.
[43]P. Viola and M. J. Jones, "Rapid object detection using a boosted cascade of simple features," in Proc. IEEE CVPR, 2001.
[44]J. Blanchette and M. Summerfield, C++ GUI Programming with Qt 4: Prentice Hall PTR, 2006.
[45]N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in Proc. IEEE Conf. Comput. Vision Pattern Recognit., 2005, pp. 886-893 vol. 1.
[46]The comparison of original CAM-shift and HDOI+CAM-shift: www.youtube.com/watch?v=EvAoE_fn0tw [Online].
[47]The proposed human following results: www.youtube.com-/watch?v=UmENoaV1W_E [Online].
[48]N. A. Abdul Rahim, C. W. Kit, and J. See, "RGB-H-CbCr skin colour model for human face detection," in Proc. M2USIC, 2006.
[49]J. A. M. Basilio, G. A. Torres, G. S. Pérez, L. K. T. Medina, and H. M. P. Meana, "Explicit image detection using YCbCr space color model as skin detection," in Proc. American Conf. on applied Math., Puerto Morelos, Mexico, 2011, pp. 123-128.
[50]H. Zhang and G. Chen, "The Research of Face Recognition Based on PCA and K-Nearest Neighbor," in Proc SOPO, 2012, pp. 1-4.
[51]M. Agarwal, H. Agrawal, N. Jain, and M. Kumar, "Face Recognition Using Principle Component Analysis, Eigenface and Neural Network," in Proc ICSAP, 2010, pp. 310-314.
[52]Y. T. Chen and K. T. Tseng, "Multiple-angle Hand Gesture Recognition by Fusing SVM Classifiers," in Proc. IEEE CASE, 2007, pp. 527-530.
[53]D.-Y. Huang, W.-C. Hu, and S.-H. Chang, "Vision-Based Hand Gesture Recognition Using PCA+Gabor Filters and SVM," in Proc IIH-MSP, 2009, pp. 1-4.
[54]Vision based steering wheel demo: www.youtube.com-/watch?v=wdckS7FiHIw [Online].