臺灣博碩士論文加值系統

本論文提出即時影像避障和室內導航應用於全向輪式移動型機器人;避障部份主要是用Intel Realsense 深度攝影機 D415，我們提取裡面的物體距離資訊並套用模糊系統來使全向輪機器人避開障礙物;導航部份用SURF特徵匹配讓機器人知道他走到了哪裡。這篇論文主要改變傳統的機器人避障方式，只使用攝影機，不用其他的測距儀器或是定位系統就能進行避障和導航。此系統中使用的演算法是用Python2.7 撰寫，全向輪和攝影機是由labview 2017來做整合。實驗結果顯示，全向輪機器人使用此系統可以避開障礙物和移動到指定的位置，並完成開門的動作。

This thesis presents application of pattern recognition and environmental feature matching to an omnidirectional wheeled mobile robot (WMR). This study used Hue-Saturation-Value color space (HSV) instead of normal RGB color space, because HSV is similar to human vision, it can intuitively express light and shade, hue and vividness. For pattern recognition, Back-propagation neural network (BPNN) and the Adaptive Resonance Theory (ART) are applied to suitable patterns, including elevator buttons, LED numbers, and doorplate. The Speeded Up Robust Features (SURF) feature matching is used to guide the robot into a preset working path and to indicate the next direction and action of the robot. If obstacles are encountered in the work route, use ultrasonic sensors and laser sensor to avoid obstacles. In addition, fuzzy control is applied to correct the route offset caused by long-distance movement of the robot. The algorithm used in the proposed system was written in MATLAB 2013. The wide-angle webcam, robotic arms, omnidirectional wheels, ultrasonic sensors, and laser sensor were integrated by LabVIEW 2014. Experimental results show that the proposed control scheme can make the omnidirectional wheeled mobile robot take the elevator to different floors and deliver documents to different rooms automatically.

Contents
Abstract (Chinese) I
Abstract II
Contents III
List of Figures V
List of Tables VII
Chapter 1 Introduction 1
1.1 Preface 1
1.2 Research Motivation and Goal 1
1.3 Literature Review 2
1.4 Thesis Organization 2
Chapter 2 Omnidirectional Wheel Mobile Robot and Experimental Hardware 3
2.1 System Description 3
2.2 Depth Camera 5
2.3 Components of Omnidirectional WMR 6
2.3.1 Model of Omnidirectional WMR 6
2.3.2 Arduino 10
2.3.3 Omni Wheel 11
2.3.4 DC Motor 12
2.3.5 Motor Controller 13
2.4 Robot Arm 16
2.5 Webcam 17
Chapter 3 Image Processing and Pattern Recognition 18
3.1 Color Space 18
3.1.2 RGB Color Space 18
3.2 Depth Theory 19
3.2.1 Camera Calibration 21
3.2.2 Stereo Rectify 23
3.2.3 Stereo Match 24
3.3 Depth Map 25
3.4 Obstacle Detection 26
3.5 SURF 30
3.5.1 Feature Matching 30
3.6 HoughCircles 32
Chapter 4 Control Scheme 34
4.1 Control Sequence 34
4.2 Fuzzy Contorl 37
4.3 Obstacle Avoidance 42
4.4 Navigation 44
4.5 Target Tracking 44
Chapter 5 Experiment Result 46
5.1 Obstacle Avoidance Test 46
5.2 System Integration 51
Chapter 6 Conclusions 69
6.1 Conclusions 69
6.2 Future Prospect and Suggestion 69
Reference 70

List of Figures

Figure 1 Robot 4
Figure 2 Omnidirectional Wheel 4
Figure 3 Intel Realsense Depth Camera D415 5
Figure 4 Omnidirectional wheels structure [13] 6
Figure 5 Omnidirectional wheels structure [13] 7
Figure 6 Relation between road speed and angle [13] 7
Figure 7 Schematic of WMR motion [13] 8
Figure 8 Motion of omnidirectional wheel-1 [13] 8
Figure 9 Motion of omnidirectional wheel-2 [13] 9
Figure 10 Motion of omnidirectional wheel-3 [13] 9
Figure 11 Arduino Uno R3[15] 10
Figure 12 DFRduino IO Expansion Shield For Arduino 11
Figure 13 Omnidirectional wheel 4202X 12
Figure 14 DC motor [17] 13
Figure 15 Motor controller [18] 14
Figure 16 Micro Maestro 6-channel USB servo controller [19] 15
Figure 17 Three kinds of motors 16
Figure 18 Microsoft LifeCam studio 1080P Full-FD 17
Figure 19 color cube [27] 19
Figure 20 Depth theory model 20
Figure 21 Depth measurement flow chart 21
Figure 22 (a~c) Zhang Zhengyou camera calibration samples 22
Figure 23 (a~c) stereo rectify 24
Figure 24 Mug distance 24
Figure 25 (a) red is far to the camera, blue is near to the camera (b) white is far to the camera, black is near to the camera (c) black is far to the camera, white is near to the camera 26
Figure 26 Detection of different obstacle 29
Figure 27 SURF matching 31
Figure 28 Threshold the image and edge detection 32
Figure 29 detect door knob 33
Figure 30 Control flowchart 35
Figure 31 Experimental environment 36
Figure 32 Robot working path 36
Figure 33 fuzzy sets 40
Figure 34 Fuzzy control model 41
Figure 35 Fuzzy control scheme 42
Figure 36 obstacle detection 43
Figure 37 Robot avoids obstacles 43
Figure 38 Robot at the specified position 44
Figure 39 The robots arm lift to the specified height 45
Figure 59 Robot obstacle avoidance test 51
Figure 40 Experimental place 51
Figure 41 obstacle on the preset route 52
Figure 42 Robot start position 52
Figure 43 Depth camera image 53
Figure 44 Distance and direction 53
Figure 45 Robot position 54
Figure 46 Find an obstacle 54
Figure 47 Robot move direction 55
Figure 48 Robot rotates left 55
Figure 49 Robot rotates back 56
Figure 50 Robot’s image 57
Figure 51 Judged direction 57
Figure 52 Robot judges target direction 58
Figure 53 After the SURF matching 59
Figure 54 Robot arrives at the specified location 60
Figure 55 Arm doorknob detection 62
Figure 56 Arm doorknob detection 63
Figure 57 Arm doorknob detection 66
Figure 58 Robot opens door 68
 
List of Tables

Table 1 Datasheet of the Intel Realsense Depth Camera D415 [12] 5
Table 2 Arduino UNO specifications 11
Table 3 Digital I/O pin connectors of DFRduino I/O expansion 11
Table 4 Omni wheel specifications 12
Table 5 DC motor specifications 13
Table 6 Motor controller interface connections 14
Table 7 Motor controller specifications 14
Table 8 Micro Maestro 6-channel USB servo controller specifications 15

Reference
[1] Li Chi, Application of Real-Time Image Recognition and Feature Matching to Wheeled Mobile Robot for Room Service, Master Thesis, Department of Communications, Navigation and Control Engineering, NTOU, ROC, 2018.
[2] Qian Jia, Mulan Wang, Shuqing Liu, Jianjing Ge, and Chen Gu, Research and Development of Mecanum-wheeled Omnidirectional Mobile Robot Implemented by Multiple Control Methods, Proceedings of 2016 23rd International Conference on Mechatronics and Machine Vision in Practice (M2VIP).
[3] Susan park, Young-jae ryoo, and Dae-yeong Im, Fuzzy Steering Control of Three-Wheels Based Omnidirectional Mobile Robot, Proceeding of 2016 International Conference on Fuzzy Theory and Its Applications (iFuzzy), 2016
[4] J.H. Chung, B.J. Yi, W.K. Kim, and H.G. Lee, The Dynamic Modeling and Analysis for an Omnidirectional Mobile Robot with Three Caster Wheels, Proceedings of IEEE International Conference on Robotics and Automation, 2003.
[5] Xiaogang Ruan, and Wangbo Li, Ultrasonic Sensor Based Two-wheeled Self-balancing Robot Obstacle Avoidance Control System, Proceedings of 2014 IEEE International Conference, 2014.
[6] Yun Jin, Shengquan Li, Juan Li, Hongbing Sun,and Yuanwang Wu, Design of an Intelligent Active Obstacle Avoidance Car Based on Rotating Ultrasonic Sensors, Proceedings of 2018 IEEE 8th Annual International Conference, 2018.
[7] Yan Peng, Dong Qu , Yuxuan Zhong, Shaorong Xie, and Jun Luo, The Obstacle Detection and Obstacle Avoidance Algorithm Based on 2-D Lidar, Proceeding of the 2015 IEEE International Conference, 2015.
[8] Teng Wang, Leping Bu, and Zhongyi Huang, A New Method for Obstacle Detection Based on Kinect Depth Image, 2015 Chinese Automation Congress (CAC), 2015.
[9] Rostam Affendi, Hamzah Hasrul, Nisham Rosly, and Saad Hamid, An Obstacle Detection and Avoidance of A Mobile Robot with Stereo Vision Camera, Proceeding of 2011 International Conference on Electronic Devices, Systems and Applications (ICEDSA), 2011.
[10] Mostafa Sharifi and XiaoQi Chen, Introducing A Novel Vision Based Obstacle Avoidance Technique for Navigation of Autonomous Mobile Robots, Proceedings of 2015 IEEE 10th Conference on Industrial Electronics and Applications (ICIEA), 2015.



[11] Q. AI-Jubouri, W. AI-Nuaimy, M. AI-Taeeand, and I. Young, Recognition of Individual Zebrafish Using Speed-Up Robust Feature Matching, Proceedings of 10th International Conference on Developments in eSystems Engineering, 2017.
[12] Intel Realseanse Depth Camera D415
https://store.intelrealsense.com/buy-intel-realsense-depth-camera-d415.html https://www.intel.com.tw/content/www/tw/zh/support/articles/000026827/emerging-technologies/intel-realsense-technology.html
[13] Q.H. Zhong, Using Omni-Directional Mobile Robot on Map Building Application, Master Thesis, Department of Engineering Science, NCKU, ROC, 2009.
[14] Arduino Uno R3, http://electricarena.blogspot.com/2015/01/arduino-uno- r3 specifications- parts.html
[15] DFRduino IO Expansion Shield For Arduino https://www.dfrobot.com/product-1009.html
[16] Omni Wheel, http://www.kornylak.com/wheels/transwheel.html
[17] DC motor IG-42CGM http://www.shayangye.com/en/product-inner.aspx?f=s&i=118
[18] Motor Controller HB-25 https://www.parallax.com/product/29144
[19] Micro Maestro 6-channel USB servo controller https://www.pololu.com/product/1350
[20] RX-64 http://emanual.robotis.com/docs/en/dxl/rx/rx-64/
[21] RX-28 http://emanual.robotis.com/docs/en/dxl/rx/rx-28/
[22] XM430-W350 http://emanual.robotis.com/docs/en/dxl/x/xm430-w350/
[23] Color space https://en.wikipedia.org/wiki/Color_space
[24] color cube http://courses.cs.vt.edu/~cs4624/s98/sspace/imgproc/index.html
[25] Yi Zhang, Xiangyang Xu, and Yaping Dai, Two-Stage Obstacle Detection Based on Stereo Vision in Unstructured Environment, Proceeding of 2014 Sixth International Conference on Intelligent Human-Machine Systems and Cybernetics, 2014.
[26] 張正友標定 https://www.microsoft.com/en-us/research/wp- content/uploads/2016/02/tr98-71.pdf
[27] Rostam Affendi Hamzah, Hasrul Nisham Rosly, and Saad HamidAn, Obstacle Detection and Avoidance of A Mobile Robot with Stereo Vision Camera, Proceedings of 2011 ICEDSA, 2011.
[28] H. Bay, T. Tuytelaars, and L.V. Gool, “Speed Up Robust Features,” Proceedings of European Conference on Computer Vision, 2006.
[29] Hong Liu, Yueliang Qian, and Shouxun Lin, Detecting Persons Using Hough Circle Transform In Surveillance Video, Proceeding of International Conference on Computer Vision Theory and Applications, 2010.
[30] Chan-Hong Chao, Bo-Yan Hsueh, Ming-Ying Hsiao, Shun-Hung Tsai, and Tzuu-Hseng S. Li Real-Time Target Tracking and Obstacle Avoidance for Mobile Robots using Two Cameras, Proceeding of ICROS-SICE International Joint Conference, 2009.
[31] Hong-Rui Su, and Kuo-Yi Chen, Design and Implementation of a Mobile Robot with Autonomous Door Opening Ability, Proceeding of 2017 International Conference on Fuzzy Theory and Its Applications (iFUZZY) Taiwan, R. O. C. , 2017.