臺灣博碩士論文加值系統

This thesis presents a 2D occupancy map from Gmapping of LiDAR Simultaneous Localization and Mapping (SLAM) integrated with Visual SLAM to achieve a robust path planning system and navigation. The proposed system also integrates with ORB-SLAM2 to derive the 6DoF for keyframe based approach which reduces the computation time. The LiDAR SLAM is the conventional method to extract the 2D occupancy map of the environment. However, the LiDAR sensor cannot detect the objects above the specific height. So, the Visual based SLAM’s have been popular because of their robust relocalization. So, these two kinds of maps are integrated for an enhanced 2D map. The pointclouds generated in Visual SLAM are refined and outliers are eliminated. Later, they are projected on to XZ plane eliminating the y coordinates. The proposed method is associated with a Service Robot to derive the LiDAR map data from LiDAR Sensor and develop the Visual SLAM map from RGB-D camera. The experiment results show the position and the dimensions of the object from the Robot along with the integrated 2D occupancy map.

TABLE OF CONTENTS
PAGE
ACKNOWLEDGMENTS i
ABSTRACT ii
TABLE OF CONTENTS iii
LIST OF FIGURES vii
LIST OF TABLES ix
LIST OF ABBREVATIONS x
1 INTRODUCTION 1
1.1 Research Background 1
1.2 Research Challenges 2
1.2.1 Challenges in Pose Estimation 2
1.2.2 Challenges in Pointcloud Registration 2
1.2.3 Challenges in Integrating the 2D occupancy maps of Visual SLAM and LiDAR SLAM 3
1.3 Motivation and Objectives 3
1.4 Contributions 4
1.5 Thesis Organization 4
2 RELATED WORKS 6
2.1 Related works on Visual SLAM 6
2.1.1 Sensors 6
2.1.2 Feature Matching 6
2.1.3 Pose Estimation 7
2.1.4 Loop Closure 7
2.1.5 Bundle Adjustment 7
2.2 Related Works on ORB-SLAM2 8
2.2.1 Interpreting KeyframeTrajectory.txt File 9
2.2.2 Map Points Extraction 10
2.2.3 RANSAC Algorithm 11
2.2.4 Bag of Words 12
2.3 Related Works on LiDAR SLAM 14
2.4 Related Works on YOLO Object Detection 15
2.5 Related Works on Mask R-CNN 16
3 INTEGRATION OF 2D MAPS OF LIDAR AND VISUAL SLAM 18
3.1 Overview of the Proposed Method 18
3.2 Visual SLAM 18
3.2.1 RGB-D Images 19
3.2.2 ORB-SLAM2 20
3.2.3 Keyframes Extraction 20
3.2.4 Image and Pointcloud Processing 20
3.2.5 Generating 2D Pointcloud Map of Visual SLAM 28
3.3 LiDAR SLAM 29
3.3.1 Robot Sensor Data 29
3.3.2 2D Map of LiDAR SLAM 29
3.4 2D Map Fusion of Visual and LiDAR SLAMs 30
3.4.1 Scaling 30
3.4.2 Rotation 30
3.4.3 Translation 30
4 EXPERIMENT RESULTS 34
4.1 Robot and System Specifications 34
4.2 Experiment Results 35
4.2.1 Analysis on Pointclouds of the Sequence 36
4.2.2 Analysis on Integration of 2D Maps 40
5 SUMMARY AND CONCLUSIONS 48
REFERENCES 49

[1]T. Bailey, H. J. I. r. Durrant-Whyte, and a. magazine, “Simultaneous localization and mapping (SLAM): Part II,” vol. 13, no. 3, pp. 108-117, 2006.
[2]Y. Abdelrasoul, A. B. S. H. Saman, and P. Sebastian, "A quantitative study of tuning ROS gmapping parameters and their effect on performing indoor 2D SLAM." pp. 1-6.
[3]S.-H. Chan, P.-T. Wu, and L.-C. Fu, "Robust 2D indoor localization through laser SLAM and visual SLAM fusion." pp. 1263-1268.
[4]K.-S. Choi, S.-G. J. I. J. o. P. E. Lee, and Manufacturing, “Enhanced SLAM for a mobile robot using extended Kalman Filter and neural networks,” vol. 11, no. 2, pp. 255-264, 2010.
[5]J. Engel, T. Schöps, and D. Cremers, "LSD-SLAM: Large-scale direct monocular SLAM." pp. 834-849.
[6]W. Hess, D. Kohler, H. Rapp, and D. Andor, "Real-time loop closure in 2D LIDAR SLAM." pp. 1271-1278.
[7]G. Hu, S. Huang, L. Zhao, A. Alempijevic, and G. Dissanayake, "A robust rgb-d slam algorithm." pp. 1714-1719.
[8]C. Kerl, J. Sturm, and D. Cremers, "Dense visual SLAM for RGB-D cameras." pp. 2100-2106.
[9]R. Li, J. Liu, L. Zhang, and Y. Hang, "LIDAR/MEMS IMU integrated navigation (SLAM) method for a small UAV in indoor environments." pp. 1-15.
[10]R. Mur-Artal, and J. D. Tardós, "Fast relocalisation and loop closing in keyframe-based SLAM." pp. 846-853.
[11]R. Mur-Artal, and J. D. J. I. T. o. R. Tardós, “Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras,” vol. 33, no. 5, pp. 1255-1262, 2017.
[12]P. Newman, D. Cole, and K. Ho, "Outdoor SLAM using visual appearance and laser ranging." pp. 1180-1187.
[13]T. Taketomi, H. Uchiyama, S. J. I. T. o. C. V. Ikeda, and Applications, “Visual SLAM algorithms: a survey from 2010 to 2016,” vol. 9, no. 1, pp. 16, 2017.
[14]G. Klein, and D. Murray, "Parallel tracking and mapping for small AR workspaces." pp. 225-234.
[15]G. P. Huang, A. I. Mourikis, and S. I. J. I. T. o. R. Roumeliotis, “A quadratic-complexity observability-constrained unscented Kalman filter for SLAM,” vol. 29, no. 5, pp. 1226-1243, 2013.
[16]A. B. S. H. Saman, and A. H. Lotfy, "An implementation of SLAM with extended Kalman filter." pp. 1-4.
[17]F. Yu, Q. Sun, C. Lv, Y. Ben, and Y. J. M. P. i. E. Fu, “A SLAM algorithm based on adaptive cubature Kalman filter,” vol. 2014, 2014.
[18]J. Zhu, N. Zheng, Z. Yuan, Q. Zhang, X. Zhang, and Y. He, "A slam algorithm based on the central difference kalman filter." pp. 123-128.
[19]M. D. Breitenstein, F. Reichlin, B. Leibe, E. Koller-Meier, and L. Van Gool, "Robust tracking-by-detection using a detector confidence particle filter." pp. 1515-1522.
[20]H. T. Niknejad, A. Takeuchi, S. Mita, and D. J. I. T. o. I. T. S. McAllester, “On-road multivehicle tracking using deformable object model and particle filter with improved likelihood estimation,” vol. 13, no. 2, pp. 748-758, 2012.
[21]T. Zhang, C. Xu, and M.-H. Yang, "Multi-task correlation particle filter for robust object tracking." pp. 4335-4343.
[22]H. Strasdat, J. M. Montiel, A. J. J. I. Davison, and V. Computing, “Visual SLAM: why filter?,” vol. 30, no. 2, pp. 65-77, 2012.
[23]Y. Ke, and R. Sukthankar, "PCA-SIFT: A more distinctive representation for local image descriptors." pp. II-II.
[24]H. Bay, T. Tuytelaars, and L. Van Gool, "Surf: Speeded up robust features." pp. 404-417.
[25]M. Calonder, V. Lepetit, C. Strecha, and P. Fua, "Brief: Binary robust independent elementary features." pp. 778-792.
[26]E. Rublee, V. Rabaud, K. Konolige, and G. Bradski, "ORB: An efficient alternative to SIFT or SURF." pp. 2564-2571.
[27]P. J. J. J. o. t. A. s. a. Rousseeuw, “Least median of squares regression,” vol. 79, no. 388, pp. 871-880, 1984.
[28]O. Chum, J. Matas, and J. Kittler, "Locally optimized RANSAC." pp. 236-243.
[29]B. Triggs, P. F. McLauchlan, R. I. Hartley, and A. W. Fitzgibbon, "Bundle adjustment—a modern synthesis." pp. 298-372.
[30]P. Dollár, R. Appel, S. Belongie, P. J. I. t. o. p. a. Perona, and m. intelligence, “Fast feature pyramids for object detection,” vol. 36, no. 8, pp. 1532-1545, 2014.
[31]E. Karami, S. Prasad, and M. J. a. p. a. Shehata, “Image matching using SIFT, SURF, BRIEF and ORB: performance comparison for distorted images,” 2017.
[32]Y. Zhang, R. Jin, Z.-H. J. I. J. o. M. L. Zhou, and Cybernetics, “Understanding bag-of-words model: a statistical framework,” vol. 1, no. 1-4, pp. 43-52, 2010.
[33]R. Huang, J. Pedoeem, and C. Chen, "YOLO-LITE: a real-time object detection algorithm optimized for non-GPU computers." pp. 2503-2510.
[34]G. Bradski, and A. Kaehler, Learning OpenCV: Computer vision with the OpenCV library: " O'Reilly Media, Inc.", 2008.
[35]T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, "Microsoft coco: Common objects in context." pp. 740-755.
[36]K. He, G. Gkioxari, P. Dollár, and R. Girshick, "Mask r-cnn." pp. 2961-2969.
[37]Q.-Y. Zhou, J. Park, and V. J. a. p. a. Koltun, “Open3D: A modern library for 3D data processing,” 2018.
[38]P. J. Besl, and N. D. McKay, "Method for registration of 3-D shapes." pp. 586-606.
[39]P. Soille, Morphological image analysis: principles and applications: Springer Science & Business Media, 2013.
[40]Z. J. I. T. o. p. a. Zhang, and m. intelligence, “A flexible new technique for camera calibration,” vol. 22, no. 11, pp. 1330-1334, 2000.
[41]S. Thrun, W. Burgard, and D. Fox, "A real-time algorithm for mobile robot mapping with applications to multi-robot and 3D mapping." pp. 321-328.
[42]H. Andreasson, T. Duckett, and A. Lilienthal, "Mini-SLAM: Minimalistic visual SLAM in large-scale environments based on a new interpretation of image similarity." pp. 4096-4101.
[43]S. García, M. E. López, R. Barea, L. M. Bergasa, A. Gómez, and E. J. Molinos, "Indoor SLAM for micro aerial vehicles control using monocular camera and sensor fusion." pp. 205-210.
[44]D. M. Cole, and P. M. Newman, "Using laser range data for 3D SLAM in outdoor environments." pp. 1556-1563.
[45]Smistad Erik, Falch Thomas L, Bozorgi Mohammadmehdi, Elster Anne C and Lindseth Frank, "Medical image segmentation on GPUs–A comprehensive review", 2015.
[46]https://www.wikipedia.org/