臺灣博碩士論文加值系統

本研究使用機器人作業系統(ROS)作為控制六個自由度機器臂的平台，並結合深度攝影機進行物件辨識。在物件辨識方面，採用了Mask R-CNN，透過其語意分割優勢來找出物體邊緣並進行姿態計算，使機械臂可以以正確角度進行夾取。Mask R-CNN是一種用於圖像分割的深度學習模型，它可以同時進行物件檢測和語意分割，即在圖像中標記每個像素屬於哪個物件。透過使用Mask R-CNN，可以提高物件檢測和分割的準確度，進而提高機器臂的夾取成功率。
本研究使用手機拍攝照片，並透過物體在不同環境下、甚至多個物體重疊的照片作為驗證集。在模型訓練過後，本研究使用產生的平均精確度(mean average precision, mAP)以及實際辨識結果來判斷訓練成效。在機器臂控制方面，本研究使用OMPL開源運動規劃庫來規劃六軸機器臂的運動路徑，並透過MoveIT作為OMPL與ROS的接口，使得OMPL能夠更簡易的在ROS上運行。最後，本研究將手臂控制、影像辨識、深度回饋結合以達到物體姿態辨識及夾取之結果。

In this study, a robot operating system (ROS) is used as a platform to control a six-degree-of-freedom robotic arm, combined with a depth camera for object recognition. In terms of object recognition, Mask R-CNN is used to find the edge of the object and perform pose calculation through its semantic segmentation advantages, so that the robotic arm can grip at the correct angle. Mask R-CNN is a deep learning model for image segmentation. It can perform object detection and semantic segmentation at the same time, that is, to mark which object each pixel belongs to in the image. By using Mask R-CNN, the accuracy of object detection and segmentation can be improved, thereby improving the success rate of robotic arm gripping.
This study uses mobile phones to take photos, and uses photos of objects in different environments, or even overlapping multiple objects, as a validation set. After the model is trained, this study uses the generated mean average precision (mAP) and actual recognition results to judge the training effectiveness. In terms of robot arm control, this research uses the OMPL open source motion planning library to plan the motion path of the six-axis robot arm, and uses MoveIT as the interface between OMPL and ROS, so that OMPL can run on ROS more easily. Finally, this research combines arm control, image recognition, and depth feedback to achieve the results of object pose recognition and gripping.

摘要 i
ABSTRACT ii
誌謝 iv
目錄 v
表目錄 viii
圖目錄 ix
1 第一章 緒論 1
1.1 研究背景 1
1.2 研究動機與目的 1
1.3 文獻探討 2
1.4 論文架構 3
2 第二章 系統架構與軟硬體說明 5
2.1 系統概述 5
2.2 軟體介紹 6
2.2.1機器人作業系統ROS 6
2.2.2 MoveIT 7
2.3 硬體介紹 8
2.3.1 筆電(TUF Gaming A15 FA506II) 8
2.3.2 Raspberry Pi 4 9
2.3.3 Realsense D435 11
2.3.4 Niryo Ned 13
3 第三章 Mask R-CNN深度學習 16
3.1 深度學習介紹 16
3.2 Computer Vision (CV) 16
3.2.1影像分類（Image Classification） 17
3.2.2 物件偵測（Object Detection） 17
3.3 卷積神經網路 18
3.3.1卷積層 19
3.3.2池化層 21
3.3.3全連接層 22
3.4 Mask R-CNN 23
4 第四章 六軸機械手臂運動學 26
4.1 順向運動學 26
4.2 MoveIT 28
4.2.1 MoveIt Action 29
4.2.2 URDF 與MoveIt Setup 30
4.3手眼校正 32
4.3.1相機標定 32
5 第五章 實驗方法設計與實現 34
5.1 實驗流程設計 34
5.2標記工具 34
5.3座標轉換 35
6 第六章 實驗結果與討論 37
6.1.1 Mask R-CNN實際辨識畫面 37
6.1.2 Mask R-CNN訓練結果 39
6.1.3 Mask R-CNN透過像素點位置找出物體姿態 39
6.2 實際夾取結果與討論 40
第七章 結論與未來展望 42
參考文獻 43

[1]蘇立珩。「基於機器學習與影像之機械手臂抓取」。碩士論文，國立清華大學動力機械工程學系，2018。https://hdl.handle.net/11296/3h7749。Shaoqing Ren, Kaiming He, Ross Girshick and Jian Sun, “Faster R-CNN: towards real-time object detection with region proposal networks,” NIPS'15: Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1, pp. 91–99, Dec. 2015.
[2]X. Zhu, Z. Wang, C. Li and X. Sun, "Research on Artificial Intelligence Network Based on Deep Learning," 2021 2nd International Conference on Information Science and Education (ICISE-IE), Chongqing, China, 2021, pp. 613-617, doi: 10.1109/ICISE-IE53922.2021.00145. Xiaohong Sun;Jinan Gu;Rui Huang ;Rong Zou;Benjamin Giron Palomares,” Surface Defects Recognition of Wheel Hub Based on Improved Faster R-CNN,”Apr.2019.
[3]H. Hu, H. Cai, Z. Ma and W. Wang, "Semantic segmentation based on semantic edge optimization," 2021 International Conference on Electronic Information Engineering and Computer Science (EIECS), Changchun, China, 2021, pp. 612-615, doi: 10.1109/EIECS53707.2021.9587939.
[4]J. Deng, W. Qu and S. Fang, "A High Accuracy and Recall Rate 6D Pose Estimation Method Using Point Pair Features for Bin-picking," 2022 34th Chinese Control and Decision Conference (CCDC), Hefei, China, 2022, pp. 6056-6061, doi: 10.1109/CCDC55256.2022.10034147. J. Redmon, S. Divvala, R. Girshick and A. Farhadi, "You only look once: unified, real-time object detection," arXiv preprint, 2015, arXiv:1506.02640.
[5]https://github.com/matterport/Mask_RCNN
[6]江偉銓（2022）。無人機上的人物檢測與性能評估。﹝碩士論文。國立臺北科技大學﹞臺灣博碩士論文知識加值系統。 https://hdl.handle.net/11296/82smc6。
[7]https://github.com/ultralytics/yolov3
[8]https://github.com/Tianxiaomo/pytorch-YOLOv4
[9]yolo v5 S. A. Bhavan and Mohana, "YOLOv5 Crop Detection Deep Learning Model using Artificial Intelligence (AI) and Edge Computing," 2022 International Conference on Augmented Intelligence and Sustainable Systems (ICAISS), Trichy, India, 2022, pp. 21-24, doi: 10.1109/ICAISS55157.2022.1001089
[10]林子翔（2022）。基於改良之 Mask R-CNN 建立 植物葉片分割系統。﹝碩士論文。國立中正大學﹞臺灣博碩士論文知識加值系統。 https://hdl.handle.net/11296/67cd2j。
[11]Mask R-CNN 簡介https://ivan-eng-murmur.medium.com/%E7%89%A9%E4%BB%B6%E5%81%B5%E6%B8%AC-s9-mask-r-cnn-%E7%B0%A1%E4%BB%8B-99370c98de28

[12]MoveIt!入門教程-簡介 https://www.ncnynl.com/archives/201610/1028.html
[13]Raspberry Pi https://www.raspberrypi.com/products/raspberry-pi-4-model-b/
[14]Niryo Ned https://docs.niryo.com/product/ned/v4.0.0/en/index.html
[15]卷積神經網路（Convolutional neural network， CNN） — CNN運算流程 https://chih-sheng-huang821.medium.com/%E5%8D%B7%E7%A9%8D%E7%A5%9E%E7%B6%93%E7%B6%B2%E8%B7%AF-convolutional-neural-network-cnn-cnn%E9%81%8B%E7%AE%97%E6%B5%81%E7%A8%8B-ecaec240a631
[16]深度學習-卷積神經網路-Pooling Layer(池化層) https://ithelp.ithome.com.tw/articles/10250745
[17]全連接層（fully connected layer）的通俗理解 https://blog.csdn.net/qq_39521554/article/details/81385159
[18]B. Zhang, "Computer vision vs. human vision," 9th IEEE International Conference on Cognitive Informatics (ICCI'10), Beijing, China, 2010, pp. 3-3, doi: 10.1109/COGINF.2010.5599750.
[19]物件偵測 Object Detection 概述 https://medium.com/ching-i/%E7%89%A9%E4%BB%B6%E5%81%B5%E6%B8%AC-object-detection-%E6%A6%82%E8%BF%B0-d4f71cafcd77
[20]A. Tahraoui, R. Kheddam, A. Bouakache and A. Belhadj-Aissa, "Affinity propagation for unsupervised classification of remotely sensed images," 2017 International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), Fez, Morocco, 2017, pp. 1-6, doi: 10.1109/ATSIP.2017.8075579.
[21]Image segmentation https://en.wikipedia.org/wiki/Image_segmentation
[22]ROI Pooling和ROI Align https://zhuanlan.zhihu.com/p/73138740
[23]ROI Align的基本原理及rpn与rcnn head锚框标签制作 https://zhuanlan.zhihu.com/p/73113289
[24]Segmentation Label 標註教學 https://medium.com/ching-i/segmentation-label-%E6%A8%99%E8%A8%BB%E6%95%99%E5%AD%B8-26b8179d661
[25]世界座標系和相機座標系,圖像座標系的關係https://www.twblogs.net/a/5ca83c19bd9eee59d3332465