臺灣博碩士論文加值系統

近年來隨著機器人學的進步，機器人的應用不再侷限於工業上，許多服務型機器人逐漸出現在日常生活中，如：家庭照顧、運輸、醫療等。而隨著應用層面與時代背景的改變，不管是在服務型機器人亦或是工業型機器人的應用中，機器人不再只負責單純的夾取任務，現在還須具備物體識別的能力，才能因應機器人的發展趨勢。因此，本論文提出一個基於深度學習網路的機器手臂視覺系統，可同時進行對物體進行識別與夾取點偵測。
本論文所提出之機器手臂視覺系統為 Eye-to-hand 架構，將 RGB-D 感測器安裝至工作環境的固定位置中，讀取機器手臂工作區域的 RGB-D 影像，再進行相應的處理，其處理主要包含三個步驟。首先，是對輸入影像進行前處理，主要進行深度資訊的正規化與透過物體輪廓偵測物體的中心，並裁切相應大小的 RGB-D 影像當作深度學習網路的輸入。其次，透過深度學習網路 (Res2NeXt) 來對輸入影像中的物體進行識別與夾取點偵測，在此部分，本論文使用Grouped Convolutions 的概念對 Res2NeXt 進行簡化，使其變得簡明易懂與易模組化，並結合 SE block 來增加深度學習網路的效能，使深度學習網路輸出的結果更加精確。最後，將辨識得到的結果 (如：夾取框位置與方向) 進行相應的轉換，轉換至 Robot base 座標系，利用逆向運動學計算機器手臂的移動路徑，最終實現夾取動作。
實驗結果顯示，本論文所提出的方法可有效地對物體進行識別與夾取點偵測。實驗中進行夾取數據集的收集，共兩種不同感測器俯角數據集，分別對應不同的工作範圍。我們將兩種數據集透過深度學習網路進行訓練，並進行實際機器手臂夾取測試，物體識別成功率分別為 93.8% 與 95.0%，而物體夾取成功率分別為 96.2% 與 93.8%。本論文的深度學習網路亦在最廣泛用於機器手臂的 Cornell Grasp Dataset 上進行驗證，透過評估指標計算出的夾取點偵測結果準確率 (Accuracy) 可達到89.4%，其效果優於其他類型的深度學習網路。

摘要 I
目錄 III
圖目錄 V
表目錄 VII
第1章 第一章 緒論 1
1.1 研究動機與背景 1
1.2 相關研究 2
1.2.1 夾取點偵測 2
1.2.2 物體識別 5
1.2.3 本論文架構 6
第2章 第二章 輸入影像前處理 9
2.1 深度資訊正規化 9
2.2 物體中心偵測 9
2.3 裁切物體區域 11
第3章 第三章 基於深度學習之物體識別與最佳夾取點偵測 12
3.1 卷積神經網路 (CNN) 12
3.2 基於Res2NeXt深度學習網路的物體識別與最佳夾取點偵測 13
3.2.1 Res2Net residual block 架構 15
3.2.2 Res2NeXt residual block架構 16
3.2.3 SE block (Squeeze-and-Excitation block) 架構 18
3.2.4 Res2NeXt residual block 結合 SE block 架構 19
3.2.5 損失函數 (Loss Function) 19
3.2.6 評估指標 (Evaluation index) 20
第4章 第四章 夾取點座標系轉換與機器手臂控制 21
4.1 夾取點座標系轉換 21
4.2 Camera base 與 Robot base 轉換 22
第5章 第五章 實驗結果與討論 25
5.1 實驗環境 25
5.2 數據集 (Dataset) 27
5.2.1 Cornell Grasp Dataset 27
5.2.2 本實驗收集的數據集 28
5.3 深度學習網路架構比較 29
5.3.1 ResNet、Res2Net與Res2NeXt實驗結果比較 30
5.3.2 使用RGB-D、RGB與深度影像訓練的實驗結果比較 31
5.3.3 結合 SE block 的實驗結果比較 33
5.3.4 增加輸入影像通道數的實驗結果比較 34
5.3.5 不同俯角數據集訓練的深度學習網路實驗結果比較 35
5.4 與其他文獻的比較 35
5.4.1 基於 Cornell Grasp Dataset 訓練之實驗結果比較 35
5.4.2 基於本實驗收集數據集訓練之實驗結果比較 37
5.5 實際機器手臂抓取測試 38
5.5.1 深度學習網路結合機器手臂實現夾取動作 38
5.5.2 本系統機器手臂實際物體識別成功率與夾取成功率 40
5.5.3 複雜場景中的測試 44
第6章 第六章 結論與未來工作 47
6.1 結論 47
6.2 未來工作 47
參考文獻 49

[1]Siddarth Jain, and Brenna Argall, “Grasp Detection for Assistive Robotic Manipulation,” Proc. of IEEE Int’l Conf. on Robotics and Automation (ICRA), pp.2015-2021, 2016.
[2]Yi-Lun Huang, Sheng-Pi Huang, Hsiang-Ting Chen, Yi-Hsuan Chen, Chin-Yin Liu, and Tzuu-Hseng S. Li, “A 3D vision based object grasping posture learning system for home service robots,” Proc. of IEEE Int’l Conf. on System, Man, and Cybernetics (SMC), pp.2690-2695, 2017.
[3]Yu-Chi Lin, Shao-Ting Wei, and Li-Chen Fu, “Grasping unknown objects using depth gradient feature with eye-in-hand RGB-D sensor,” Proc. of IEEE Int’l Conf. on Automation Science and Engineering (CASE), pp.1258-1263, 2014.
[4]Simon Christoph Stein, Markus Schoeler, Jeremie Papon, and Florentin Wörgötter, “Object Partitioning Using Local Convexity,” Proc. of IEEE Int’l Conf. on Computer Vision and Pattern Recognition (CVPR), pp.304-377, 2014.
[5]Anh Nguyen, Dimitrios Kanoulas, Darwin G. Caldwell, and Nikos G. Tsagarakis, “Detecting object affordances with Convolutional Neural Networks,” Proc. of IEEE Int’l Conf. on Intelligent Robots and Systems (IROS), pp.2765-2770, 2016.
[6]Edward Johns, Stefan Leutenegger, amd Andrew J. Davison, “Deep Learning a Grasp Function for Grasping under Gripper Pose Uncertainty,” Proc. of IEEE Int’l Conf. on Intelligent Robots and Systems (IROS), pp.4461-4468, 2016.
[7]Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, “Deep Residual Learning for Image Recognition,” Proc. of IEEE Int’l Conf. on Computer Vision and Pattern Recognition (CVPR), pp.770-778, 2016.
[8]Sulabh Kumra, and Christopher Kanan, “Robotic Grasp Detection using Deep Convolutional Neural Networks,” Proc. of IEEE Int’l Conf. on Intelligent Robots and Systems (IROS), pp.769-776, 2017.
[9]Xinwen Zhou, Xuguang Lan, Hanbo Zhang, Zhiqiang Tian, Yang Zhang, and Narming Zheng, “Fully Convolutional Grasp Detection Network with Oriented Anchor Box,” Proc. of IEEE Int’l Conf. on Intelligent Robots and Systems (IROS), pp.7223-7230, 2018.
[10]Shehan Caldera, Alexander Rassau, and Douglas Chai, “Robotic Grasp Pose Detection Using Deep Learning,” Proc. of Int’l Conf. on Control, Automation, Robotics and Vision (ICARCV), pp.1966-1972, 2018.
[11]Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” Proc. of Advances in Neural Information Processing Systems (NIPS), pp.1-9, 2012.
[12]Matthew Zeiler, and Rob Fergus, “Visualizing and Understanding Convolutional Networks,” Proc. of European Conf. on Computer Vision (ECCV), pp.818-833, 2014.
[13]Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich, “Going deeper with convolutions,” Proc. of IEEE Int’l Conf. on Computer Vision and Pattern Recognition (CVPR), pp.1-9, 2015.
[14]Shang-Hua Gao, Ming-Ming Cheng, Kai Zhao, Xin-Yu Zhang, Ming-Hsuan Yang, and Philip Torr, “Res2Net: A New Multi-scale Backbone Architecture,” arXiv preprint arXiv: 1904.01169v1, 2019.
[15]Jie Hu, Li Shen, and Gang Sun, “Squeeze-and-Excitation Networks,” Proc. of IEEE Int’l Conf. on Computer Vision and Pattern Recognition (CVPR), pp.7132-7141, 2018.
[16]Di Guo, Tao Kong, Fuchun Sun, and Huaping Liu, “Object discovery and grasp detection with a shared convolutional neural network,” Proc. of IEEE Int’l Conf. on Robotics and Automation (ICRA), pp.2038-2043, 2016.
[17]Sebastian Ruder, “An Overview of Multi-Task Learning in Deep Neural Networks,” arXiv preprint arXiv: 1706.05098v1, 2017.
[18]Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He, “Aggregated Residual Transformations for Deep Neural Networks,” Proc. of IEEE Int’l Conf. on Computer Vision and Pattern Recognition (CVPR), pp.5987-5995, 2017.
[19]Yun Jiang, Stephen Moseson, and Ashutosh Saxena, “Efficient grasping from RGBD images: Learning using a new rectangle representation,” Proc. of IEEE Int’l Conf. on Robotics and Automation (ICRA), pp.3304-3311, 2011.
[20]Joseph Redmon, and Anelia Angelova “Real-Time Grasp Detection Using Convolutional Neural Networks,” Proc. of IEEE Int’l Conf. on Robotics and Automation (ICRA), pp.1316-1322, 2015.
[21]Ian Lenz, Honglak Lee, and Ashutosh Saxena, “Deep Learning for Detecting Robotic Grasps,” Proc. of Int’l Journal of Robotics Research, pp.705-724, 2015.
[22]Di Guo, Fuchun Sun, Huaping Liu, Tao Kong, Bin Fang, and Ning Xi, “A hybrid deep architecture for robotic grasp detection,” Proc. of IEEE Int’l Conf. on Robotics and Automation (ICRA), pp.1609-1614, 2017.
[23]Edward Johns, Stefan Leutenegger, and Andrew J. Davison, “Deep learning a grasp function for grasping under gripper pose uncertainty,” Proc. of IEEE Int’l Conf. on Intelligent Robots and Systems (IROS), pp.4461-4468, 2016.
[24]Cornell University, “Robot Learning Lab: Learning to Grasp,” Available: http://pr.cs.cornell.edu/grasping/rect_data/data.php.
[25]Fisher Yu, Dequan Wang, Evan Shelhamer, and Trevor Darrell, “Deep Layer Aggregation,” Proc. of IEEE Int’l Conf. on Computer Vision and Pattern Recognition (CVPR), pp.2403-2412, 2018.
[26]Dongwon Park, Yonghyeok Seo, and Se Young Chun, “Real-Time, Highly Accurate Robotic Grasp Detection using Fully Convolutional Neural Networks with High-Resolution Images,” arXiv preprint arXiv: 1809.05828v1, 2018.