臺灣博碩士論文加值系統

本研究針對全自主機器手臂智慧物件操作系統提出一個適合擴增與成長的軟體系統架構。機器手臂之全自主智慧物件操作系統都需要開發各種智慧視覺系統對物件進行辨識與定位並全自主控制機器手臂進行動作。然而現有的智慧視覺系統除數目眾多外，且各有其適用範圍。而且，更多的新視覺演算法和機器手臂物件操作模組也不斷被研發推出。一個實用的機器手臂物件操作軟體系統必須能隨時更新，並增加和更新其技術模組內容。因此針對這項問題，本研究設計出一套可擴增和成長的軟體系統架構，藉由事先規劃好的指定工作路徑與指定的子程式規格，達到可以方便更換辨識法而不必改寫主程式。本研究結合機械手臂控制與多種2D/3D機械視覺，開發出一套智慧型機器手臂全自主物件操作系統，可藉由環境中的深度相機對欲操作的物件先進行三維物體的辨識和六維姿態的估計，接著將指揮機械手臂移動到該物件預設的抓取點，接著使用手臂末端的2D相機，藉由物件上的標記點進行視覺伺服控制，指揮機器手臂移動到精確的抓取點抓取物件。本系統也根據各物件對不同智慧視覺模組的適用環境和可靠度進行登錄，增加機器手臂系統因應環境變異的執行強健性。本研究也藉由多個實驗以檢驗研發系統的能力，並仔細探討未來改進方向。

This research proposes a software system architecture suitable for amplification and growth for the fully autonomous robot arm object operating system. The autonomous intelligent object operating system needs to integrate with various intelligent vision systems for identifying and locating the object, and controlling the movement of the robot arm. However, the existing intelligent vision systems are numerous, and each has its scope of application. Moreover, more new visual algorithms and robot object operation modules are being continuously developed. A practical robot arm operating software system must be able to update and add new technical modules. Therefore, for this issue, this research designed a scalable and adapt-for-growth software system architecture. By preplanning the specified working directories and the specified specifications of subprogram, the user can easily replace the recognition codes without rewriting the main program. This research combines the robot arm control with a variety of 2D/3D vision systems to develop a smart robotic fully-automatic object operating system that can perform object recognition and 6D pose estimation of object in cluttered scenes by a 3D camera, control the robot arm to move to the preset grabbing point of the object, use the 2D camera at the end of the arm to manipulate the visual servo control based on the landmark on the object, and finally move the robot arm to the precise grabbing point and grab the object. The system also registers the applicable environment and reliability of different intelligent vision modules for each of the object in data set, and increases the robustness of the robot arm system in response to environmental variation. This research also examines the capabilities of the system through multiple experiments and carefully explores future directions for improvement.

摘要 I
Abstract II
圖目錄 V
表目錄 VIII
第一章 緒論 1
1-1 前言 1
1-2 研究動機與研究目的 3
1-2-1 研究動機 3
1-2-2 研究目的 4
1-3 本文架構 5
第二章 軟體系統設計 6
2-1 軟體系統設計概念 6
2-1-1 子程式規格與工作路徑規劃 6
2-1-2 人機介面 7
2-2 軟體系統運作流程 9
第三章 機械手臂抓取系統 11
3-1 相機系統 11
3-1-1 相機成像原理 11
3-1-2 內部參數 12
3-1-3 形變參數 13
3-1-4 外部參數 15
3-1-5 深度相機 17
3-2 機械手臂 19
3-2-1 機械手臂運動學 19
3-5-2 手臂末端點之座標轉換與控制 21
3-3 三維物體的辨識和六維姿態估計 23
3-4 視覺伺服控制 25
3-4-1 基於影像之視覺伺服系統(Image-Based Visual Servo, IBVS) 25
3-4-2 交互矩陣(Interaction matrix) 26
3-5 影像處理 – 人工標記點跟自然標記點偵測 29
3-5-1 人工標記點-1 29
3-5-2 人工標記點-2 32
3-5-3 自然標記點 35
第四章 實驗結果 39
4-1 系統結果 40
4-1-1 GUI介面與系統運行 40
4-1-2 註冊物件與辨識法更換 47
4-2 情境模擬 58
4-2-1 環境光源不足情境 58
4-2-2 相機自動對焦失敗情境 61
4-3 實驗結果與評論 64
4-4 不同類型手臂測試 67
第五章 結論與未來展望 71
5-1 結論 71
5-2 未來展望 72
參考文獻 73

1. Zeng, Andy, et al. "Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching." 2018 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2018.
2. van Osch, Michiel, et al. "Tele-operated service robots: ROSE." Automation in Construction 39 (2014): 152-160.
3. Heikkila, J. and O. Silven. A four-step camera calibration procedure with implicit image correction. in Computer Vision and Pattern Recognition, 1997. Proceedings., 1997 IEEE Computer Society Conference on. 1997. IEEE.
4. 針孔投影介紹，取自<https://zh-yue.wikipedia.org/wiki/%E6%9A%97%E7%AE%B1>
5. Duane, C.B., Close-range camera calibration.Photogramm. Eng, 1971. 37(8): p. 855-866.
6. Banglei Guan, Yang Shang, and Qifeng Yu, "Planar self-calibration for stereo cameras with radial distortion," Appl. Opt. 56, 9257-9267 (2017)
7. 相機校正介紹，取自<http://silverwind1982.pixnet.net/blog/post/153218861-pinhole-camera%3A-%E7%9B%B8%E6%A9%9F%E6%A0%A1%E6%AD%A3-%28camera-calibration%29>
8. Fryer, J.G. and D.C. Brown, Lens distortion for close-range photogrammetry.Photogrammetric engineering and remote sensing, 1986. 52(1): p. 51-58.
9. 相機座標轉換，取自http://marcelojo.org/
10. Kinect V1 基礎介紹，取自<https://zh.wikipedia.org/wiki/Kinect>
11. Jason Geng, "Structured-light 3D surface imaging: a tutorial," Adv. Opt. Photon. 3, 128-160 (2011)
12. 機器人學，取自<https://zh-tw.coursera.org/lecture/robotics1/3-2-shou-bi-ji-he-miao-shu-heeBp>
13. Vidal, Joel, Chyi-Yeu Lin, and Robert Martí. "6D pose estimation using an improved method based on point pair features." 2018 4th International Conference on Control, Automation and Robotics (ICCAR). IEEE, 2018.
14. François Chaumette, S. Hutchinson. Visual servo control, Part II: Advanced approaches. IEEE Robotics and Automation Magazine, Institute of Electrical and Electronics Engineers, 2007, 14 (1), pp.109-118.
15. E. Rub Lee, V. Rabaud, K. Konolige, G., "Bradski ORB: an efficient alternative to SIFT or SURF[C]", International Conference of Computer Vision(ICCV), pp. 2564-2571, 2011.
16. E. Rosten, T. Drummond, "Machine learning for highspeed corner detection[C]", European Conference on Computer Vision, pp. 430-443, 2006.
17. M. Calonder', V. Lepetit, C. Strecha, P. Fua, "Brief: Binary robust independent elementary features[C]", European Conference on Computer Vision, pp. 778-792, 2010.
18. Bay, Herbert, Tinne Tuytelaars, and Luc Van Gool. "Surf: Speeded up robust features." European conference on computer vision. Springer, Berlin, Heidelberg, 2006.
19. Lindeberg, Tony. "Scale invariant feature transform." (2012): 10491.