臺灣博碩士論文加值系統

現今許多先進國家逐漸受到人口老化與人口衰減衝擊，面臨勞工短缺以及勞動力成本上漲等問題。因此為了解決上述問題，產業界陸續採用機械手臂與工業自動化系統來減少成本並增加生產率。然而在進行較複雜的加工任務或物件取放任務中，機械手臂經常難以獨自完成。解決前述困難之可能解決方案為導入機器視覺系統以提供機械手臂視覺功能，因此本論文主要目的為研究與開發基於視覺之自動化物件取放系統。普遍上，用於自動化取放系統之機器視覺可分為兩種不同的組態－眼在手攝影機組態與眼對手攝影機組態。兩者差異在於，眼在手攝影機組態使用單一攝影機，且安裝於機械手臂的末端工具中心點上；而眼對手攝影機組態使用雙眼立體攝影機，並擺置在一固定位置上。此兩組態之架構可分為三個主要部份進行深入探討－機器視覺系統之校正、物體識別、以及物件座標系的轉換。實驗結果顯示本論文所開發之眼在手自動化物件取放系統與眼對手自動化物件取放系統都可成功地完成物件取放任務。此外，實驗結果也顯示眼在手系統擁有較高的精準度，但卻不易成功識別三維物件；相反地，眼對手系統較容易進行三維物件識別，但精準度卻相對較低，且需事先進行較複雜之系統校正與加工物件模型建立。

Nowadays, because of population aging and population decline, labor force has become insufficient and labor cost keeps rising up. Thus, most manufacturers adopt industrial automation systems with robotic arms in order to reduce costs and increase productivities. However, when dealing with complicated tasks, such as object pick-and-place, it is harder for robotic arms alone to complete them. One of the possible solutions to overcoming the aforementioned difficulties is to introduce machine vision into the robotic arm system. This thesis focuses on the development of vision-based automatic pick-and-place systems. Generally, there are two types of machine vision configuration – eye-in-hand and eye-to-hand. For the eye-in-hand configuration, a camera is attached on the tool center point of a robotic arm. For the eye-to-hand configuration, a stereo camera is placed on a fixed position. The structures of both configurations are similar and can be divided into three main sections – the calibration of machine vision system, object recognition, and transformations of object coordinates. Experimental results indicate that both the systems with eye-in-hand and eye-to-hand configurations can successfully perform automatic pick-and-place tasks. Furthermore, experimental results also indicate that the system with eye-in-hand has a higher accuracy, but is harder to recognize 3D objects. On the contrary, the system with eye-to-hand is easier to recognize 3D objects, but has a lower accuracy and requires more training and calibration.

ENGLISH ABSTRACT I
中文摘要 II
ACKNOWLEDGEMENT III
TABLE OF CONTENTS IV
LIST OF TABLES VIII
LIST OF FIGURES IX
CHAPTER 1 INTRODUCTION 1
1.1 RESEARCH MOTIVATION AND PURPOSE 1
1.2 LITERATURE REVIEW 2
1.3 STRUCTURE OF THE THESIS 6
CHAPTER 2 PINHOLE CAMERA MODEL AND BINOCULAR STEREO VISION 7
2.1 PINHOLE CAMERA 7
2.1.1 Structure of Camera 7
2.1.2 Pinhole Camera Model 8
2.1.3 Intrinsic Matrix 10
2.1.4 Extrinsic Matrix 13
2.1.5 Distortion Coefficients 16
2.2 BINOCULAR STEREO VISION 19
2.2.1 Structure of Binocular Stereo Vision 19
2.2.2 Image Rectification 20
2.2.3 Depth Estimation Based on Disparity 21
CHAPTER 3 CALIBRATION OF MACHINE VISION SYSTEM 24
3.1 DEFINITION OF TRANSFORMATION MATRIX 24
3.2 CALIBRATION RIG 27
3.3 CAMERA CALIBRATION 29
3.3.1 Basic Concept of Camera Calibration 29
3.3.2 The Procedure of Camera Calibration 31
3.4 STEREO CALIBRATION 32
3.4.1 Basic Concept of Stereo Calibration 32
3.4.2 The Procedure of Stereo Calibration 34
3.5 HAND-EYE CALIBRATION 35
3.5.1 Eye-in-Hand Calibration 36
3.5.2 Eye-to-Hand Calibration 37
CHAPTER 4 OBJECT RECOGNITION AND TRANSFORMATIONS OF OBJECT COORDINATES 39
4.1 OBJECT RECOGNITION 39
4.1.1 Interest Point Detection 40
4.1.1.1 SIFT 41
4.1.1.2 SURF 43
4.1.1.3 Canny 46
4.1.1.4 Harris 48
4.1.1.5 FAST 51
4.1.2 Interest Point Descriptor 52
4.1.2.1 SIFT Interest Point Descriptor 53
4.1.2.2 SURF Interest Point Descriptor 54
4.1.3 Match Point Error Elimination 55
4.1.3.1 Eliminate Match Point Error by RANSAC 55
4.1.3.2 Eliminate Match Point Error by Disparity Vector 57
4.1.3.3 Eliminate Match Point Error by Depth 58
4.1.4 Recognition of the Model 58
4.2 TRANSFORMATIONS OF OBJECT COORDINATES 59
4.2.1 Coordinate Transformations for Eye-in-Hand Configuration 60
4.2.2 Coordinate Transformations for Eye-to-Hand Configuration 62
CHAPTER 5 VSCLAB MACHINE VISION SYSTEM SOFTWARE
PROGRAM 64
5.1 BASIC CONCEPT OF OBJECT-ORIENTED DESIGN 64
5.1.1 Classification 64
5.1.2 Inheritance 65
5.1.3 Dynamic Polymorphism 66
5.2 VSCLAB MACHINE VISION SYSTEM 67
CHAPTER 6 EXPERIMENTAL SETUPS AND RESULTS 72
6.1 EXPERIMENTAL SETUPS 72
6.1.1 6-Axis Robotic Arms 72
6.1.2 Cameras and Lenses 73
6.1.3 Gripper and Nozzle 74
6.1.4 Target Objects 76
6.1.5 Environmental Setup for Eye-in-Hand Configuration 77
6.1.5.1 Setup of Single Eye-in-Hand Camera 78
6.1.5.2 Structure of Single Camera Holder 79
6.1.5.3 Experimental Platform 80
6.1.6 Environmental Setup for Eye-to-Hand System 80
6.1.6.1 Setup of Stereo Camera 81
6.1.6.2 Structure of Stereo Camera Holder 82
6.1.6.3 Experimental platform 84
6.2 EXPERIMENTAL RESULTS 85
6.2.1 Camera Calibration Results 86
6.2.2 Stereo Calibration Results 89
6.2.3 Hand-Eye Calibration Results 93
6.2.4 Object Recognition Results 96
6.2.4.1 2D Object Recognition Results 96
6.2.4.2 3D Object Recognition Results 99
6.2.5 Object Pick-and-Place Results 104
6.2.6 Comparison of the two machine vision systems 106
CHAPTER 7 CONCLUSION AND FUTURE WORKS 108
7.1 CONCLUSION 108
7.2 FUTURE WORKS 109
REFERENCES 110

[1]P. J. Sanz, A. P. del Pobil, J. M. Iñesta, and G. Recatalá, “Vision-guided grasping of unknown objects for service robots, in Proc. of the IEEE Int. Conf. on Robotics ＆ Automation, 1998, vol. 4, pp. 3018-3025.
[2]A. Schrott, “Feature-based camera-guided grasping by an eye-in-hand robot, in Proc. of the IEEE Int. Conf. on Robotics ＆ Automation, 1992, vol. 2, pp. 1832-1837.
[3]G. D. Haǵer, W. C. Change, and A. S. Morse, “Robot hand-eye coordination based on stereo vision, IEEE Control Systems, vol. 15, pp. 30-39, 1995.
[4]H. J. Tsai, “Application of homography matrix based 3D reconstruction algorithm on six-axis articulated robot, M.S. thesis, Dept. Elect. Eng., National Cheng Kung Univ., Tainan, Taiwan, June 26, 2014.
[5]C. C. Lin, “Study on vision based object grasping of industrial manipulator, M.S. thesis, Dept. Elect. Eng., National Cheng Kung Univ., Tainan, Taiwan, July 13, 2015.
[6]Z. Zhang, “A flexible new technique for camera calibration, IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, pp. 1330-1334, 2000.
[7]R. Y. Tsai. “A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses, IEEE Journal of Robotics and Automation, vol. 3, pp. 323-344, 1987.
[8]W. Li, T. Gee, H. Friedrich, and P. Delmas, “A practical comparison between Zhang’s and Tsai’s calibration approaches, in Proc. of the 29th Int. Conf. on Image and Vision Computing New Zealand, 2014, pp. 166-171.
[9]Itseez. (2016). OpenCV | OpenCV [Online]. Available: http://opencv.org/
[10]HALCON Application Note: Machine Vision in World Coordinates, MVTec Software GmbH, München, Germany, 2003.
[11]T. C. Chiang, “Study on Hand-Eye Calibration of Six-Axis Articulated Robot, M.S. thesis, Dept. Elect. Eng., National Cheng Kung Univ., Tainan, Taiwan, Aug. 2014.
[12]Wikipedia. (2016, Apr. 15). Outline of Object Recognition [Online]. Available:
https://en.wikipedia.org/wiki/Outline_of_object_recognition
[13]K. Alhamzi, M. Elmogy, and S. Barakat, “3D object recognition based on image features: a survey, International Journal of Computer and Information Technology, vol. 3, pp. 651-660, May 2014.
[14]D. G. Lowe, “Distinctive image features from scale-invariant keypoints, International Journal of Computer Vision, vol. 60, pp. 91-110, 2004.
[15]H. Bay, A. Ess, T. Tuytelaars, and L. V. Gool, “SURF: Speeded up robust features, Computer Vision and Image Understanding, vol. 110, pp. 346-359, 2008.
[16]J. Canny, “A computational approach to edge detection, IEEE Trans. Pattern Anal. Mach. Intell., vol. 8, pp. 679-698, Nov. 1986.
[17]C. Harris and M. Stephens, “A combined corner and edge detector, in Proc. of the 4th Alvey Vision Conference, 1988, pp. 147-151.
[18]E. Rosten and T. Drummond, “Machine learning for high-speed corner detection, in Computer Vision–ECCV 2006, ed: Springer, 2006, pp. 430-443.
[19]K. S. Fu, R. C. Gonzalez, and C. S. G. Lee, Robotics: Control Sensing Vision and Intelligence, Vis: Tata McGraw-Hill Education, 1987.
[20]M. W. Spong, S. Hutchinson, and M. Vidyasagar, Robot Modeling and Control, Wiley, 2006.
[21]G. Bradski and A. Kaehler, “Camera models and calibration, in Learning OpenCV: Computer Vision with the OpenCV Library, O'Reilly Media, 2008, ch. 11, pp. 370-404.
[22]G. Bradski and A. Kaehler, “Projection and 3D vision, in Learning OpenCV: Computer Vision with the OpenCV Library, O'Reilly Media, 2008, ch. 12, pp. 405-458.
[23]Wikipedia. (2016, Feb. 26). Distortion (optics) [Online]. Available:
https://en.wikipedia.org/wiki/Distortion_(optics)
[24]OpenCV Dev. Team. (2016, May 3). Camera Calibration and 3D Reconstruction [Online]. Available:
http://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html
[25]OpenCV Dev. Team. (2016, May 3). Camera Calibration with OpenCV [Online]. Available:
http://docs.opencv.org/2.4/doc/tutorials/calib3d/camera_calibration/camera_calibration.html
[26]Wikipedia. (2016, May 1). EmguCV: OpenCV in .NET (C#, VB, C++ and more) [Online]. Available:
http://www.emgu.com/wiki/index.php/Main_Page
[27]J. More, “The Levenberg-Marquardt Algorithm, Implementation, and Theory, Numerical Analysis, G.A. Watson, ed., Springer-Verlag, 1977.
[28]R. T. Chin and C. R. Dyer, “Model-based recognition in robot vision, ACM Computing Surveys, vol. 18, no. 1, pp. 67-108, Mar. 1986.
[29]R. A. Young, R. M. Lesperance, and W. W. Meyer, “The Gaussian Derivative model for spatial-temporal vision: I. Cortical model, Spatial Vision, vol. 14, no. 3-4, pp. 261-319, 2001.
[30]T. Lindeberg, “Feature detection with automatic scale selection, Int. J. of Comput. Vision, vol. 30, no. 2, pp. 79-116, Nov. 1998.
[31]P. Viola and M. Jones, “Rapid object detection using a boosted cascade of simple features, in Proc. of the 2001 IEEE Comput. Soc. Conf. on Comput. Vision and Pattern Recognition, CVPR 2001, 2001, vol. 1, p. I-511 – I-518.
[32]W. Hoff. (2012, Feb. 1). CSCI 512 - Lecture 09-1 Edge Detection [Online]. Available: https://www.youtube.com/watch?v=07qwMn2c5iY
[33]H. P. Moravec, “Obstacle avoidance and navigation in the real world by a seeing robot rover, Robotics Inst., Carnegie Mellon Univ. ＆ doctoral dissertation, Stanford Univ., Tech. Rep. CMU-RI-TR-80-03, Sept. 1980.
[34]M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography, Commun. of the ACM, vol. 24, no. 6, pp. 381-395, June 1981.
[35]R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. Cambridge Univ. Press, 2004.
[36]M. Weisfeld, The Object-Oriented Thought Process: An Object Lesson Plan, 3rd ed. Addison-Wesley Professional, 2008. Reprinted in Just the Facts101 e-StudyGuides, Cram101, 2013.
[37]M. R. Blaha and J. R. Rumbaugh, Object-Oriented Modeling and Design with UML, 2nd ed. Pearson, 2004.
[38]MVTec Software GmbH. (2016). HALCON: the power of machine vision [Online]. Available: http://www.halcon.com/
[39]MVTec Software GmbH. (2016). HALCON Operator Reference 12.0.2 [Online]. Available:
http://www.halcon.com/download/reference/evaluate_class_gmm.html
[40]P. Torgashov. (2014, Jun. 8). Contour Analysis for Image Recognition in C# [Online]. Available:
http://www.codeproject.com/Articles/196168/Contour-Analysis-for-Image-Recognition-in-C