(3.236.222.124) 您好!臺灣時間:2021/05/19 11:08
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果

詳目顯示:::

我願授權國圖
: 
twitterline
研究生:吳秉謙
研究生(外文):Bing-CianWu
論文名稱:基於頭戴式顯示器與擴增虛擬技術實現遠端操控微組裝系統
論文名稱(外文):Implementation of head-mounted display with augmented virtuality in micro-assembly teleoperation systems
指導教授:張仁宗張仁宗引用關係
指導教授(外文):Ren-Jung Chang
學位類別:碩士
校院名稱:國立成功大學
系所名稱:機械工程學系
學門:工程學門
學類:機械工程學類
論文種類:學術論文
論文出版年:2020
畢業學年度:108
語文別:中文
論文頁數:135
中文關鍵詞:遠端操控擴增虛擬混合實境微組裝系統手眼配置
外文關鍵詞:TeleoperationAugmented virtualityMixed realityMicro-assembly systemEye-in-hand
相關次數:
  • 被引用被引用:0
  • 點閱點閱:39
  • 評分評分:
  • 下載下載:0
  • 收藏至我的研究室書目清單書目收藏:0
本研究將擴增虛擬(Augmented virtuality)技術應用於微組裝系統,操作者透過頭戴式顯示器監控並使用手把操作,達成遠端即時操控微物件組裝之應用。虛擬環境的創建,首先透過CAD軟體繪製仿真微組裝機台,接著匯入Unity遊戲引擎中;真實世界之網路攝像頭架設,採用手眼配置(Eye-in-hand)進行安裝。核心流程為將估測出的攝像頭內外參數與畸變係數,結合光線追蹤演算法,推得微物件銅棒之端點與組裝物軸孔的三維座標,並將估測之三維座標,經由網路傳輸至已建立虛擬環境之遠處客戶端主機中,使虛擬環境中的虛擬物件與虛擬組裝物上特徵點具備真實世界之位置座標資訊,達到使用擴增虛擬技術遠端操控微組裝之應用;此應用令操作者即使身在遠端操作虛擬微組裝機台,亦能有身歷其境的感受,並且能以任意視角或尺度下觀測軸孔位置,增加組裝成功機率,最後本文完成以雙顆網路攝像頭於伺服器端進行觀測,操作者於遠處客戶端透過頭盔顯示器監控操作,成功完成微物件組裝作業,其中銅棒物件直徑為190μm,組裝物孔洞直徑為240μm,組裝間隙比為0.208。
In this thesis, augmented virtuality is applied to the micro-assembly system, where the operator can monitor and operate with a headset and controllers to achieve the application of micro-assembly teleoperation. To create the virtual environment, firstly draw the simulation micro-assembly machine through CAD software, and then import it into the Unity game engine. Real world’s webcams are installed with eye-in-hand configuration. The main technique assess the intrinsic matrix, extrinsic matrix, and distortion coefficients, combined with the ray casting algorithm, calculate the 3D pose of the copper rod’s endpoint and the hole. The estimated 3D pose are transmitted over the network to a remote client host in an established virtual environment, so that the virtual copper rod and virtual hole in the virtual environment have the pose information in real world coordinate, and the implementation of head-mounted display with augmented virtuality in micro-assembly teleoperation systems is achieved. This application enables the operator to have the immersive experience of operating the virtual micro-assembly machine in the remote environment, and monitor the 3D pose of the hole from any viewing angle and viewing scale, increasing the probability of successful assembly. In this thesis, two webcams were used to observe the micro objects on the server side. The operator monitored and operated the micro objects on the remote client through the headset and the controller, and the micro object assembly was successfully completed in the end. The operation is with copper bar of diameter 190μm, assembled to a hole of diameter 240μm, i.e., clearance ratio of 0.208.
摘要 i
致謝 vii
目錄 viii
表目錄 xi
圖目錄 xii
符號表 xvii
第一章 緒論 1
1-1 前言 1
1-2 研究動機 2
1-3 文獻回顧 3
1-3.1混合實境 3
1-3.2 虛擬實境、擴增實境和擴增虛擬於微組裝應用 5
1-3.3 攝影機配置位置 9
1-3.4物體三維姿態估測 11
1-4 研究目標與方法 14
1-5 本文架構 16
第二章 單眼攝影機成像模型與校正 17
2-1針孔成像模型 17
2-2 成像畸變 23
2-2.1 徑向畸變 23
2-2.2 切向畸變 25
2-3 校正方法 26
2-4 本章總結 30
第三章 攝影機估測三維座標 31
3-1 特徵點擷取 31
3-1.1 自動對焦 32
3-1.2 微物件特徵點擷取 35
3-1.3 棋盤格特徵點擷取 41
3-1.4 高精度次像素量測 42
3-2 攝影機外參數估測 45
3-2.1 外參數粗估 46
3-2.2 外參數細調 48
3-3 光線投影估測三維座標 51
3-3.1 單攝影機於已知平面估測三維座標 51
3-3.2 雙攝影機估測三維座標 56
3-4 本章總結 58
第四章 虛擬實境建立與網路傳輸 59
4-1 虛擬實境與虛擬三維物件之建立 59
4-2 頭戴式頭盔追蹤定位 63
4-3 渲染技術種類 67
4-4 渲染管線 68
4-5 虛擬成像模型座標轉換 73
4-5.1 模型矩陣(Model matrix) 74
4-5.2 視圖矩陣(View matrix) 77
4-6 虛擬微組裝系統 79
4-7 網路傳輸 82
4-7.1 通訊協定 82
4-7.2 插座介面(Socket API) 83
4-8 本章總結 86
第五章 系統之實現 87
5-1 系統介紹 87
5-1.1 真實微組裝系統硬體規格 87
5-1.2 虛擬微組裝系統硬體規格 95
5-1.3 Unity使用者介面 99
5-2 遠端微組裝實驗驗證 102
5-2.1 估測雙攝影機之內參數與畸變係數 104
5-2.2 棋盤格特徵點擷取 107
5-2.3 攝影機外參數估測 109
5-2.4 單攝影機基於已知平面估測軸孔三維座標 111
5-2.5 雙攝影機估測組裝件上端點之三維座標 114
5-2.6 TCP/IP網路傳輸 118
5-2.7 擴增虛擬遠端輔助微組裝 120
5-2.8 實驗結果討論 127
5-3 本章總結 127
第六章 結論與未來展望 128
6-1 結論 128
6-2 未來展望 130
參考資料 132
[1]R. C. Goertz, and R. Thompson, “Electronically Controlled Manipulator, Nucleonics, pp. 46-47, 1954.
[2]G. Burdea, and P. Coiffet, “Virtual Reality Technology, New York:Wiley-IEEE Press, 2003.
[3]P. Milgram, and F. Kishino, “A Taxonomy of Mixed Reality Visual Displays, IEICE Transactions on Information and Systems, Vol.77, No.12, pp. 1321-1329, 1994.
[4]P. Milgram , H. Takemura, A. Utsumi, and F. Kishino, “Augmented Reality: A Class of Displays on the Reality-virtuality Continuum, SPIE Telemanipulator and Telepresence Technologies, Vol.2351, pp. 282-292, 1994.
[5]D. Kragic, and H. I. Christensen, “Survey on Visual Servoing for Manipulation, Computational Vision and Active Perception Laboratory Tech. Rep., February 2002.
[6]A. Sulzmann, J. Breguet, and J. Jacot, “Microvision System (MVS): A 3D Computer Graphic-based Microrobot Telemanipulation and Position Feedback by Vision, Photonics East'95, International Society for Optics and Photonics, 1995.
[7]J. Alex, B. Vikramaditya, and B. Nelson, “A Virtual Reality Teleoperator Interface for Assembly of Hybrid MEMS Prototypes, In: Proceedings of DETC, pp. 13-16, 1998.
[8]J. Cecil, and J. Jones, “VREM: An Advanced Virtual Environment for Micro Assembly, The International Journal of Advanced Manufacturing Technology, pp.1-10, 2014.
[9]R. J. Chang, and J. C. Jau, “Augmented Reality in Peg-in-hole Microassembly Operations, Int. J. of Automation Technology, Vol.10, No.3, pp.438-446, 2016.
[10]G. Flandin, F. Chaumette, and E. Marchand, “Eye-in-hand / Eye-to-hand Cooperation for Visual Servoing, IEEE International Conference on Robotics & Automation, April 2000.
[11]A. Kaehler, and G. Bradski, “Learning OpenCV 3, O'Reilly Media Inc, 2016. ISBN: 1491937998, 9781491937990
[12]D. C. Brown, “Close-range Camera Calibration, Photogrammetric Engineering, Vol.37, No.8, pp.855-866, 1971.
[13]Z. Zhang, “A Flexible New Technique for Camera Calibration, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.22, No.11, pp.1330-1334, 2000.
[14]G. Xu, X. Li, J. Su, H. Pan, and G. Tian, “Precision Evaluation of Three-dimensional Feature Points Measurement by Binocular Vision, Journal of the Optical Society of Kore, Vol.15, No.1, pp.30-37, 2011.
[15]E. R. Pi, “Implementation of a 3D Pose Estimation Algorithm, Master’s Degree in Automatic Control and Robotics, 2015.
[16]Y. I. Abdel-Aziz, and H. M. Karara, “Direct Linear Transformation from Comparator Coordinates into Object Space Coordinates in Close-Range Photogrammetry, Photogrammetric Engineering & Remote Sensing, Vol.81, No.2, pp.103-107, 2015.
[17]J. He, R. Zhou, and Z. Hong, “Modified Fast Climbing Search Auto-focus Algorithm with Adaptive Step Size Searching Technique for Digital Camera, IEEE Transactions on Consumer Electronics, Vol. 49, No. 2, pp.257-262, 2003.
[18]Z. Liang, H. M. Gao, Z. J. Wang, and L. Wu, “Sub-pixels Corner Detection for Camera Calibration, Transactions of the China Welding Institution, Vol. 27, No. 2, 2006.
[19]V. Lepetit, and P. Fua, “Monocular Model-Based 3D Tracking of Rigid Objects: A Survey, Foundations and Trends in Computer Graphics and Vision, Vol. 1, No. 1, pp.1-89, 2005.
[20]R. Hartley, and A. Zisserman, “Multiple View Geometry in Computer Vision, Cambridge University Press, 2003.
[21]R. C. Gonzalez, and R. E. Woods, “Digital Image Processing(Third edition) , Pearson, 2012.
[22]C. Doignon, “An Introduction to Model-Based Pose Estimation and 3-D Tracking Techniques, Rijeka, Croatia: INTECH Open Access Publisher, 2007.
[23]J. Siswantoro, A. S. Prabuwono, and A. Abdullah, “Real World Coordinate from Image Coordinate Using Single Calibrated Camera Based on Analytic Geometry, Soft Computing Applications and Intelligent Systems, Vol. 378, pp. 1-11, 2013.
[24]李文政,“遠端監控微撓性機械夾持系統之研究,國立成功大學機械工程學系論文,中華民國九十一年。
[25]趙家成,“擴增實境於微物件組裝之實現,國立成功大學機械工程學系論文,中華民國一百零三年。
[26]劉俊甫,“三維模型基礎之擴增實境與微組裝之應用,國立成功大學機械工程學系論文,中華民國一百零六年。
[27]簡威任,“ 擴增實境於具手眼配置微組裝系統之實現,國立成功大學機械工程學系論文,中華民國一百零七年。
[28]余振瑋、葉子祺、張栩毓、張仁宗,“頭戴VR裝置於微組裝系統之開發,中國機械工程學會第三十五屆全國學術研討會,中華民國一百零七年。
[29]“Socket Programming in Python,
https://realpython.com/python-sockets/, 2020/05/24.
[30]“OpenGL Transformation,
http://www.songho.ca/opengl/gl_transform.html, 2020/05/07.
[31]“HTC Vive Teardown,
https://zh.ifixit.com/Teardown/HTC+Vive+Teardown/62213?lang=en#, 2020/05/24.
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top