臺灣博碩士論文加值系統

本論文研究三目相機目前階段在現實世界可以適用的範圍及需要的配置。我們採用ROS系統整合各個設備間的通信，利用SLAM技術完成地圖建置，再透過AMCL對整合三目相機的載具進行路線評估與整體計時，由導航過程產生的問題，來了解在使用三目相機導航後有哪些問題需要克服。

我們驗證在室外環境光達失敗時，三目相機仍能運作。但是三目相機系統的FPS運算速度與光達系統是1.08秒與0.03秒的不同。且由於準確度關係，完成某有障礙物路線的時間是120秒與30秒的不同。

經由實驗可以得知三目相機在水平型障礙物上避障有特殊的效果但在對於邊緣裝置的適配上仍有不少困難需要克服，如果將三目相機輕量化，相當於可以邊緣裝置應用上更多的偵測技術，會使移動載具的導航更準確也更有可靠性，或是配置更多相機使獲得的資訊量更多，使原本不清晰或品質較不佳的區域，可以透過資訊量增加，使品質提升。

This paper investigates the applicability and necessary configurations for using a trinocular camera system in real-world scenarios at the current stage. We utilize the ROS system to integrate communication between various devices, employing SLAM technology to build maps. Through AMCL, we assess the route and overall timing of the vehicle equipped with the trinocular camera. By examining the issues that arise during the navigation process, we aim to understand the challenges that need to be addressed when using a trinocular camera for navigation.

We verified that the trinocular camera can still operate in outdoor environments when LiDAR fails. However, the computation speed of the trinocular camera system's FPS differs significantly from that of the LiDAR system, with the former being 1.08 seconds and the latter 0.03 seconds. Due to accuracy concerns, the time required to complete a specific obstacle route also varies, being 120 seconds for the trinocular camera system and 30 seconds for the LiDAR system.

Through experiments, we found that the trinocular camera system has a unique advantage in obstacle avoidance for horizontal obstacles. However, there are still many challenges to overcome in adapting to edge devices. If the trinocular camera system is made more lightweight, it would enable the application of more detection technologies on edge devices. This improvement would enhance the accuracy and reliability of mobile vehicle navigation. Alternatively, deploying more cameras could increase the amount of acquired information, thereby improving the quality of previously unclear or lower-quality areas through increased data.

摘要 i
Abstract ii
目錄 iii
表目次 v
圖目次 vi
第一章 序論 1
第二章 基礎知識 3
2.1立體視覺匹配（Stereo Vision Matching） 3
2.2相機標定（Stereo Calibration） 3
2.2.1內部參數 4
2.2.2外部參數 4
2.2.3張正友標定法 5
2.3視差圖（Disparity Map） 5
2.4深度圖（Depth Map） 6
2.5影像校正 7
2.6 GPS衛星定位 8
2.7飛行時間測距 9
2.7.1 直接飛行測距法（DTOF） 9
2.7.2 間接飛行測距法（ITOF） 10
2.8 機器人作業系統（ROS） 11
2.9 載具定位 12
2.9.1 自適應蒙特卡羅定位(AMCL) 12
第三章 相關文獻探討 13
3.1 Pyramid Stereo Matching Network 13
3.2三目相機立體視覺網路 13
3.3 立體視覺資料集（Stereo Dataset） 14
3.3.1 SceneFlow 資料集 14
3.3.2 KITTI 資料集 15
3.3.3無人機的立體視覺資料集 15
3.3.4三目相機合成資料集 16
3.3.5三目相機現實資料集 16
3.4 階層式導航[10] 17
3.4.1廣域導航 18
3.4.2窄域導航 18
3.5 基於密度之空間聚類法（DBSCAN）[15] 19
第四章 三目相機系統 20
4.1 Tri-PSMN演算法與Tri-Scene-Virtual資料集 20
4.1.2水平方向匹配 22
4.1.3 Tri-PSMN與PSMN含水平線評估 24
4.2三目相機與PSMN用含水平線資料集測試 27
4.3 PSMN在三目相機與雙目相機性能評估 28
4.4三目相機使用範圍 28
第五章 三目相機系統在現實場景應用 29
5.1三目相機 29
5.1.1三目相機裝置 29
5.1.2三目相機模型運行環境 30
5.1.3無人載具配置 31
5.1.4影像校正 31
5.2實驗流程 32
5.3深度估計與SLAM地圖建立 33
5.4限制FOV顯示範圍來消除障礙物假象 34
5.5使用Clustering分類物體消除點雲連續性雜訊 34
5.6導航架構 35
5.7實驗場景 36
第六章 研究結果與比較 38
6.1光達與深度相機建圖比較 38
6.2應用場景限制 39
6.3三目相機處理速度 40
6.4路線比較(光達與三目相機) 40
6.5在邊緣裝置上運行三目相機 41
第七章 結論與未來展望 42
7.1結論 42
7.2未來展望 42
第八章 參考文獻 44

[1]J. -R. Chang and Y. -S. Chen, "Pyramid Stereo Matching Network," 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 5410-5418, doi: 10.1109/CVPR.2018.00567.
[2]蘭松耀., & Lan, S.-Y. (2023). 偵測水平線的三目相機立體視覺: 現實世界測試= Trinocular Vision To See Horizontal Wires: Real World Testing. 國立中興大學.
[3]徐明志., & Hsu, M.-C. (2023). 偵測水平線的三目相機立體視覺: 演算法= Trinocular Vision to See Horizontal Wires: Algorithm. 國立中興大學.
[4]林鼎鈞., & Lin, D.-J. (2023). 偵測水平線的三目相機立體視覺: 資料集= Trinocular Vision to See Horizontal Wires: Dataset. 國立中興大學.
[5]Z. Zhang, "A flexible new technique for camera calibration," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 11, pp. 1330-1334, Nov. 2000, doi: 10.1109/34.888718.
[6]邱., Chiou, Y., 杭., 蔡., Hang, H., & Tsai, J. (2011). 使用景深感應器與RGB相機對用於影像合成的景深改善. http://hdl.handle.net/11536/49161.
[7]Wu, W., Zhu, H. & Zhang, Q. Epipolar Rectification by Singular Value Decomposition of Essential Matrix. Multimed Tools Appl 77, 15747–15771 (2018). https://doi.org/10.1007/s11042-017-5149-0
[8]N. Mayer et al., "A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 4040-4048, doi: 10.1109/CVPR.2016.438
[9]https://www.ddcar.com.tw/article/26602
[10]張惟智. (2021). 階層式且考慮密度人流之無人載具導航 = HIERARCHICAL UNMANNED VEHICLE NAVIGATION CONSIDERING DENSITY AND PEDESTRIAN FLOW. 國立中興大學.
[11]Faranak Shamsafar, Andreas Zell, "TriStereoNet: A Trinocular Framework for Multi-baseline Disparity Estimation", arXiv:2111.12502v2 [cs.CV] 4 Sep 2022
[12]E. I. Parisi, A. Masiero, G. Tucci, I. Cortesi and F. Mugnai, "Thermal camera geometric self-calibration supported by RTK measurements," 2022 IEEE Workshop on Metrology for Agriculture and Forestry (MetroAgriFor), Perugia, Italy, 2022, pp. 249-254, doi: 10.1109/MetroAgriFor55389.2022.9964630.
[13]https://www.cnblogs.com/li-yao7758258/p/11191878.html
[14]X. Zhang, X. Cao, A. Yu, W. Yu, Z. Li and Y. Quan, "UAVStereo: A Multiple Resolution Dataset for Stereo Matching in UAV Scenarios," in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 16, pp. 2942-2953, 2023, doi: 10.1109/JSTARS.2023.3257489.
[15]https://jason-chen-1992.weebly.com/home/-dbscan
[16]Geiger A, Lenz P, Stiller C, Urtasun R. Vision meets robotics: The KITTI dataset. The International Journal of Robotics Research. 2013;32(11):1231-1237. doi:10.1177/0278364913491297
[17]https://www.dji.com/tw/phantom-4-pro-v2
[18]https://blog.csdn.net/weixin_39822993/article/details/111579641
[19]Q. Jia, X. Wan, B. Hei and S. Li, "DispNet Based Stereo Matching for Planetary Scene Depth Estimation Using Remote Sensing Images," 2018 10th IAPR Workshop on Pattern Recognition in Remote Sensing (PRRS), Beijing, China, 2018, pp. 1-5, doi: 10.1109/PRRS.2018.8486195.
[20]K. He, X. Zhang, S. Ren and J. Sun, "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 9, pp. 1904-1916, 1 Sept. 2015, doi: 10.1109/TPAMI.2015.2389824. keywords: {Training;Feature extraction;Accuracy;Convolutional
[21]Newell, Alejandro, Kaiyu Yang, and Jia Deng. "Stacked Hourglass Networks for Human Pose Estimation." arXiv preprint arXiv:1603.06937 (2016).
[22]潘裕升., & Pen, Y.-S. (2024). 建立三目深度相機導航環境與製作含水平線的現實場景資料集= Establishing a Trinocular Depth Camera Navigation Environment And Producing a Realistic Scene Data Set Containing Horizontal Lines. 國立中興大學.
[23]洪政煒., & Hong, Z.-W. (2024). 偵測水平線的輕量化三目相機立體視覺模型= Lightweight Trinocular Stereo Vision Model for Horizon Wires Detection. 國立中興大學.
[24]Y. Wang et al., "MAVIS: Multi-Camera Augmented Visual-Inertial SLAM using SE2(3) Based Exact IMU Pre-integration," 2024 IEEE International Conference on Robotics and Automation (ICRA), Yokohama, Japan, 2024, pp. 1694-1700, doi: 10.1109/ICRA57147.2024.10609982.