臺灣博碩士論文加值系統

科技日新月異，三維重建的技術在過去的十幾年內快速成長，而基於影像的三 維重建方法一直以來是研究的焦點。由於感測器在單一位置的視角有限，取得的環境資 訊並不完全，因此主流方法是將相機安裝於移動裝置如車輛上，隨著相機移動對場景連 續採集序列影像，再透過影像特徵點的擷取、匹配與追蹤，計算相機本體移動軌跡，將 不同位置計算之場景結構融合成單一模型。然而視覺移動估計是有挑戰性的，隨著系統 於大範圍區域中移動，細微的估計失準易以非線性趨勢累積而產生巨大誤差，且估計過 程須仰賴影像特徵產生時序對應關係，而相機在高速移動時產生的模糊影像、無特徵點 與結構重複的影像以及光線不充足等問題，都會大幅影響軌跡分析之準確度。本研究結 合基於特徵點的匹配方法與直接採用光度資訊的分析法，實現更精確、更穩定的移動路 徑估測，增強重建模型的準確性。 基於特徵點的本體移動估計，是以特徵空間中的匹配建立影像對應性，再以一系列 過濾演算法限制並保留匹配良好的特徵，並透過雙眼視覺測量匹配特徵點之空間座標以 及相機參數，解出 N 點透視 (perspective-n-point, PnP) 問題，取得相機於兩幅影像間之 姿態變化，最後輔以非線性最佳化調整相機的的旋轉、位移等移動參數。本論文探討一 種混合三種擬合模型之方法來改善特徵點的匹配與本體移動估計的最佳化過程，除了使 用傳統之空間－影像 (3D-to-2D) 的對應關係外，又加入了極幾何限制 (epipolar constraint) 與光度誤差 (photo-consistency) 模型。此新式的混合模型以 KITTI Vision Benchmark Suite 實驗評估後，證實能有效降低累積的估計誤差並獲得穩健的結果，與現 行方法相較下，在準確性的提升超過 50%以上。
關

Three-dimensional (3D) reconstruction technology has been rapidly grown in the last decade. Among a variety of developed scanning techniques, the image-based approaches have been extensively studied and applied. Due to limited field of view, the optical sensor is usually mounted on a moving vehicle to record an image sequence, from which large-scale 3D reconstructions are computed with extended scanning range. To align the 3D data computed in different positions, the trajectory of the moving camera must be estimated. The feature-based visual odometry (VO) approaches recover such a trajectory using image features extracted, matched, triangulated and tracked through the sequence. The estimation of the camera’s pose between two frames is achieved by solving a perspective-n-point (PnP) problem which minuses 3D-to-2D geodesic error. The accuracy of pose estimation using the geodesic model can be jeopardised when it is not able to find sufficient high-quality feature points from the imagery data, due to blurring, over-/under-exposure, presence of repetitive textures or other issues. This thesis studies an enhanced stereo VO algorithm that takes into account photo-consistency to complement the 3D-to-2D alignment model. The multi-objective strategy also considers epipolar constraint to further reduce false matchings. Evaluations using the KITTI Vision Benchmark Suit have demonstrated the effectiveness of the proposed method. An improvement more than 50% over mono-objective stereo VO model is attainable.

致謝 i
中文摘要 ii
Abstract iii
符號定義 vi
圖目錄 viii
表目錄 x
第一章 緒論 1
1.1 前言 1
1.2 研究動機 2
1.3 研究目的 2
1.4 研究方法 3
1.5 論文架構 4
第二章 文獻探討 5
2.1 前言 5
2.2 SfM、V-SLAM 與 VO 6
2.3 視覺測程的近年發展 7
第三章 視覺測程 9
3.1 相機模型 10
3.2 雙眼視覺 13
3.3 特徵點偵測 16
3.3.1 積分影像 (Integral Image) 17
3.3.2 快速海森特徵檢測 18
3.3.3 SURF 敘述子 21
3.4 特徵點匹配 23
3.5 本體移動估計 26
3.6 最佳化 28
3.7 狀態更新 29
第四章 多重模型本體移動估測 30
4.1 投影誤差模型 30
4.2 極幾何模型 30
4.3 光度誤差模型 32
4.4 結合多目標之最佳化模型 34
第五章 實驗結果與分析 35
5.1 實驗序列一 36
5.2 實驗序列二 41
5.3 實驗序列三 46
5.4 小結 51
第六章 結論與未來方向 52
參考文獻 53

[1]H. Moravec, “Obstacle avoidance and navigation in the real world by a seeing robot rover,” Ph.D. dissertation, Stanford Univ., Stanford, CA, 1980.
[2]D. G. Lowe, “Object recognition from local scale-invariant features,” Proc. Int. Conf. on Computer Vision, vol. 2, pp. 1150–1157, 1999.
[3]H. Bay, T. Tuytelaars and L. Van Gool, “SURF: Speeded up robust features,” Proc. European Conf. Computer Vision, 2006.
[4]R. A. Newcombe, S. J. Lovegrove and A. J. Davison, “DTAM: dense tracking and mapping in real-time,” Proc. IEEE Int. Conf. on Computer Vision, pp. 2320-2327, 2011.
[5]J. Engel, J. Stueckler and D. Cremers, “Large-scale direct SLAM with stereo cameras,” Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp1935-1942, 2015.
[6]C. Forster, M. Pizzoli, and D. Scaramuzza, “SVO: Fast semi-direct monocular visual odometry”, Proc. IEEE Int. Conf. on Robotics and Automation, pp1050-4729, 2014.
[7]D. Nister, O. Naroditsky and J. Bergen, “Visual Odometry,” Proc. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, vol. 1, pp. 652-659, 2004.
[8]M. Irani, B. Rousso and S. Peleg, “Recovery of ego-motion using image stabilization,” Proc. IEEE Conf. on Computer Vision and Pattern Recognition, pp. 21–23, 1994.
[9]M. Maimone, Y. Cheng, L. Matthies, “Two years of Visual Odometry on the Mars Exploration Rovers,” J. of Field Robotics, vol. 24, no. 3, pp. 169–186.
[10]H. Durrant-Whyte and T. Bailey, “Simultaneous Localization and Mapping (SLAM): Part I. The essential algorithms,” Proc. IEEE Int. Conf. Robotics and Automation, vol. 13, no. 2, pp. 99-110, 2006.
[11]H. Durrant-Whyte and T. Bailey, “Simultaneous Localisation and Mapping (SLAM): Part II. State of the art,” Proc. IEEE Int. Conf. on Robotics and Automation, vol. 13, no. 3, pp. 108-117, 2006.
[12]B. Williams and I. Reid, “On combining visual SLAM and visual odometry,” Proc. IEEE Int. Conf. on Robotics and Automation, pp. 3494-3500, 2010.
[13]D. Scaramuzza and F. Fraundorfer, “Visual odometry: Part I - The first 30 years and fundamentals,” IEEE Robotics Automation Magazine, vol. 18, pp. 80–92, 2011.
[14]P. J. Besl and N. D. McKay, “A method for registration of 3-d shapes,” IEEE Transaction on Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp.239-256, 1992.
[15]W. L. D. Lui, T. J. J. Tang, T. Drummond, and W. H. Li, “Robust egomotion estimation using ICP in inverse depth coordinates,” Proc. IEEE Int. Conf. on Robotics and Automation, pp. 1671-1678, 2012.
[16]D. Nister, O. Naroditsky, and J. Bergen, “Visual odometry,” Proc. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, vol.1, pp. 652-659, 2004.
[17]A. Comport, E. Malis and P. Rives, “Accurate quadrifocal tracking for robust 3d visual odometry,” Proc. IEEE Int. Conf. Robotics and Automation 2007, pp. 40-45, 2007.
[18]L. Wei, C. Cappelle, Y. Ruichek, and F. Zann, "GPS and stereovision-based visual odometry: application to urban scene mapping and intelligent vehicle localization," Int. J. of Vehicular Technology, vol. 2011, 2011.
[19]D. Scaramuzza and F. Fraundorfer, "Visual odometry: Part II Matching, robustness, optimization, and applications," IEEE Robotics and Automation Magazine, vol. 19, no. 2, pp. 78-90, 2012.
[20]H. Badino, A. Yamamoto and T. Kanade, "Visual odometry by multi-frame feature integration," Proc. Workshop on Computer Vision for Autonomous Driving (Collocated with ICCV”13), Sydney, Australia, 2013.
[21]A. Geiger, ‘The KITTI Vision Benchmark Suite’, 2016, [Online]. Available: http://www.cvlibs.net/datasets/kitti/. [Accessed: 13-Oct-2017].
[22]D. Nister, “An efficient solution to the five-point relative pose problem,” Proc. Int. Conf. Computer Vision and Pattern Recognition, pp. 195–202, 2003.
[23]D. Scharstein and R. Szeliski, "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms," Int. J. of Computer Vision, vol. 47, no. 1, pp. 7-42, 2002.
[24]H. Hirschmuller, "Accurate and efficient stereo processing by semi-global matching and mutual information," Proc. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, vol. 2, pp. 807-814, 2005.
[25]M. J. Milford and G. Wyeth, “Single camera vision-only SLAM on a suburban road network,” Proc. IEEE Int. Conf. Robotics and Automation, pp. 3684–3689, 2008.
[26]D. Scaramuzza and R. Siegwart, “Appearance-guided monocular omnidirectional visual odometry for outdoor ground vehicles,” IEEE Trans. Robot. (Special Issue on Visual SLAM), vol. 24, no. 5, pp. 1015– 1026, Oct. 2008.
[27]I. Scollar, "Radial Distortion Correction," 2005, http://www.uni-koeln.de/~al001/radcor_files/hs100 [Accessed 13-Oct-2017].
[28]R. Klette, Concise Computer Vision: An Introduction into Theory and Algorithms. Springer-Verlag London, 2014.
[29]S. D. Cochran and G. Medioni, “3D surface description from binocular stereo,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 10, pp. 981-994, 1992.
[30]L. Juan and O. Gwon, “A comparison of SIFT, PCA-SIFT and SURF,” Int. J. of Image Processing, vol. 3, no. 4, pp. 143-152, 2009.
[31]H.-J. Chien, C.-Y. Chen, C.-C. Chuang, “When to use what feature? SIFT, SURF, ORB, or A-KAZE features for monocular visual odometry,” Proc. Conf. on Image and Vision Computing New Zealand, pp.2151-2205, 2016.
[32]P. Viola, and M.Jones, “Rapid object detection using a boosted cascade of simple features,” Proc. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, vol. 1, pp. 511-518, 2001.
[33]Tornadomeet,"SURF,"http://www.cnblogs.com/tornadomeet/archive/2012/08/17/2644903.html. [Accessed 13-Oct-2017].
[34]T. H. Huang, Establishing Ego-motion Estimation from Monocular UAV Image Sequences, M. Eng. thesis, Dept. of Computer Science and Information Engineering, National University of Kaohsiung, 2016.
[35]V. Lepetit, F. Moreno-Noguer and P. Fua, "EPnP: An accurate O(n) solution to the PnP problem," Int. J. Computer Vision, vol. 81, no. 2, pp. 155-166, 2009.
[36]M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Comm. ACM, vol.24, no. 6, pp. 381-395, 1981.
[37]Q. Zhang and R. Pless, “Extrinsic calibration of a camera and laser range finder (improves camera calibration),” Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems, vol. 3, pp. 2301-2306, 2004.
[38]C. Tomasi and T. Kanade, “Detection and tracking of point features,” Carnegie Mellon University Technical Report, CMU-CS-91-132, 1991.
[39]B. K. P. Horn, “Closed-form solution of absolute orientation using unit quaternions,” J. Optical Society of America A, vol. 4, no. 4, pp. 629-642, 1987.
[40]J. Stuehmer, S. Gumhold, and D. Cremers. “Real-time dense geometry from a handheld camera”. Proc. DAGM Symposium on Pattern Recognition, 2010.
[41]J.-H. Zhang, Adaptive Feature Tracking Based on Epipolar Geometry and Disparity for Ego-Motion Estimation, M. Eng. thesis, Dept. Computer Science and Information Engineering, National University of Kaohsiung, 2014.
[42]P. D. Sampson, “Fitting conic sections to ‘very scattered’ data: An iterative refinement of the Bookstein algorithm,” Computer Graphics Image Processing, vol. 18, pp. 97–108, 1982.
[43]R. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision, 2003.
[44]K. A. Levenberg, “Method for the solution of certain non-linear problems in least squares,” The Quarterly Applied Math., vol. 2, pp. 164–168, 1944.
[45]C. Engels, H. Stewenius, and D. Nister “Bundle adjustment rules,” J. Photogrammetric Computer Vision, vol 2, no. 32, 2006.