臺灣博碩士論文加值系統

三維重建已成為機器人學和計算機視覺領域中最活躍的研究主題之 一。重建物體的幾何形狀、物體的運動以及觀察到的紋理和外觀的能力已 經被使用在計算機視覺和圖形學的眾多應用上。儘管現在重建已經發展出 各式各樣不同的方法，然而，動態重建仍然是一個巨大挑戰，周遭的環境 對於重建是一個很大的影響因素。如何有效地重建移動物體的運動仍然是 一個巨大的挑戰。為了能有效降低 Kinect 在物體運動速度過快無法反應的 問題，同時也可以提高傳統結構光方法可能會出現圖形辨識的問題，我們 提出了一種結合神經網絡的單張結構光方法，可以有效省下傳統結構光方 法中所需要大量時間計算幾何特徵，並且在不同條件下可能無法很好地重 建，而我們的方法在物體變得更複雜時仍然能夠表現出色。在動態場景中， 儘管我們的方法可能會在路徑重建方面產生更多的噪音，但我們的方法在 單擺中獲得的半徑可能比 Kinect 更穩定。

Three-dimensional reconstruction has become one of the most active research topics in the fields of robotics and computer vision. The ability to reconstruct the geometry of objects, the motion of objects, and observed texture and appearance have unlocked numerous applications in computer vision and graphics. Recent advances in research of 3D reconstruction uses depth maps fusion methods and multiple RGB-D cameras. However, dynamic reconstruction is still a big challenge due to constraints such as the captured environment. How to efficiently reconstruct the motion of a moving object is still largely unsolved. Reducing the problem of motion blur or the short reaction time that Kinect might have also improves the performance of pattern identification that the conventional structured light method might have. We proposed a one-shot structured light method combining the neural network. Compared to the conventional structured light method, which takes up a lot of time to calculate the geometric feature and might not reconstruct well under different conditions, our method can perform well when the object becomes more complicated. In the dynamic scene, in spite of the fact that our method might reconstruct the route with more noise, the radius our method can get in the simple pendulum could be more stable than Kinect.

第一章 Introduction.............................................................................................. 1
第二章 Related Works.......................................................................................... 1
第三章 Time of Flight Method............................................................................. 3
3.1 Kinect ................................................................................................... 5
第四章 Structured Light Method.......................................................................... 6
4.1 Pattern Generation................................................................................. 6
4.1.1 De Bruijn Sequence ................................................................ 6
4.1.2 Pattern Design ......................................................................... 9
4.2 Image Segmentation ........................................................................... 10
4.2.1 Histogram Equalization ........................................................ 10
4.2.2 Thresholding Method ............................................................ 12
4.3 Decode Method .................................................................................. 13
4.3.1 Convolutional Neural Network ............................................. 14
4.3.2 Data Augmentation ............................................................... 17
4.3.3 Direction ............................................................................... 17
4.3.4 Nearest Neighbor .................................................................. 19
4.3.5 The Corresponding Pair ........................................................ 20
4.4 Camera Projector Calibration ............................................................. 21
4.4.1 Camera and Projector Model ................................................. 21
4.4.2 Distortion Model ................................................................... 25
4.4.3 Camera Calibration ............................................................... 26
4.4.4 Projector Calibration ............................................................. 29
4.4.5 Stereo System Calibration .................................................... 30
4.5 Three Dimensional Reconstruction..................................................... 31
4.6 Moving Track Reconstruction ............................................................ 33
4.6.1 Sphere Fitting ........................................................................ 34
第五章 Experiment and Result........................................................................... 35
5.1 Experiment Environment ................................................................... 35
5.2 Training and Testing .......................................................................... 35
5.3 Result ................................................................................................. 38
5.3.1 Static ..................................................................................... 38
5.3.2 Dynamic ................................................................................ 39
第六章 Discussion and Conclusion..................................................................... 44
參考文獻 ............................................................................................................ 47

[1] Zhenzhou Wang. Review of real-time three-dimensional shape measurement techniques. Mea- surement, 156:107624, 2020.
[2] Anupama K Ingale et al. Real-time 3d reconstruction techniques applied in dynamic scenes: A systematic literature review. Computer Science Review, 39:100338, 2021.
[3] Soonmin Hwang, Namil Kim, Yukyung Choi, Seokju Lee, and In So Kweon. Fast multiple objects detection and tracking fusing color camera and 3d lidar for intelligent vehicles. In 2016 13th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), pages 234–239. IEEE, 2016.
[4] Ryusuke Sagawa, Tatsuya Kawamura, Ryo Furukawa, Hiroshi Kawasaki, and Yoshio Mat- sumoto. One-shot 3d reconstruction of moving objects by projecting wave grid pattern with diffractive optical element. In Proc. 11th IMEKO Symposium Laser Metrology for Precision Measurement and Inspection in Industry, 2014.
[5] Xin Tian, Rui Liu, Zhongyuan Wang, and Jiayi Ma. High quality 3d reconstruction based on fusion of polarization imaging and binocular stereo vision. Information Fusion, 77:19–28, 2022.
[6] Ahsan Elahi, Jun Lu, Qi-Dan Zhu, and Li Yong. A single-shot, pixel encoded 3d measurement technique for structure light. IEEE Access, 8:127254–127271, 2020.
[7] JaeWook Ha, Soon Kwon, Jaekyo Jeong, and HyunWoo Kim. 3d reconstruction method based on binary coded pattern. International Journal of Signal Processing, Image Processing and Pattern Recognition, 6(6):59–68, 2013.
[8] Eita Shoji, Atsuki Komiya, Junnosuke Okajima, Masaki Kubo, and Takao Tsukada. Three-step phase-shifting imaging ellipsometry to measure nanofilm thickness profiles. Optics and Lasers in Engineering, 112:145–150, 2019.
[9] Haibo Lin, Lei Nie, and Zhan Song. A single-shot structured light means by encoding both color and geometrical features. Pattern Recognition, 54:178–189, 2016.
[10] Zhenzhou Wang, Qi Zhou, and YongCan Shuang. Three-dimensional reconstruction with single- shot structured light dot pattern and analytic solutions. Measurement, 151:107114, 2020.
[11] David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using a multi-scale deep network. Advances in neural information processing systems, 27, 2014.
[12] Hieu Nguyen, Yuzeng Wang, and Zhaoyang Wang. Single-shot 3d shape reconstruction using structured light and deep convolutional neural networks. Sensors, 20(13):3718, 2020.
[13] Fayao Liu, Chunhua Shen, and Guosheng Lin. Deep convolutional neural fields for depth estimation from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5162–5170, 2015.
[14] Yu He and Shengyong Chen. Recent advances in 3d data acquisition and processing by time- of-flight camera. IEEE Access, 7:12495–12510, 2019.
[15] Jungong Han, Ling Shao, Dong Xu, and Jamie Shotton. Enhanced computer vision with microsoft kinect sensor: A review. IEEE transactions on cybernetics, 43(5):1318–1334, 2013.
[16] Zhengyou Zhang. Microsoft kinect sensor and its effect. IEEE multimedia, 19(2):4–10, 2012.
[17] Yi-Chih Hsieh. Decoding structured light patterns for three-dimensional imaging systems.
Pattern Recognition, 34(2):343–349, 2001.
[18] Yang Lei, Kurt R Bengtson, Lisa Li, and Jan P Allebach. Design and decoding of an m-array pattern for low-cost structured light 3d reconstruction systems. In 2013 IEEE international conference on image processing, pages 2168–2172. IEEE, 2013.
[19] Suming Tang, Xu Zhang, Zhan Song, Lifang Song, and Hai Zeng. Robust pattern decoding in shape-coded structured light. Optics and Lasers in Engineering, 96:50–62, 2017.
[20] Jing Xu, Ning Xi, Chi Zhang, Quan Shi, and John Gregory. Real-time 3d shape inspection system of automotive parts based on structured light pattern. Optics & Laser Technology, 43(1):1–8, 2011.
[21] M ́ario LL Reiss and Antonio MG Tommaselli. A low-cost 3d reconstruction system using a single-shot projection of a pattern matrix. The Photogrammetric Record, 26(133):91–110, 2011.
[22] Daniel Moreno and Gabriel Taubin. Simple, accurate, and robust projector-camera calibration. In 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission, pages 464–471. IEEE, 2012.