臺灣博碩士論文加值系統

　　在這篇論文當中，我們建立一個能從多張數位相片自動擷取實景的三維座標資訊和重建其立體模型的系統。關於被建的場景中相關的位置資訊不用事先取得，所有的三維座標資訊包括相機所在的位置和面對的方位都可以從本系統中自動算出。因此，對於本系統所需要用到的相片，可以用手持相機拍攝，不需要事先測量相機的相對位置。唯一的限制就是必須先取得相機的內部參數。
　　本三維立體模型重建系統包含四個主要步驟。首先我們需要用相機拍攝校正板，利用這些校正版的影像取得相機的內部參數。第二，相機的位置和兩張照片間的空間關係可藉由兩張影像間的對應點推算出來。接下來，連續的相片被分成兩兩一組，然後分別針對這幾對相片增加其間的共同對應點。最後，把這些對應點連接起來，並推算出該點的三維座標，然後，利用繪圖程式把這些三維的點顯示出來，並貼上對應的紋理圖片以增加它的真實感。我們已經使用這系統重建出數個實景三維立體模型，在這篇論文中，將會展示其中一些重建的結果。

　In this paper, we present a system which automatically extracts the 3D information and reconstructs a textured 3D model from a sequence of images of a real scene. No prior knowledge about the scene is needed to build the 3D models. All information such as camera pose and orientation will be estimated through the processes. Therefore, this system offers a high degree of flexibility when taking photographs. The only constraint is that the intrinsic camera parameters need to be obtained first.

　The 3D modeling task is decomposed into 4 successive steps. The camera intrinsic parameters are calibrated using a calibration board first. Second, the camera pose and the epipolar geometry between a stereoscopic image pair are estimated by the corresponding points of this pair. Next, consecutive images of the sequence are treated as stereo pair and the disparity maps are computed by area matching. Finally, the dense 3D points are estimated by the linking matches through consecutive image pairs. Then, these 3D points are visualized as a 3D model which is also texture mapped for photo-realistic appearance. This system has been tested on several real scenes, and some of the reconstructed models are shown in this paper.

1. Introduction.......................................1

2. Backgrounds........................................5
2.1 Camera Model for Center Projection...............5
2.2 Epipolar Geometry................................7
2.3 Triangulation Method.............................9

3. Camera Calibration and Fundamental Matrix.........12
3.1 Camera Calibration..............................12
3.1.1 Basic Equations..............................13
3.1.2 Extracting Intrinsic Parameters..............15
3.1.2.1 Linear Solution...........................15
3.1.2.2 Nonlinear Minimization....................16
3.1.2.3 Dealing with Radial Lens Distortion.......17
3.2 Fundamental Matrix and Camera Extrinsic Matrix..18
3.2.1 Finding Correspondences between Images.......18
3.2.2 Estimation of Fundamental Matrix.............18
3.2.2.1 Linear Solution – Normalized DLT Method..19
3.2.2.2 Nonlinear Minimization –
Least-Median-Squares Method...............20
3.2.3 Extracting Rotation and Translation Matrix...25

4. Dense Correspondences.............................27
4.1 Image Rectification.............................28
4.1.1 Generate the Transformation Matrix of
the two Images...............................28
4.1.1.1 Mapping the Epipole to Infinity...........28
4.1.1.2 Matching Transformations..................28
4.1.1.3 Refining the Transformation Matrix........30
4.1.2 Re-mapping the images........................33
4.2 Building Disparity Map..........................35
4.2.1 Constraints..................................36
4.2.2 Dynamic Programming Scheme...................37

5. Correspondence Linking, 3D Points Estimation
and Modeling......................................43
5.1 Correspondence Linking..........................44
5.1.1 Linking Scheme...............................44
5.1.2 Occlusions...................................45
5.2 Estimation of 3D Points.........................46
5.2.1 Normalize the Translation Matrix.............46
5.2.3 3D Points Fusion.............................47
5.3 Modeling........................................48

6. Experiment Results................................49
6.1 The South Gate..................................50
6.2 Tainan Confucius Temple.........................53

7. Conclusion and Future Works.......................55

References...........................................56

[1] R. Hartley, and A Zisserman: Multiple View Geometry in Computer Vision

[2] Z. Zhang: A Flexible New Technique for Camera Calibration. Microsoft Research. Aug. 2002.

[3] Z. Zhang, R. Deriche, O. Faugeras, and Q. Luong: A Robust Technique for Matching Two Uncalibrated Images through the Recovery of the Unknown Epipolar Geometry. INRIA. 1994

[4] R. Koch, M. Pollefeys, and L. Van Gool: Realistic Surface Reconstruction of 3D Scenes from Uncalibrated Image Sequences.

[5] R. Koch, M. Pollefeys, and L. Van Gool: Multi Viewpoint Stereo from Uncalibrated Video Seuences. Proc. ECCV’98, Freiburg, June 1998.

[6] M. Pollefey, R. Koch, M. Vergauwen, L. Van Gool: Automated Reconstruction of 3D Scenes from Sequences of Images.

[7] David G. Lowe: Distinctive Image Features from Scale-Invariant Keypoints. Computer Science Department, University of British Columbia. Jan, 2004.

[8] Paul E. Debevec, Camillo J. Taylor, and J. Malik: Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image- based approach. SIGGRAPH 96’, 1996

[9] R. Klette, K. Schlűns, and A. Koschan: Computer Vision: Three-Dimensional Data from Images.

[10] Minpack in Java. http://www1.fpl.fs.fed.us/optimization.html

[11] C. Lawrence Zitnick, and T. Kanade: A Cooperative Algorithm for Stereo Matching and Occlusion Detection. The Robotics Institute, Carnegie Mellon University. Oct., 1999.


[12] H. Mayer: Analysis of Means to Improve Cooperative Disparity Estimation. ISPRS Archives, Vol. XXXIV, Part 3/W8, Munich, 17-19. Sept. 2003.

[13] Jama: A Java Matrix Package. http://math.nist.gov/javanumerics/jama/

[14] Paul J. Besl: A Method for Registration of 3-D Shapes. IEEE Transaction on PAMI, Vol. 14, NO.2, Feb. 1992

[15] Tzu-Yang Li, Chia-Ming Wang, Yi-Ping Hung, Kuo-Chin Fan: A Visual Monitoring System Using Wide-Angle Cameras and Pan-Tilt Cameras.

[16] X. Qin and E.N. Sanei: Automatically Compositing Still Images and Landscape Video Sequences.

[17] M. Lhuillier and L. Quan: Image-Based Rendering by Joint View Triangulation. IEEE Transaction on Circuits and Systems for Video Technology, Vol.13, No.11, Nov. 2003.

[18] Shijun Sun: Semiautomatic Video Object Segmentation Using VSnakes. IEEE Transaction on Circuits and Systems for Video Technology, Vol.13, No.1, Jan. 2003.