臺灣博碩士論文加值系統

從多張影像中建構三維臉部模型是電腦圖學中的一個具有挑戰性的問題，其在電腦動畫和人機互動方面在有非常豐富的應用。在論文中，我們提出了一個新的方法從影像串列來重構照片式的彷真的三維臉部模型，我們可以由照相機或攝影機取得影像串列。對比之前的研究，我們的方法具有低資源、半自動(減少人工介入)和一般性(適用不同形狀的人頭)等優點。還有很多吸引人的特點如下所描述:
我們的方法能夠從一個給定的通用人頭模型重建三維臉部模型和三維頭髮模型。在重建三維臉部模型方面可分為兩個主要的步驟:首先在影像中選定一些特徵點，然後我們藉由一個通用的人頭模型和特徵點來決定人臉的位移和旋轉角度， 如此就可以得到特徵點在空間中的座標， 之後我們可以使用Bundle-Adjustment 的方法來最佳化特徵點的三維結構，並使用radial base function對通用人頭模型做變形。第二，我們根據影像資訊來調整變形後的人頭模型在影像上的投影點，然後我們就有這些投影點的新的三維位置，使用圓柱投影將這些和模型做對應，在一次使用radial base function對模型做變形。我們假設頭髮的輪廓的是落在通過模型中心點和影像平行的平面上，如此根據頭髮的輪廓點我們可以使用radial base function對人頭做變形，最後我們可以得到具有頭髮形狀的人頭模。我們發展了兩個方法對我們的人頭做紋理貼圖:view-independent 方法和view-depentent 方法，我們使用了透視投影的概念和觀察者和模型的相對關係來實作這兩種方法。在本論文中，我們實驗了所提出的方法，並且認為這個方法在三維臉部重構方面是相當具有未來性的。此外，在論文的第一章我們也對和先前的研究作比較，並提出我們的貢獻。

Three-dimensional facial modeling from multiple images is a challenging problem in computer graphics. It has very rich applications such as computer animation and human computer interaction. In this paper, we propose a novel approach/framework to reconstruct photo-realistic 3D facial model from frame sequence. The frame sequence can be from either camera images or video stream. In contrast to previous work, our approach is low-cost, semi-automatic (with less human intervention) and general (suitable for different type of heads). There are many interesting features proposed and are described as follows:
Our methods can reconstruct both 3D face models and 3D hair models from a given generic head model. Our approach has two main steps to reconstruct a face model. First, some feature points are selected in some frames. Then, we can compute the face pose with the help of a generic model and the selected feature points. We can
compute the 3D positions of the feature points. Then we use a bundle-adjustment algorithm to globally optimize the 3D structure of the feature points. After, we can use a radial base function to deform a generic head model by the feature points. Second, we project the points of the adapted head model on frames. Then we can adjust the position of the projected points according to frame information. We have new 3D position of the projected points. We use cylindrical projection to correspond the projected points and the head model. Then, We use a radial base function to deform the head model again. We assume the contour of the hair is located on the plane passing through the center of model and paralleled the image frame. We can use a radial base function to deform the head model by contour points. We can get a head model with hair shape. We explore two methods for texturing our 3D head model: view-independent method and view-dependent method. We implement the two methods with perspective projection and the relation between eye and model. In this research, we thoroughly experiment the proposed methods and we conclude the proposed method is very promising in facial model reconstructions. In addition, we also compare the different previous work and outline our contributions in the Chapter 1.

第１章 導論 2
　　第１節 研究動機與背景 2
　　第２節 本論文的系統流程 3
　　第３節 本論文的貢獻 4
　　第４節 實用化報告之評估 6
第２章 先前工作 7
　　第１節 Calibration 7
　　第２節 三維人頭之相關研究 11
　　第３節 討論 31
第３章 臉部特徵之擷取及相機位置之計算 32
　　第１節 選取特徵點 32
　　第２節 特徵點追蹤 35
　　第３節 取得相機內部參數 39
　　第４節 取得相機外部參數 44
第４章 三維人頭之重構 48
　　第１節 計算特徵點三維位置 48
　　第２節 Bundle Adjustment 51
　　第３節 增加特徵點 59
　　第４節 調整通用的三維人頭模型 62
第５章 三維頭髮模型之重構 66
　　第１節 Snake演算法 66
　　第２節 三維頭髮模型之重構 68
第６章 紋理貼圖 74
　　第１節 View-independent貼圖 76
　　第２節 View-dependent貼圖 85
第７章 實驗結果 89
第８章 結論與未來展望 96
　　第１節 結論 96
　　第２節 未來展望 97
參考文獻　98
附錄一:三維人頭重構系統與應用　104

[1]Y. C. Lee, D. Terzopoulos, and K.Waters. “Realistic modeling for
facial animation.” In Computer Graphics,Annual Conference Series, pages 55–62. SIGGRAPH, 1995.
[2]P. Marc, V.G. Luc, “Reading between the lines-a method for extracting dynamic 3D with texture.” Symposium on Virtual Reality Softwareand Technology’97, 95-102, 1997.
[3]Ming-Shing Su, Chun-Yen Chen and K. Y. Cheng, “The Reconstruction of 3D Head Model from Two Orthogonal-View 2D Face Images.” National Computer Symposium 2001, Taiwan, D320-D329, 2001.
[4]S. H. Lai and C.-M. Cheng, “Three-dimensional Face Model Creation from Video”, Proc. of SPIE Conf. on Three-dimensional Image Capture and Applications IV, Vol. 4298, San Jose, Jan. 2001.
[5]T. Akimoto, Y. Suenaga, and R. S. Wallace. “Automatic 3d facial models.” IEEE Computer Graphics and Applications.” 13(5):16–22, September 1993.
[6]H. H.S.Ip and L.Yin. “Constructing a 3d individualized head model from two orthogonal views.” The Visual Computer, (12):254–266, 1996.
[7]F. Pighin, J. Hecker, D. Lischinski, R. Szeliski, and D. H. Salesin. “Synthesizing realistic facial expressions from photographs.” In Computer Graphics, Annual Conference Series, pages 75–84. Siggraph, July 1998.
[8]P. Fua, “Face models from uncalibrated video sequences,” CAPTECH'98 Lecture Notes in Artificial Intelligence (LNAI) 1537, 214-228.
[9]P. Fua, “Regularized Bundle-Adjustment to Model Heads from Image Sequences without Calibration Data” International Journal of Computer Vision, , 38(2). 153--171, 2000.
[10]Zicheng Liu, Zhengyou Zhang, Chuck Jacobs, Michael Cohen, “Rapid Modeling of Animated Faces From Video.” Journal of Visualization and Computer Animation, Vol 12, No.4 (Sep. 2001), Page 227-240. Also in MSR-TR-2000-11
[11]Ying Shan, Zicheng Liu, Zhengyou Zhang, “Model-Based Bundle Adjustment with Application to Face Modeling.” ICCV 2001,Vol II, Page644-651.
[12]Z. Zhang. “Flexible Camera Calibration By Viewing a Plane From Unknown Orientations.” International Conference on Computer Vision (ICCV'99), Corfu, Greece, pages 666-673, September 1999.
[13]Z. Zhang. “ A Flexible New Technique for Camera Calibration.” IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330-1334, 2000.
[14]K.S Lee, K.H Wong, S.H Or, Y.F Fung. “3D Face Modelling From Perspective-Views and Contour-Based Generic-Model.” Accepted for publication in Real-Time Imaging, 2000.
[15]Abdel-Aziz, Y.I., Karara, H.M:Direct linear transform into object space coordinates in close-range photogrammetry. Proc. ASP Symposium on Close Range Photogrammetry, Urbana, Illinois, USA, 1971, pp. 1-1
[16]J. Shi and C. Tomasi. Good features to track. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR94), Seattle, June 1994
[17]M. Kass, A. Witkin and D. Terzopoulos, Snakes: Active contour models, Int. J. Computer Vision 1, 1988, 321-331
[18]Ming-Shing Su, Chun-Yen Chen and K. Y. Cheng, “An Automatic Construction of a Person's Face Model from the Person's Two Orthogonal Views.” accepted by Geometric Modeling and Processing, Japan, 2002.
[19]K.-Y.K. Wong, P.R.S. Mendonça and R. Cipolla. Head Model Acquisition from Silhouettes. In Proc. 4th International Workshop on Visual Form (IWVF4), pages 787-796, Capri, Italy, May 2001. Springer.
[20]Ming Shing Su, Ming Tat Ko and K. Y. Cheng. “Control of Feature-Point-Driven Facial Animation Using a Hypothetical Face” Computer Graphics Forum, 20(4):179-188, 2001.
[21]H.H.S. Ip and L. Yin, “Constructing a 3D individualized head model from two orthogonal views,” The Visual Computer, 12:254-266, 1996.
[22]C.W. Chen and C.L. Huang, “Human face recognition from a single front view,” International Journal of Pattern Recognition and Artificial Intelligence, 6(4);571-593, 1992.
[23]P.J. Burt and E.H. Adelson, “A multiresolution spline with application to image mosaics,” ACM Transactions on Graphics, 2:217-236, 1983.
[24]Ming Shing Su, W.L. Hwang and K. Y. Cheng,“A variational calculus approach to multiresolution image mosaic,” will appear in International Conference on Image Processing 2001.
[25]D. Brown. “The bundle adjustment - progress and prospect”, XIII Congress of the ISPRS, Helsinki, 1976.
[26]C. Slama, Manual of Photogrammetry, American Society of Photogrammetry, Falls Church, VA, USA, 4th edition, 1980.
[27]Levenberg, K., "A Method for the Solution of Certain Problems in Last Squares," Quart. Appl. Math. Vol. 2, pp. 164-168, 1944
[28]R.Y. Tsai, ™A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-the-Shelf TV Cameras and Lenses,º IEEE J. Robotics and Automation, vol. 3, no. 4, pp. 323-344, Aug. 1987.
[29]Y. C. Lee, D. Terzopoulos, and K.Waters. Realistic modeling for facial animation. In Computer Graphics, Annual Conference Series, pages 55–62. SIGGRAPH, 1995.
[30]Y. C. Lee, D. Terzopoulos, and K. Waters. Constructing physics-based facial models of individuals. In Proceedings of Graphics Interface, pages 1–8, 1993.
[31]V. Blanz and T. Vetter. A morphable model for the synthesis of 3d faces. In Computer Graphics, Annual Conference Series, pages 187–194. Siggraph, August 1999.
[32]Q.-T. Luong and O. Faugeras, ™Self-Calibration of a Moving Camera from Point Correspondences and Fundamental Matrices,™ Int'l J. Computer Vision, vol. 22, no. 3, pp. 261-289, 1997.
[33]Williams. Donna and Shah. Mubarak. "A Fast Algorithm for Active Contours and Curvature Estimation", CVGIP: Image Understanding. Vol. 55, No. 1, January 1992. pp. 14-26.
[34]M. Kass, A. Witkin, D. Terzopoulos. Snake: Active Contour Models. International Journal of Computer Vision. Vol.1, No.4, pp.321-331, 1988.
[35]G. Taubin, A Signal Processing Approach to Fair Surface Design, Proc. ACM SIGGRAPH '95, pp. 351-358, 1995.
[36]G.M. Nielson. Scattered data modeling. IEEE Computer Graphics and Applica-tions, 13(1):60{70, January 1993.
[37]Richard Hartley, Andrew Zisserman. “Multiple View Geometry in Computer Vision.” Cambridge University Press.
[38]G. H. Golub and C. F. Van loan. “Matrix Compuation.” The John Hopkins University Press, Baltimore, MD, second edition, 1989.
[39]Newton, Methodus fluxionum et serierum infinitarum, (1664-1671)
[40]Marquardt, D., “An Algorithm for Least Squares Estimation of Nonlinear Parameters,” SIAM J. Appl. Math. Vol. 11, pp. 431-441, 1963.
[41]R. O. Duda and P. E. Hart, “Pattern Classification and Scene Analysis”, New York: Wiley, 1973.
[42]R. C. Gonzalez and R. E. Woods, “Digital Image Processing (third edition), ” Reading, Massachusetts: Addison-Wesley, 1992.
[43]Reinhard Klette, Karsten Schluns and Andreas Koschan, “Computer Vision Three-Dimensional Data from Images”, ISBN 981-3083-71-9, Springer-Verlag Singapore Pte. Ltd. 1998.