臺灣博碩士論文加值系統

在本論文中，將針對即時頭部三維運動軌跡的問題進行研究。主要目的在藉由對連續二維影像的分析，以獲得使用者頭部的三維運動位移及角度。所牽涉的問題可分為二方面：臉部特徵點的追蹤，及三維運動的推算。透過臉部特徵點的追蹤，可以找出兩張影像間的特徵點對應關係，而這些對應關係可作為後續的三維運動推算的依據，以計算出此時頭部的位移及角度。
針對此三維運動軌跡追蹤的問題，我們分別發展出即時特徵點追蹤及三維運動推算的方法。在所提出的即時特徵點追蹤方法中，藉由計算特徵點間不同特性的相關性，可以獲得較為可靠的特徵點對應關係。而在所提出的三維運動推算方法中，則是藉由求取一目標方程式的最小值來獲得三維運動的參數。此一目標方程式是由特徵點間的對應關係，配合對於頭部運動的觀察所設計而成的。在所提出的方法中，只需三個特徵點便能推算出頭部的三維運動，此三個特徵點分別是雙眼及鼻子。而所推算出的運動參數則包含了三個旋轉角度及三個平移量。
所發展出的技術可應用在不同的系統中。目前我們已經應用此項技術在一套以模型為基礎的視覺通訊系統中。在此系統中，利用所推算出的運動參數，配合一預先存在的頭部三維質紋貼圖模型，我們可以合成出一具有相對應運動的臉部影像。因此，此系統只需極低的網路頻寬即可進行視訊會議。在本論文中，已發展出此一雛型系統，而在單一Pentium-II 400MHz處理器上，可達到每秒25個畫面以上的處理速度。
事實上，三維頭部運動軌跡追蹤的技術不僅可應用在上述的視覺通訊系統中，也可用來作為更自然的人機界面工具，讓使用者可藉由頭部動作來和電腦或智慧型家電等進行溝通。

This dissertation addresses the problem of real-time 3D head motion tracking with a single camera. Through the analysis on a sequence of continuous facial images captured from a single video camera, the subject''s head motion in 3D space can be estimated. Two issues are discussed: feature tracking and 3D motion estimation. The purpose of feature tracking is to provide enough feature correspondence between two consecutive images, and the purpose of 3D motion estimation is to infer 3D motion from these feature correspondence.
To tackle the motion-tracking problem, techniques of real-time feature tracking and 3D motion estimation are proposed. The proposed feature tracking technique is developed based on cross-correlation between two feature patterns, with additional constraints to obtain better tracking results. The proposed 3D motion estimation method includes knowledge of head motion to develop an objective function that is minimized iteratively to obtain motion parameters. Three feature points, two eye cor-ners and one nostril, are required, and the recovered motion parameters are three rota-tion angles and three translation offsets, respectively.
The proposed techniques can be applied in different applications. One potential application is the model-based visual communication, where remote face images are synthesized based on parameters including those for facial expression and head mo-tion. Model-based coding is an emerging technique that can achieve very low bit-rate image coding, and is in particular applicable for mobile or network applications. A similar prototype system is developed in this dissertation. In the prototype, 3D head motion is inferred using the proposed techniques, and the recovered motion parame-ters are applied to a face synthesis module which renders a 3D texture-mapped head model in real-time.
The techniques developed can not only be used in model-based visual communi-cation, but also be applied to applications requiring natural human-computer interface (HCI). A user can interact with computers or other intelligent appliances via head motions.
The performance of the proposed 3D head motion tracking techniques is meas-ured on a PC with a Pentium-II 400 MHz CPU. The 3D motion estimation can update over 250 Hz, and the overall performance of the model-based visual communication system is over 25 frames/sec, with feature tracking and head model rendering in-volved.

封面
致謝
摘要
ABSTRACT
CONTENTS
LIST OF FIGURES
LIST OF TABLES
CHAPTER 1. INTRODUCTION
1.1 ISSUES IN HEAD MOTION TRACKING
1.2 CONTRIBUTIONS
1.3 ORGANIZATION
CHAPTER 2. SURVEY AND THE PROPOSED 3D HEAD MOTION TRACKING
2.1 REVIEW OF RELATED RESEARCHES
2.2 AN OVERVIEW TO THE PROPOSED 3D HEAD MOTION TRACKING TECHNIQUE
CHAPTER 3. FEATURE TRACKING
3.1 PROBLEM ANALYSIS
3.2 TRACKING WITH FBM
3.3 TRACKING WITH ARTIFICIAL WHITE MARKERS
CHAPTER 4. 3D HEAD MOTION ESTIMATION
4.1 PROBLEM FORMULATION
4.2 RELATED RESEARCHES
4.3 THE PROPOSED 3D MOTION ESTIMATION METHOD WITH THREE POINT CORRESPONDENCES
4.4 IMPLEMENTATION
4.5 RESULTS
CHAPTER 5. APPLICATONS IN MODEL-BASED VISUAL COMMUNICATION
5.1 VR-FACE: A 3D HEAD MOTION TRACKING APPLICATION
5.2 GRNERATION OF A CUSTOMIZED 3D HEAD MODEL
CHAPTER 6. CONCLUSIONS AND FUTURE WORK
6.1 CONTRIBUTIONS
6.2 FUTURE WORK
BIBLIOGRAPHY
APPENDIX A. MATHEMATICS OF CROSS-CORRELATION
APPENDIX B. OPTIMIZATION TECJMOQIES
APPENDIX C.MORE ON THE COST FUNCTIONS
RESUME
PUBLICATION LIST

[Aiza95] K. Aizawa, and T. S. Huang, "Model-Based Image Coding: Advanced Video Coding Techniques for Very Low Bit-Rate Applications," Proceedings of the IEEE, Vol. 83, No. 2, pp. 259-271, 1995.
[Aiza89] K. Aizawa, H. Harashima, T. Saito, "Model-Based Analysis Synthesis Im-age Coding (MBASIC) System for a Person''s Face," Signal Processing: Im-age Communication, Vol. 1, pp. 139-152, 1989.
[Arun87] K. S. Arun, T. S. Huang, S. D. Blostein, "Least-Squares Fitting of Two 3-D Point Sets," IEEE Transactions on Pattern Analysis and Machine Intelli-gence, Vol. 9, No. 5, pp. 698-700, 1987.
[Bala97] L. Bala, K. Talmi, J. Liu, "Automatic Detection and Tracking of Faces and Facial Features in Video Sequences," Picture Coding Symposium, Berlin, Germany, 1997.
[Basu96] S. Basu, I. Essa, A. Pentland, "Motion Regularation for Model-Based Head Tracking," Proceedings of International Conference on Pattern Recognition (also available in Technical Report 362, MIT Media Laboratory, ftp://whitechapel.media.mit.edu/pub/tech-reports/TR-362.ps.Z), Vienna, Austria, 1996.
[Beier92] T. Beier, S. Neely, "Feature-Based Image Metamorphosis," Proceedings of Computer Graphics (SIGGRAPH''92), Chicago, USA, pp. 35-42, 1992.
[Birc98] S. Birchfield, "Elliptical Head Tracking Using Intensity Gradients and Color Histograms," Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (also available at http://vision.stanford.edu/public /publication/birchfield/birchfieldCvpr98.pdf), Santa Barbara, USA, 1998.
[Black95] M. J. Black, Y. Yacoob, "Tracking and Recognizing Rigid and Non-Rigid Facial Motions using Local Parametric Models of Image Motion," Proceed-ings of the International Conference on Computer Vision, Cambridge, USA, pp. 374-381, 1995.
[Blos88] S. D. Blostein, T. S. Huang, "Algorithms for Motion Estimation Based on Three-Dimensional Correspondences," Motion Understanding, W. Martin and J. K. Aggrawal Ed., Norewell, MA: Kluwer, pp. 329-352, 1988.
[Brad98] G. R. Bradski, "Computer Vision Face Tracking for Use in a Perceptual User Interface," Intel Journal 1998 Q2, http://developer.intel.com/ technol-ogy/itj/q21998/pdf/camshift.pdf, 1998.
[Casc98] M. L. Cascia, S. Sclaroff, "Fast, Reliable Head Tracking under Varying Il-lumination," Boston University Computer Science Technical Report, No. 98-018, http://cs.ftp.bu.edu/techreports/98-018-fast.ps.gz, 1998.
[Chen98] Q. Chen, H. Wu, T. Fukumoto, M. Yachida, ŗD Head Pose Estimation without Feature Tracking," Proceedings of the International Conference on Automatic Face and Gesture Recognition, Nara, Japan, pp. 88-93, 1998.
[Chong96] E. K. P. Chong, S. H. Zak, An Introduction to Optimization, John Wiley & Sons, Inc., New York, 1996.
[Colm97] A. Colmenarez, R. Lopez, T. S. Huang, ŗD Model-Based Head Tracking," Proceedings of Visual Communications and Image Processing (also avail-able at http://troi.ifp.uiuc.edu/~antonio/Papers/vcip97.ps.gz), San Jose, USA, 1997..
[Cosa98] E. Cosatto, H. P. Graf, "Sample-Based Synthesis of Photo-Realistic Talking Heads," Proceedings of Workshop on Computer Animation, Philadelphia, USA, pp. 103-110 1998.
[Cove95] M. Covell, "Autocorrespondence: Feature-Based Match Estimation and Image Metamorphosis," Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics (also available at http://www.interval.com/ ~covell/pubs.html), Vancouver, Canada, 1995.
[DeCa96] D. DeCarlo, D. Metaxas, "The Integration of Optical Flow and Deformable Models with Applications to Human Face Shape and Motion Estimation," Proceedings of Computer Vision and Pattern Recognition (CVPR''96), pp. 231-238, 1996.
[Fua98] P. Fua, "Face Models from Uncalibrated Video Sequences," Proceedings of CAPTECH''98, Geneva, Switzerland, pp. 214-228, 1998.
[Gill81] P. E. Gill, W. Murray, M. H. Wright, Practical Optimization, Academic Press, London, UK, 1981.
[Graf95] H. P. Graf, T. Chen, E. Petajan, E. Cosatto, "Locating Faces and Facial Parts," Proceedings of the 1st International Conference on Automatic Face and Gesture Recognition, Zurich, Switzerland, pp. 41-46, 1995.
[Graf96] H. P. Graf, E. Cosatto, D. Gibbon, M. Kocheisen, "Multi-Modal System for Locating Heads and Faces," Proceedings of the 2nd International Confer-ence on Automatic Face and Gesture Recognition, Killington, USA, pp. 88-93, 1996.
[Guen98] B. Guenter, C. Grimm, D. Wood, H. Malvar, F. Pighin, "Making Faces," Proceedings of Computer Graphics (SIGGRAPH''98), Florida, USA, 1998.
[Hara92] R. M. Haralick, L. G. Shapiro, Computer and Robot Vision, Vol. I, Addision-Wesley Co., 1992.
[Hara93] R. M. Haralick, L. G. Shapiro, Computer and Robot Vision, Vol. II, Ad-dision-Wesley Co., 1993.
[Hein97] J. Heinzmann, A. Zelinsky, "Robust Real-Time Face Tracking and Gesture Recognition," Proceedings of the International Joint Conference on Artifical Intelligence (IJCAI''97), Vol. 2, pp. 1525-1530, 1997.
[Hein98] J. Heinzmann, A. Zelinsky, ŗ-D Facial Pose and Gaze Point Estimation using a Robust Real-Time Tracking Paradigm," Proceedings of the 3rd In-ternational Conference on Automatic Face and Gesture Recognition, Nara, Japan, pp. 142-147, 1998.
[Heip96] C. Heipke, "Overview of Image Matching Techniques," Proceedings of OEEPE Workshop on the Application of Digital Photogrammetric Worksta-tions (also available at Http://dgrwww.epfl.ch/PHOT/publicat/wks96/ Art_3_1.html), Lausanne, Switzerland, 1996.
[Horp96] T. Horprasert, Y. Yacoob, L. S. Davis, "Computing 3-D Head Orientation from a Monocular Image Sequence," Proceedings of the 2nd International Conference on Automatic Face and Gesture Recognition, Killington, USA, pp. 242-247, 1996.
[Hott90] M. Hotter, "Object-Oriented Analysis-Synthesis Coding Based on Moving Two-Dimensional Objects," Signal Processing: Image Communication, Vol. 2, pp. 409-428, 1990.
[Huang94] T. S. Huang, and A. N. Netravali, "Motion and Structure from Feature Correspondences: A Review," Proceedings of the IEEE, Vol. 82, No. 2, pp. 252-268, 1994.
[Jaco72] S. L. S. Jacoby, J. S. Kowalik, J. T. Pizzo, Iterative Methods for Nonlinear Optimization Problems, Prentice-Hall, Inc., Englewood Cliffs, New Jersey, USA, 1972.
[Jahn91] B. Jahne, Digital Image Processing: Concepts, Algorithms, and Scientific Applications, Springer-Verlag, New York, 1991.
[Kamp97] M. Kampmann, J. Ostermann, "Automatic Aadaptation of a Face Model in a Layered Coder with an Object-Based Analysis-Synthesis Layer and a Knowledge-Based Layer," Signal Processing: Image Communication, Vol. 9, pp. 201-220, 1997.
[Lee98] W. Lee, N. M. Thalmann, "Head Modeling from Pictures and Morphing in 3D with Image Metamorphosis Based on Triangulation," Proceedings of CAPTECH''98, Geneva, Switzerland, pp. 254-267, 1998.
[Lopez97] R. Lopez, A. Colmenarez, T. S. Huang, "Head and Feature Tracking for Model-based Video Coding," Proceedings of International Workshop on Synthetic-Natural Hybrid Coding and 3-D Imaging (also available at http://troi.ifp.uiuc.edu/~antonio/Papers/snhc97.ps.gz), Greece, 1997.
[Mart97] G. Martinez, "Shape Estimation of Articulated 3D Objects for Object-Based Analysis-Synthesis Coding (OBASC)," Signal Processing: Image Commu-nication, Vol. 9, pp. 175-199, 1997.
[MPE97] MPEG-4 committee, "FBA Core Experiments," ISO/IEC JTC1/SC29/ WG11/ N1775, 1997.
[MPE98a] MPEG-4 committee, "MPEG-4 Applications," ISO/IEC JTC1/SC29/ WG11/N2195, Tokyo, Japan, 1998.
[MPE98b] MPEG-4 committee, "MPEG-4 Coding of Audio-Visual Objects: Visual," ISO/IEC 14496-2, Committee Draft, Tokyo, 1998.
[Musm89] H. G. Musmann, M. Hotter, J. Ostermann, "Object-Oriented Analysis-Synthesis Coding of Moving Images," Signal Processing:Image Communi-cation, Vol. 1, pp. 117-138, 1989.
[Netr85] A. N. Netravali, J. Salz, "Algorithms for Estimation of Three-Dimensional Motion," AT&T Technical Journal, Vol. 64, No. 2, pp. 335-346, 1985.
[Oste90] J. Ostermann, "Modelling of 3D-Moving Objects for an Analysis-Synthesis Coder," Proceedings of SPIE 1260: SPIE/SPSE Symposium on Sensing and Reconstruction of Three-Dimensional Objects, pp. 240-249, 1990.
[Oste94a] J. Ostermann, M. Kampmann, "Automatic Adaptation of a Facial Mask in an Analysis-Synthesis Coder Based on Moving Flexible 3D Objects," Pro-ceedings of International Workshop on Coding Techniques for Very Low Bit-rate Video (VLBV 94) (also available at ftp://ftp.tnt.uni-hannover.de/pub/papers/ 1994/VLBV94-JOMK.ps.gz), Colchester, UK, 1994.
[Oste94b] J. Ostermann, "Object-Based Analysis-Synthesis Coding Based on the Source Model of Moving Rigid 3D Objects," Signal Processing: Image Communication, Vol. 6, pp. 143-161, 1994.
[Oste98] J. Ostermann, M. Beutnagel, A. Fischer, Y. Wang, "Integration of Talking Heads and Text-To-Speech Synthesizers for Visual TTS," Proceedings of International Conference on Speech and Language Processing (ICSLP''98) (also available at http://www.research.att.com/~osterman/AnimatedHead/ Icslp/Icslp931.pdf), Sydney, Australia, 1998.
[Pappu98] R. Pappu, P. A. Beardsley, "A Qualitative Approach to Classifying Gaze Direction," Proceedings of the 3rd International Conference on Automatic Face and Gesture Recognition, Nara, Japan, pp. 160-165, 1998.
[Pigh98] F. Pighin, J. Hecker, D. H. Salesin, D. Lischinski, R. Szeliski, "Modeling Realistic Facial Expressions From Photographs," Proceedings of Computer Graphics (SIGGRAPH''98) (also available in technical report UW-CSE-97-01-03), Orlando, USA, 1998.
[Press88] W. H. Press, B. P. Flannery, S. A. Teukolsky, W. T. Vetterling, Numerical Recipes in C: The Art of Scientific Computing, Cambridge University Press, Cambridge, UK, 1988.
[Ross88] S. Ross, A First Course in Probability, 3rd Edition, Macmillan Publishing Company, New York, USA, 1988.
[Rous87] P. J. Rousseeuw, A. M. Leroy, Robust Regression and Outlier Detection, John Wiley & Sons, Inc, New York, USA, 1987.
[Rudin76] W. Rudin, Principles of Mathematical Analysis, Third Edition, McGraw-Hill, Inc., New York, 1976.
[Rydf87] M. Rydfalk, "CANDIDE - a Parameterised Face," Report LiTH-ISY-I-0866, Linkoping University, 1987. A demonstration is available at http://www.bk.isy.liu.se/candide/candemo.html.
[Saber98] E. Saber, A. M. Tekalp, "Frontal-View Face Detection and Facial Feature Extraction using Color, Shape and Symmetry Based Cost Functions," Pat-tern Recognition Letters, Vol. 19, pp. 669-680, 1998.
[Seit96] S. M. Seitz, C. R. Dyer, "View Morphing," Proceedings of Computer Graphics (SIGGRAPH''96), New Orleans, USA, pp. 21-30, 1996.
[Seit97] S. M. Seitz, "Image-Based Transformation of Viewpoint and Scene Appear-ance," PhD Dissertation, University of Wisconsin-Madison, 1997.
[Stew94] D. E. Stewart, Z. Leyk, Meschach Library, Version 1.2b, 1994. Available at http://www.netlib.org/c/meschach/.
[Toya98a] K. Toyama, "Prolegomena for Robust Face Tracking," Microsoft Research Technical Report, MSR-TR-98-65, 1998.
[Toya98b] K. Toyama, "Radial Spanning for Fast Blob Detection," Proceedings of International Conference on Computer Vision, Pattern Recognition, and Im-age Processing, 1998.
[Weng89] J. Weng, T. S. Huang, N. Ahuja, "Motion and Structure from Two Perspec-tive Views: Algorithms, Error Analysis, and Error Estimation," IEEE Trans-actions on Pattern Analysis and Machine Intelligence, Vol. 11, No. 5, pp. 451-476, 1989.
[Wu94] J.-R. Wu, M. Ouhyoung, "Reducing The Latency in Head-Mounted Display by A Novel Prediction Method Using Grey System Theory," Computer Graphics Forum (also in EUROGRAPHICS''94, Oslo, Norway, 1994), Vol. 13, No. 3, pp. C503-512, 1994.
[Yang95] J. Yang, A. Waibel, "A Real-Time Face Tracker," Proceedings of the 3rd IEEE Workshop on Applications of Computer Vision (also in CMU Techni-cal Report CMU-CS-95-210), Sarasota, USA, pp. 142-147, 1996.
[Yang97] J. Yang, W. Lu, A. Waibel, "Skin-Color Modeling and Adaptation," CMU Technical Report CMU-CS-97-146, 1997.
[Yang98] T.-J. Yang, F.-C. Wu, M. Ouhyoung, "Real-Time 3D Head Motion Estima-tion in Facial Image Coding," Proceedings of Multimedia Modeling Confer-ence (MMM''98), Lausane, Switzerland, pp. 50-51, 1998.
[Zhang95] Z. Zhang, R. Deriche, O. Faugeras, Q.-T. Luong, "A Robust Technique for Matching Two Uncalibrated Images Through the Recovery of the Unknown Epipolar Geometry," Artificial Intelligence, Vol. 78, pp. 87-119, 1995.
[Zhang97] L. Zhang, "Tracking a Face for Knowledge-Based Coding of Videophone Sequences," Signal Processing: Image Communication, Vol. 10, pp. 93-114, 1997.