臺灣博碩士論文加值系統

想要產生一張表情栩栩如生的三維人臉一向都要花費許多人力與時間，即使是經驗豐富的動畫師，仍然要花許多時間在人臉表情的細部調整與修改上。在今日的電影與遊戲製作中，真實表情的動作資料，可藉由三維動態捕捉技術擷取，並且被用來加速三維人臉表情的製作。然而，一般的三維動態捕捉技術僅能擷取較大維度的臉部幾何，對於臉部細微的紋理變化則很難有效捕捉。
在本篇論文中，我們針對臉部細紋的合成與表情的編輯，提出了一個以特徵點驅動的具細紋之三維臉部表情合成系統，此系統可分為兩個主要部分—三維臉部表情的編輯與臉部細微紋理的擷取。我們的系統提供了使用者一個較為直覺的編輯介面，並利用一群指定的臉部特徵點之間的幾何關係，使用一群範例影像估算並合成出新的表情的近似三維幾何結構與相對的法向量或二維外觀影像。除了用在單純的具細紋表情之編輯上，此一技術所產生的臉部細紋資料亦可進一步用在強化現有三維動態捕捉技術所產生的人臉表情上。

Producing a life-like 3D facial expression is usually a labor-intensive process. Even for an experienced animator, it spends much time on trial-and-error procedure. In movie and game industries, motion capture techniques and 3D scanning, acquiring motion data from real persons, are used to speed up the production. However, acquiring dynamic and subtle details, such as wrinkles, on a face are still difficult or expensive.
In this thesis, we developed a feature-point-driven approach to synthesize novel expressions with details. Our work can be divided into two main parts: acquisition of 3D facial details and expression synthesis. By employing geometric relation between specific feature points, our system provides an intuitive editing tool to synthesize 3D geometry and corresponding 3D detailed normal or 2D textures of novel expressions. Besides expression editing, the proposed method can also be extended to enhance existing motion capture data with facial details.

摘要 I
ABSTRACT II
ACKNOWLEDGEMENTS III
CONTENTS IV
LIST OF FIGURES V
1. INTRODUCTION 1
1.1 Motivation 1
1.2 Introduction 2
1.3 System Overview 3
2. RELATED WORK 5
3. ACQUISITION OF EXPRESSION DETAILS 11
3.1 Reconstruction of 3D Geometry 11
3.2 Normal Recovery 11
3.3 Normal difference map 14
3.4 Post Processing of Normal Difference Maps 16
4. FEATURE-POINT-DRIVEN SYNTHESIS OF NOVEL EXPRESSIONS 17
4.1 Calculation of Blending Weights 17
4.2 Synthesis of Textures and Normals 20
4.3 Editing and Synthesis of 3D geometry 22
4.3.1 Editing Interface 23
5. EXPERIMENT AND RESULTS 25
5.1 Collection and Pre-Processing of Prototype Images 25
5.2 Results 26
6. CONCLUSION AND FUTURE WORK 34
6.1 Conclusion 34
6.2 Future Work 34
7. REFERENCE 35

[1] Beier, T., and Neely, S. “Feature-based Image Metamorphosis”, Proceedings of the 19th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), Vol. 26, Pages 35-42, 1992.
[2] Blanz, V., Basso, C., Poggio, T., and Vetter, T. “Reanimating Faces in Images and Video”, Computer Graphics Forum 22 (3), EUROGRAPHICS 2003, Pages 641 - 650, 2003.
[3] Blanz, V., and Vetter, T. “A Morphable Model for the Synthesis of 3D Faces”, Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), Pages 187-194, 1999.
[4] Brand. M. “Morphable 3D models from video”, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Vol. 2, Pages 456-463, 2001.
[5] Breger, C., Covell, M., and Slaney, M. “Video rewrite: Driven Visual Speech with Audio”, Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), Page 353-360, 1997.
[6] Deng, Z., Chiang, P-Y., Fox, P., and Neumann, U. “Animating Blendshape Face by Cross-Mapping Motion Capture Data”, Proceedings of the 2006 Symposium on Interactive 3D Graphics and Games, Pages 43-48, 2006.
[7] Ezzat, T., Geiger, G., and Poggio, T. “Trainable Videorealistic Speech Animation”, Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), Pages 388-398, 2002.
[8] Fang, H., and Hart, J. C. “Textureshop: Texture Synthesis as a Photograph Editing Tool”, ACM Transactions on Graphics (TOG), Volume 23, Issue 3 (August 2004), Special Issue: Proceedings of the 2004 SIGGRAPH Conference., Pages 354-359, 2004.
[9] Fletcher, R. “Practical Methods of Optimization”, Vol. 1, John Wiley & Sons, 1980.
[10] Forsyth. Ponce “Computer Vision: a Modern Approach”, Prentice Hall, 2003.
[11] Golovinskiy, A., Matusik, W., and Pfister, H. “A Statistical model for Synthesis of Detailed Facial Geometry”, ACM Transactions on Graphics (TOG), Volume 25, Issue 3 (July 2006), SESSION: Appearance modeling, Pages: 1025-1034, 2006.
[12] Guenter, B. K., Grimm, C., Wood, D., Malvar, H. S., and Pighin, F. H., “Making Faces”, Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), Pages 55-66, 1998.
[13] Horn, B.K. 1990. “Height and Gradient from Shading”, International Journal of Computer Vision, Vol. 5(1), Pages 37-75, 1990.
[14] Lin, I.-C., and Ouhyoung, M. “Mirror Mocap: Automatic and Efficient Capture of Dense 3D Facial Motion Parameters from Video”, The Visual Computer, Vol. 12(6), Pages 355-372, 2005.
[15] Lin, I.-C., Yeh, J.-S., and Ouhyoung, M. “Extracting 3D Facial Animation Parameters from Multiview Video Clips”, IEEE Computer Graphics and Applications (special issue on tracking), Vol. 22(6), Pages 72-80, 2002.
[16] Liu, Z., Shan, Y., and Zhang, Z. “Expressive Expression Mapping with Ratio Images”, Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, Pages 271-276, 2001.
[17] Nielson, G. M. “Scattered Data Modeling” IEEE Computer Graphics and Applications, Vol.13, Issue 1, Pages 60-70, 1993.
[18] Noh, J-Y., and Neumann, U. “Expression Cloning”, Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), Pages 277-288, 2001.
[19] Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., and Salesin, D. H. “Synthesizing Realistic Facial Expressions from Photographs”, Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), Pages 75-84, 1998.
[20] Seitz, S. M., and Dyer, C. R. “View Morphing”. Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), Pages 21-30, 1996.
[21] Sifakis, E., Neverov, I. and Fedkiw, R., “Automatic Determination of Facial Muscle Activations from Sparse Motion Capture Marker Data”, ACM Transactions on Graphics (TOG), Volume 24, Issue 3 (July 2005), Proceedings of ACM SIGGRAPH 2005, Pages 417-425, 2005.
[22] Tu, P-H., Lin, I-C., Yeh, J-S., Liang, R-H., and Ouhyoung, M. “Surface Detail Capturing for Realistic Facial Animation”, Journal of Computer Science and Technology, Special Issue on Advances in Computer Graphics, Vol. 19(5), Pages 618-625, 2004.
[23] Waters, K. “A Muscle Model for Animating Three-Dimensional Facial Expression”, Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), Pages 17-24, 1987.
[24] Wenger, A., Gardner, A., Tchou, C., Unger, J., Hawkins, T., and Debevec, P. “Performance Relighting and Reflectance Transformation with Time-Multiplexed Illumination”, ACM Transactions on Graphics (TOG), Volume 24, Issue 3 (July 2005), Proceedings of ACM SIGGRAPH 2005, Pages 756-764, 2005.
[25] Zhang, L., Snavely, N., Curless, B., and Seitz, S. M. “Spacetime Faces: High-Resolution Capture for Modeling and Animation”, ACM Transactions on Graphics (TOG), Volume 23, Issue 3 (August 2004), Special Issue: Proceedings of the 2004 SIGGRAPH Conference, Pages 548-558, 2004.
[26] Zhang, Q., Liu, Z., Guo, B., and Shum, H. “Geometry-Driven Photorealistic Facial Expression Synthesis”, Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Pages 177-186, 2003.
[27] Zhang, Q., Liu, Z., Guo, B., Terzopoulos, D., and Shum, H. “Geometry-Driven Photorealistic Facial Expression Synthesis”, IEEE Trans. On Visualization and Computer Graphics, Vol. 12(1), Pages 48-60, 2006.
[28] Zhang, R., Tsai, P.-S., Cryer, J., and Shah, M. “Shape from Shading: A Survey”, IEEE Trans. On Pattern Analysis and Machine Intelligence, Vol. 21(8), Pages 690-706, 1999.