臺灣博碩士論文加值系統

在這篇論文中，我們提出了一個以統計學習的方式，產生有表情的誇大漫畫的系統。此系統能從畫家的作品中學習此人的作畫風格，並能從無表情╱生氣╱開心的人臉照片自動產生有誇大效果的無表情╱生氣╱開心的漫畫圖片。現有的產生誇大漫畫的方法大多只能處理正面、脫帽、沒戴眼鏡、面無表情的臉部照片，並且無法同時處理兩種以上畫家可能使用的誇張效果。我們所提出的漫畫產生系統不但可以誇大臉部圖片的特徵及表情，也可以同時學習畫家多樣的誇張手法。我們的系統流程可以分為下列三個部分：臉部特徵的誇大、圖片紋理轉換、以及紋理映射。系統以LPH(Locality Preserving Hallucination)統計學習演算法來分析照片與畫家所畫的漫畫之間的關係，從而學習畫家是如何誇張人臉的特徵。接著使用Sobel邊緣檢測器、和已知的臉部特徵點等資訊，進而產生想要的漫畫紋理。將得到的漫畫紋理以及剛產生的誇大特徵點以RBF(Radial Basis Function)變形方法加以處理後，便可以得到有著我們想要的誇張效果的漫畫圖片。從實驗結果中可以看出，我們的系統可以找出某些畫家會特別著墨的臉部特徵，並對這些特徵以類似的手法加以誇大。

In this thesis, we proposed a learning-based system for generating exaggerative caricatures with expression. This system is capable of learning the drawing style of artists from their caricature works as the training data, and automatically creates exaggerative neutral/angry/happy caricatures from neutral/angry/happy images. Most previous works can only deal with frontal-view faces with neutral expression without glasses or hats, and cannot apply more than one drawing prototype learned from the caricatures drawn by a cartoonist at a time. The proposed caricature generation system exaggerates facial images with expression and learns the drawing prototypes from training data as well. The generation process is decomposed into three parts: facial feature exaggeration, texture transformation, and texture mapping. To learn how the cartoonist exaggerates the facial features of distinct facial expressions, the system analyzes the correlation between the photo caricature pairs using LPH (Locality Preserving Hallucination). Then apply Sobel edge detector as well as information of feature points to synthesize the desired texture. After combining exaggerated feature shapes with sketches by RBF (Radial Basis Function) warping, we can obtain caricatures with desired exaggeration. Experimental results show that our system can capture some features selected by the artist and exaggerate them in similar ways.

Contents

1 Introduction 1
2 Related Work 5
2.1 PICASSO system 6
2.2 Caricature generation by analyzing ficial feature 6
2.3 Caricature synthesis with inter and intra correlations 7
2.4 Example-based generation 8
3 Proposed Method 12
3.1 Caricature Model 12
3.2 Data 14
3.3 System Framework 18
3.4 Shape Exaggeration 20
3.4.1 LPP 22
3.4.2 Training Process : Exaggeration Analysis 24
3.4.3 Testing Process 26
3.5 Texture Transformation 28
3.6 Combination of texture and points 31
3.7 Adjust the exaggertion rate 32
4 Experimental Results 34
5 Conclusions 45
6 Reference 46





List of Figures

1: Face photos and their caricatures 1
2: Examples of desired system inputs and outputs 4
3: Linear model for exaggeration with constrained scaling. 7
4: Decompose face images into shape and texture 13
5: Examples of face and caricature images with the labeled feature points for different expressions 15
6: Examples of the female training data for caricature generation 16
7: Examples of the male training data for caricature generation 16
8: The transformation of the facial feature points during the face alignment procedure 18
9: Flow chart of the proposed caricature generation system 19
10: The six components of the facial feature points 21
11: Shape points (red) and reference points (blue) 22
12: Red points are the original feature points and blue points are the generated feature points for neutral, angry, and happy faces 27
13: Neutral / angry / happy photos and it generated gray scale sketch 31
14: Procedure of texture mapping. 32
15: Generated caricatures with different exaggeration rate 33
16: Examples of caricature generation for a person with different expressions compared to the ground truth 36
17: Examples of caricature generation for a person with different expressions compared to the ground truth 36
18: Examples of caricature generation for a person with different expressions compared to the ground truth 37
19: Examples of caricature generation for a person with different expressions compared to the ground truth 37
20: Examples of caricature generation for a person with different expressions compared to the ground truth 38
21: Examples of caricature generation for a person with different expressions compared to the ground truth 38
22: Examples of caricature generation for a person with different expressions compared to the ground truth 39
23: Examples of caricature generation for a person with different expressions compared to the ground truth 39
24: Photos which generate weird sketches 41
25: An example of a person with special features or expression that may not yield desired results 42
26: Unexaggerated neutral sketches, shown in the top row, and the caricatures obtained by applying the happy/angry models, shown in the middle and bottom rows 43
27: Unexaggerated happy sketches, shown in the top row, and the caricatures obtained by applying the neutral/angry models, shown in the middle and bottom rows 44
28: Unexaggerated angry sketches, shown in the top row, and the caricatures obtained by applying the neutral/happy models, shown in the middle and bottom rows 44





List of Tables

1. The execution environment of the proposed caricature generating system 34
2: The average execution time for all components of the proposed caricature generation system 35

[1] S. Brennan, "Caricature Generator," Master’s thesis, Cambridge, MIT, 1982
[2] B. Gooch, E. Reinhard, A. Gooch, "Human Facial Illustrations: Creation and Psychophysical Evaluation," ACM Transactions on Graphics, Vol. 23, pages 27-44, 2004
[3] F.J.J. Blommaert, J.-B. Martens, "An Object-Oriented Model for Brightness Perception," Spatial Vision, 5, 1, pages 15–41, 1990
[4] E. Akleman, "Making Caricature with Morphing," Proceeding of ACM SIGGRPH, page 145, 1997
[5] Q. Liu, X. Tang, H. Jin, H. Lu, S. Ma, "A Nonlinear Approach for Face Sketch Synthesis and Recognition," IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, pages 1005-1010, 2005
[6] H. Chen, Y. Xu, H. Shum, S. Zhu, N. Zheng, "Example-based facial sketch generation with non-parametric sampling," IEEE International Conference on Computer Vision Systems, Vol. 2, page 433-438, 2001
[7] H. Chen, Y. Xu, H. Shum, S. Zhu, N. Zheng, "Example-based Caricature Generation with Exaggeration," Proceedings of the 10th Pacific Conference on Computer Graphics and Applications, pages 386-393, 2002
[8] P.Y. Chiang, W.H. Liao, T.Y. Li, "Automatic Caricature Generation by Analyzing Facial Features," Proceedings of Asian Conference on Computer Vision, pages 89-94, 2004.
[9] C.-C. Tseng, J.-J.J. Lien, "Synthesis of Exaggerative Caricature with Inter and Intra Correlations," Proceedings of Asian Conference on Computer Vision, pages 314-323, 2007.
[10] Web Picasso (http://www.koshi-lab.sist.chukyo-u.ac.jp/pica2/)
[11] H. Koshimize, M. Tominaga, T. Fujiwara, K. Murakami, "On KANSEI Facial Processing for Computerized Facial Caricaturing System Picasso," Proceedings of IEEE International Conference on Systems, Man and Cybernetics, pages 294–299, 1999
[12] A. Hertzmann, "Painterly Rendering with Curved Brush Strokes of Multiple Sizes," Proceedings of the 25th annual conference on Computer Graphics and interactive Techniques, pages 453-460, 1998
[13] Y. Zhuang, J. Zhang, F. Wu, "Hallucinating Faces: LPH Super-resolution and Neighbor Reconstruction for Residue Compensation," Pattern Recognition, Vol. 40, pages 3178-3194, November 2007
[14] http://psychoprogs.com/pictures/celebrity-caricatures/
[15] http://www.floppingaces.net/upload/2007/11/adam_sandler_caricature.jpg
[16] http://movies.popcrunch.com/wp-content/uploads/2008/04/adam_sandler.jpg