臺灣博碩士論文加值系統

人臉置換系統在娛樂業是相當受到重視的工作，但現今大部分的工作都是利用手動與特殊的道具輔助，目前存在人臉置換的文獻並沒有很多，在本篇論文中提出一個新的人臉置換系統，目的在於能將人臉置換的工作純粹使用影像處理來完成。
在系統中，利用了膚色分析與形態學(morphology)分割人臉區域，由邊緣偵測(edge detection)與連通量(connectivity)分析，配合唇色分析，找到嘴巴的特徵點，接著利用嘴巴與眼睛的幾何關係與投影函數(projection function)的分析將眼睛定位出來，再由梯度(gradient)與熵(entropy)找出下巴特徵點配合最小平方法(least square method)找到下巴的曲線。根據兩張臉的特徴點比對(feature matching)，我們可以將我們想要置換上的人臉做出適當的形變，並貼在適當的地方。而我們對人臉區域中的膚色部分分離出來，對原始臉與目標臉的膚色區域做直方圖比對(histogram matching)，將原始臉的膚色近似目標臉的膚色，使兩張人臉膚色相近。再借由影像混合(image blending)與平滑化消除接縫。實驗結果顯示，本研究提出的人臉置換系統，當在人臉有存在雙眼的角度中，人臉上沒有太大的光影變化與皮膚質感相差不大的情況下，都有不錯的效果。

Face replacement system plays an important role in the entertainment industries. Recently, however, most of these are assisted by hand and specific tools. Therefore, there are not many papers in this field. In this thesis, we describe a new face replacement system for automatically replacing a face with image processing.
In the system, we segment the facial region by skin color analysis and morphology. Feature detection and connectivity analysis are applied to find candidates of facial feature. Then we combine mouth color analysis to obtaining features of the mouth. The eyes are located by geometric relation with mouth and projection function. We obtain features of the chin by information of gradient and entropy. Then we use the least square method to construct the chin line. We can wrap and locate the target face by feature matching. We extract skin color pixels on face region. We make the skin color of the original face similar to that of the original face by histogram matching. Image blending and smoothing is used to eliminate the seam. As can be seen, the experimental results show that this face replacement system has good performance if both original and target faces are front-facing as long as there are no large variations in illumination and skin tones.

目錄

中文摘要 i
英文摘要 ii
誌謝 iii
目錄 iv
圖目錄 vii
表目錄 x

1 簡介 1-1
1.1 相關研究 1-1
1.2 論文架構 1-2
1.3 研究動機 1-6

2 影像處理技術 2-1
2.1 膚色色彩分析與臉部分割 2-1
2.1.1 顏色空間模型 2-2
2.1.2 形態學運算子 2-4
2.2 眼睛定位與投影函數 2-6
2.2.1 平均積分投影函數 2-6
2.2.2 標準差投影函數 2-7
2.3 邊緣偵測 2-8
2.4 影像混合 2-9
2.5 影像幾何透視轉換 2-10
2.5.1 透視轉換 2-10
2.5.2 空間轉換 2-11
2.6 影像內差 2-14

3 五官定位與人臉置換系統 3-1
3.1 系統流程概述 3-2
3.2 膚色偵測與分割 3-3
3.3 五官定位 3-7
3.3.1 嘴唇定位 3-8
3.3.2 眼睛定位 3-11
3.3.3 下巴曲線的建立 3-14
3.3.4 鼻子定位 3-17
3.4 人臉角度比對 3-19
3.5 膚色一致性的調整 3-24
3.6 影像形變 3-28
3.7 影像混合 3-30

4 實驗結果與分析 4-1
4.1 實驗環境 4-1
4.2 人臉置換的結果 4-2
4.2.1五官定位的結果 4-2
4.2.2 人臉置換的結果 4-4
4.3 實驗分析 4-15

5 結論與未來發展 5-1

參考文獻 I

[1] J. Kleiser, “Kleiser-Walczak on The One”. [Online]. Available: http://www.kwcc.com/works/ff/the_one.html

[2] P. Debevec, T. Hawkins, C. Tchou, H. P. Duiker, and W. Sarokin, “Acquiring the reflectance field of a human face,” in Proc. ACM SIGGRAPH, Louisiana, USA, Aug. 2000, pp. 145-156.
[3] S. Plelg and J. Herman, “Panoramic mosaics by manifold projection,” in Proc. IEEE Computer Society Conf. Computer Vision and Pattern Recognition, Puertok, Rico, June 1997, pp. 338-343.
[4] P. J. Burt and E. H. Adelson, “A multiresolution spline with application to image mosaics,” ACM Trans. Graphics, vol. 2, pp. 217-236, 1983.
[5] K. Sobottka and I. Pitas, “Extraction of facial regions and features using color and shape information,” in Proc. 13th IEEE Int. Conf. Pattern Recognition, Vienna, Austria, Sept. 1996, vol. 3, pp. 421-425.
[6] Z. Xin, X. Yanjun, and D. Limin, “Locating facial features with color information,” in Proc. 4th IEEE Int. Conf. Signal Processing, California, USA, Oct. 1998, vol. 2, pp. 889-892.
[7] M. Soriano, B. Martinkauppi, S. Huovinen, and M. Laaksonen, “Using the skin locus to cope with changing illumination conditions in color-based face tracking,” in Proc. IEEE Nordic Signal Processing Symposium (NORSIG2000), Kolmården, Sweden, June 2000, pp. 383-386.
[8] G. R. Bradski, “Computer vision face tracking for use in a perceptual user interface,” Intel Technology Journal, vol. 2, issue 2, pp. 12-26, 1998.
[9] D. Chai and K. N. Ngan, “Face segmentation using skin-color map in videophone applications,” IEEE Trans. on circuits and systems for video technology, vol. 9, no. 4, pp. 551-564, June 1999.
[10] Q. Chen, H. Wu, T. Shioyama, and T. shimada, “3D head pose estimation using color information,” in Proc. IEEE Int. Conf. Multimedia Computing and Systems, Florence, Italy, June 1999,vol.1, pp. 697-702.
[11] E. Maeland, “On the comparison of interpolation methods,” IEEE Trans. Medical Imaging, vol. 7, no. 3, pp. 213-217, Sept. 1988.
[12] P. V. Sankar and L. A. Ferrari, “Simple algorithms and architectures for B-spline interpolation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 10, no. 2, pp. 271-276, March 1988.
[13] T. M. Lehmann, C. Gonner, and K. Spitzer, “Addendum: B-spline interpolation in medical image processing,” IEEE Trans. Medical Imaging, vol. 20, no. 7, pp. 660-665, July 2001.
[14] J. Bala, K. DeJong, J. Huang, H. Vafaie, and H. Wechsler, “Visual routine for eye detection using hybrid genetic architectures,” in Proc. 13th IEEE Int Conf. Pattern Recognition, Vienna, Austria, Aug. 1996, vol. 3, pp. 606-610.
[15] M. J. T. Reinders, R. W. C. Koch, and J. J. Gerbands, “Locating facial feature in image sequences using neural networks,” in Proc. 2nd IEEE Int. Conf. Automatic Face and Gesture Recognition, California, USA, Oct. 1996, pp. 230-235.
[16] T. Kanade, Picture processing by computer complex and recognition of human faces [Ph. D. Thesis], Dept. Information Science, Kyoto University, 1973.
[17] G. C. Feng and P. C. Yuen, “Variance projection function and its application to eye detection for human face recognition,” Pattern Recognition Letters, vol. 19, no. 9, pp. 899-906, July 1998.
[18] R. C. Gonzalez and R. E. Woods, Digital Image Processing, Upper Saddle River, NJ: Prentice Hall, 2002.
[19] N. Eveno, A. Caplier, and P. Y. Coulon, “A new color transformation for lips segmentation,” in Proc. MMSP’ 01, Cannes, France, Oct. 2001, pp. 3-8.
[20] S. H. Leung, S. L. Wang, and W. H. Lau, “Lip image segmentation using fuzzy clustering incorporating an elliptic shape function,” IEEE Trans. Image Processing, vol. 13, no 1, pp. 51-62, Jan. 2004.
[21] R.T. Kumar, S.K. Raja, and A.G. Ramakrishnan, “Eye detection using color cues and projection function,” in Proc. IEEE Int. Conf. Image Processing, vol. 3, NY, USA , June 2002, pp. 337-340.
[22] H. Wu, Q. Chen, and M. Yachida, “Face detection from color images using fuzzy pattern matching method,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21 no. 6, pp. 557-563, June 1999.
[23] Y. Mitsukura, M. Fukumi, and N. Akamatsu, “A design of face detection system by using lip detection neural network and skin distinction neural network,” in Proc. IEEE Int. Conf. Systems, Man, and Cybernetics, vol. 4, TN, USA, Oct. 2000, pp. 2789-2793.
[24] H. Ai, L. Liang, and G. Xu, “Face detection based on template matching and support vector machines,” in Proc. IEEE Int. Conf. Image Processing, Thessaloniki, Greece, vol. 1, Oct. 2001, pp. 1006-1009.
[25] T. Yokoyama, H. Wu, and M. Yachida, “Automatic detection of facial feature points and contours,” in Proc. 5th IEEE Int. Workshop on Robot and Human Communication, Tsukuba, Japan, Nov. 1996, pp. 335-340.
[26] C. H. Lin and J. L. Wu, “Automatic facial feature extraction by genetic algorithms,” IEEE Trans. Image Processing, vol. 8, no. 6, pp. 834-845, June 1999.
[27] C. Lin and K.C. Fan, “Human face detection using geometric triangle relationship,” in Proc. 15th IEEE Int. Conf. Pattern Recognition, vol. 2, Barcelona, Spain, Sept. 2000, pp. 941-944.
[28] B. Hu, Q. Lin, X. Kang, and G. Chen, “A New Algorithm for Automatic White Balance with Priori,” in Proc. IEEE Asia-Pacific Conf. Circuits and Systems, Tianjin, China, Dec. 2000, pp. 109-112.