臺灣博碩士論文加值系統

摘要
頭部姿勢估計(Head Pose Estimation)在虛擬和即時電腦視覺領域是一項非常重要的研究。這個研究主要分析攝影機中人臉頭部在三個軸的旋轉角度。此研究可應用在許多虛擬幻像或實體層面。在一個虛擬的影像遊戲，3D圖像可以移動並旋轉不同的角度。在即時影像捕捉，攝影機也可以通過臉部偵測去假設駕駛目前的身心狀態(即專心程度)。

隨著科學技術的快速發展，頭部姿勢估計也在電腦和行動電話系統中變得越來越複雜，這是做為工作和娛樂必要的；應用範圍廣泛的工具。在多媒體方面，頭部姿勢估計可以應用在2D到3D的圖像處理，虛擬實境（Virtual Reality, VR）也演進到現在的擴增實境（Augmented Reality, AR）。擴增實境和頭部姿勢估計結合可被廣泛的應用，例如，教育，遊戲，醫療，交通，商業和其他領域。因此，多重即時頭部姿勢估計和擴增實境在科技上的結合發展將可以開發出許多有用的技術系統。雖然已有許多頭部姿勢估計的相關研究被發表，但這些技術的系統都有一定的局限性。在先前的頭部姿勢估計的研究中，頭部圖像的旋轉只能偵測到某個角度，不能較準確的計算出目前轉到的角度。

頭部姿勢估計系統證明了如何結合多重頭部姿勢估計系統和擴增實境技術。頭部姿勢估計系統被分為三個模組：(1)多重人臉偵測模組，(2)頭部姿勢估計模組，(3)擴增實境模組。多重人臉偵測模組利用了Adaboost 分類器，在多張圖片上擷取人臉特徵並產生一分類決策樹，決策樹可以決定和偵測在螢幕上有多少張人臉。多重頭部姿勢估計模組包含了區域化梯度方向(Localized Gradient Orientation, LGO)，我們利用區域化梯度方向將螢幕上圖像中的人臉特徵做向量分析，以及使用支援向量機(Support Vector Machine ，SVM)去進行Yaw(X軸)，Roll(Z軸)和Pitch(Y軸)的估計和頭部圖像在三軸所產生的位置。擴增實境模組利用擴增實境去產生3D頭部圖像，讓頭像可以旋轉多重角度和軸向。

在結論的部分，頭部姿勢估計系統的獨特之處是可以一次偵測到多張人臉以及一次進行多個頭部的姿勢估計。該系統可應用在如今的電子商務領域、公共安全，以及交通和教育。對於未來，頭部姿勢估計可以被應用到醫療、娛樂以及軍事領域等超越想像的各種用途。

Abstract
Head Pose Estimation is a very important research issue in the virtual and real-time computer vision domains. This study analyzed how a 3D image’s head can be rotated at various angles at three different axes. The application of this study can be utilized in various aspects of virtual fantasy or reality. In a virtual videogame, 3D images can be rotated at different angles while in motion. In real-time video capture, traffic cameras can detect and assume drivers’ current physical and emotional status via facial detect while driving. In the rapid development of science and technology, Head Pose Estimation can also be applied in computers and mobile phones systems that have become increasing complex, which are essential tools for a wide range of applications for work and play. In multimedia, Head Pose Estimation can be applied in image processing from 2D to 3D, Virtual Reality (VR) evolution to Augmented Reality (AR).
Augmented Reality (AR) and Head Position Estimation combined can be used in a wide range of applications, for example, education, video game consoles, medicine, transportation, commercial and other areas. Therefore, the development that combines the techniques of multiple real-time head pose estimation with augmented reality technology will be able to develop many useful technological systems. Although there are many related research published for Head Position Estimation, these technological systems all have had some limitations. In previous research in conducting Head Pose Estimation, the images of heads could only be detected to a certain angle and could not
be more accurate calculation of the current angle.
The Head Pose Estimation system demonstrates how the development of a set of multiple head pose estimations system combined with augmented reality technology was achieved. The Head Pose Estimation system is divided into three modules: (1) multiple face detection module, (2) head pose estimation module, and (3) augmented reality modules. Multiple face detection modules utilize the Adaboost classifier to captures facial features in multiple images that generates a classifier decision tree, which can determine and detect how many various faces are on the screen. Head pose estimation modules incorporate the Localized Gradient Orientation (LGO), which allow the facial features of an image on the screen use data generated by the LGO vector analysis and Support Vector Machine (SVM) to carry out Yaw (x-axis), Roll (z-axis) and Pitch (y-axis) estimate and produce positions of head images in the three axes. Augmented reality modules utilize AR to generate 3D head images that can rotate various angles and axes.
In conclusion, the characteristics of Head Pose Estimation system can detect more than one face at once as well as conduct multiple head pose estimation in unison. The application of this system can be utilized in today’s e-commerce arena, public safety, as well as transportation and education. For the future, Head Pose Estimation can be applied in the even in medical, entertainment or even military arena for various usages beyond today’s imagination.

目錄
圖次 V
表次 IX
第一章、緒論
第一節、研究背景與動機 1
第二節、研究目的 2
第二章、相關研究
第一節、臉部偵測 (Face Detection) 相關研究 3
壹、以知識為基礎的方法(Knowledge-based Methods) 3
貳、以特徵為基礎的方法(Feature-based Methods) 4
參、以模板比對為基礎的方法(Template-based Methods) 10
肆、圖像學習法(Image-based Methods) 10
第二節、頭部姿勢估計(Head Pose Estimation)相關研究 11
壹、外觀模板法(Appearance Template Methods) 12
貳、偵測器陣列法(Detector Array Methods) 13
參、非線性迴歸法(Nonlinear Regression Methods) 14
肆、多樣式嵌入法(Manifold Embedding Methods) 15
伍、彈性模型(Flexible Models) 16
陸、幾何法(Geometric Methods) 16
柒、追蹤法(Tracking Methods) 17
捌、混合法(Hybrid Methods) 18
第三節、擴增實境相關研究 18
壹、早期的擴增實境系統架構 19
貳、標記式與無標記式擴增實境 21
第三章、系統實作
第一節、系統架構 29
 多重人臉偵測模組 29
 頭部姿勢估計模組 29
 擴增實境模組 30
第二節、多重人臉偵測模組 30
壹、積分圖像(Integral Image)模組 30
貳、Adaboost模組 34
參、多重人臉偵測 40
第三節、人臉轉向估計模組 41
壹、人臉灰階方向梯度 41
貳、支援向量機 42
參、轉向估計定位與計算 44
第四節、擴增實境(Augmented Reality ,AR) 45
第四章、結果與驗證
第一節、多重人臉偵測模組執行結果 50
第二節、人臉轉向估計模組執行結果 51
壹、區域化梯度方向 (Localized Gradient Orientation, LGO) 51
貳、支援向量機(Support Vector Machine ，SVM) 52
i.SVM訓練樣本 52
ii.SVM訓練樣本檔案 53
iii.SVM樣本製作 53
iv.訓練樣本畫面 56
參、訓練樣本說明 58
肆、實際測試 65
伍、結合到擴增實境(Augmented Reality ,AR) 72
第三節、實際執行的結果 72
第五章、結論與未來工作
A.多重人臉偵測模組 78
B.頭部姿勢估計模組 78
C.擴增實境模組 78
參考文獻 80

參考文獻
一、中文
1.歐俊岳(2003)，一個以人臉影像為基礎的影像搜尋系統，國立臺灣大學資訊工程學研究所碩士論文
2.王科翔(2005)，多重人臉偵測與識別系統，國立成功大學工程科學研究所碩士論文
3.鄭凱方(2005)，人臉可辨識度計算用於監控系統中人臉正面最佳影像判定，國立中央大學資訊工程研究所碩士論文
4.林宗勳(2006)，Support Vector Machines 簡介，台灣大學通訊與多媒體實驗室
5.林家吉(2008)，利用人臉辨識追蹤系統於鎖定物體之保護，逢甲大學資訊電機工程碩士在職專班碩士論文
6.林建良(2008)，使用三維虛擬人臉之辨識系統，國立成功大學工程科學研究所碩士論文
7.陳彥劭(2008)，即時人臉辨識系統之研製，南台科技大學電機工程研究所碩士論文
8.楊淳凱(2008)，基於自我組織特徵映射圖之人臉表情辨識，國立中央大學資訊工程研究所碩士論文
9.林立偉(2009)，應用網路攝影機來辨識頭部轉向動作，南台科技大學機械工程研究所碩士論文
10.劉翁昌 (2009)，複雜環境下之即時人臉偵測與辨識系統，國立台灣科技大學電子工程系碩士學位論文
11.孫詩華(2011)，人臉偵測與頭部姿勢估計在擴增實境上的應用，實踐大學資訊科技與管理學系



二、英文
1.G. Andrew & C. Roberto (1996) , Fast visual tracking by temporal consensus , Image and Vision Computing, 14(2) , pp. 105–114.
2.S. Ba & J.M. Odobez, (2004), A probabilistic framework for joint head tracking and pose estimation, Proceedings of Int’l. Conf. Pattern Recognition, pp. 264–267.
3.V. Balasubramanian, J. Ye, & S. Panchanathan, (2007), Biased manifold embedding: A framework for person-independent head pose estimation, Proceedings of IEEE Conf. Computer Vision and Pattern Recognition.
4.H. Bay, A. Ess, T. Tuytelaars & Van Gool, L.(2006), Speeded-Up Robust Features (SURF), CVIU, 110(3) , pp. 346–359.
5.M. Belkin & P. Niyogi, (2003), Laplacian eigenmaps for dimensionality reduction and data representation, Neural Computation, 15 ( 6) , pp. 1373–1396.
6.D. Beymer, (1994), Face recognition under varying pose, Proceedings of IEEE Conf. Computer Vision and Pattern Recognition, pp. 756–761.
7.S. Birchfield, (1997), An Elliptical Head Tracker, Proceedings of Record of the Asilomar Conference on Signals, Systems & Computers, 2 , pp.1710–1714.
8.S. Birchfield, (1998), Elliptical Head Tracking Using Intensity Gradients and Color Histograms, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp.232–237.
9.C. Boehnen & T. Russ, (2005), A fast multi-modal approach to facial feature detection, Proceedings of the 7th IEEE Workshop on Applications of Computer Vision, pp. 135–142.
10.C. Boehnen, & T. Russ, (2005), A fast multi-modal approach to facial feature detection, Proceedings of the 7th IEEE Workshop on Applications of Computer Vision, pp. 135–142.
11.J. Y. Bouguet, (2001), Pyramidal Implementation of the Lucas Kanade Feature Tracker Description of the Algorithm, Intel Corporation Microprocessor Research Labs.
12.M. L. Cascia, S. Sclaroff, & V. Athitsos, (2000), Fast, reliable head tracking under varying illumination: An approach based on registration of texture-mapped 3d models, IEEE Trans. Pattern Anal. Machine Intell., 22(4) , pp. 322–336.
13.C. C. Chiang, , W. K.. Tai, , Yang, M. T., Haung, Y. T. & Huang C. J.(2003), A Novel Method for Detecting Lips, Eyes and Faces in real time, Real-Time Imaging, 9(4) , pp. 277–278.
14.L. Chiunhsiun, & Fan. K.C.,(2004), Triangle-based Approach to the Detection of Human Face, Pattern Recognition, 34 , pp.1271–1284.
15.T. Cootes, K. Walker, & C. Taylor, (2000), View-based active appearance models, in Proceedings of Int’l. Conf. Automatic Face and Gesture Recognition, pp. 227–232.
16.G. L. David , (2004), Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision on 2004 , 60(2), pp.91–110
17.D. F. DeMenthon, & L. S. Davis, (1995), Model-Based Object Pose in 25 Lines of Code, International Journal of Computer Vision, 15(1), pp.123–141.
18.M.C, Eric & M. M, Trivedi, (2007), Head pose estimation for driver assistance systems: A robust algorithm and experimental evaluation, Proceedings of IEEE Conf. Intelligent Transportation Systems, pp. 709–714.
19.M.C. Eric, & M. M, Trivedi, (2008), Hybrid head orientation and position estimation (HyHOPE): A system and evaluation for driver support, Proceedings of IEEE Intelligent Vehicles Symposium, pp. 512–517.
20.M.C, Eric, & M. M. Trivedi, (2008), Head Pose Estimation in Computer Vision: A Survey, IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(4) , pp.607–626.
21.Y. Fu & T. Huang, (2006), Graph embedded analysis for head pose estimation, Proceedings of IEEE Int’l. Conf. Automatic Face and Gesture Recognition, pp. 3–8.
22.C. Garcia, & G. Tziritas, (1999), Face Detection Using Quantized Skin Color Regions Merging and Wavelet Packet Analysis, IEEE Transactions on Multimedia, 1(3) , pp. 264–277.
23.C. Garcia, & M. Delakis, (2004), Convolutional Face Finder：A Neural Architecture for Fast and Robust Face Detection, IEEE Transactions on pattern analysis and machine intelligence, 26(11) , pp. 1048–1423.
24.A. Gee & R. Cipolla, (1994), Determining the gaze of faces in images, Image and Vision Computing, 12 (10) , pp. 639–647.
25.N. Gourier, D. Hall, & J. Crowley, (2004), Estimating face orientation from robust detection of salient facial structures, Proceedings of Pointing 2004 Workshop: Visual Observation of Deictic Gestures, pp. 17–25.
26.C. Harris & M.J. Stephens, (1988), A combined corner and edge detector, In Alvey Vision Conference, pp.147–152.
27.T. Horprasert, Y. Yacoob, & L. Davis, (1996), Computing 3-d head orientation from a monocular image sequence, Proceedings of Int’l. Conf. Automatic Face and Gesture Recognition, pp. 242– 247.
28.R. L. Hsu, A. M. Mohamed, & A. K. Jain, (2002), Face Detection in Color Image, IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(5) , pp. 696–706.
29.N. Hu, W. Huang, & S. Ranganath, (2005), Head pose estimation by non-linear embedding and mapping, Proceedings of IEEE Int’l. Conf. Image Processing, (2), pp. 342–345.
30.J. Huang, X. Shao, & H. Wechsler, (1998), Face pose discrimination using support vector machines (SVM), Proceedings of Int’l. Conf. Pattern Recognition, pp. 154–156.
31.T. Jebara & A. Pentland, (1997), Parametrized structure from motion for 3d adaptive feedback tracking of faces, Proceedings of IEEE Conf. Computer Vision and Pattern Recognition, pp. 144–150.
32.M. Jones, & Viola, P.(2003), Fast Multi-view Face Detection. Technical Report, MERL.
33.N. Krüger, , M. ötzsch, & C. V. D. Malsburg, (1997), Determination of Face Position and Pose with a Learned Representation based on Labeled Graphs, Image and Vision Computing, 15(8), pp. 665–673.
34.T. Lee, & T. Höllerer, (2007), Handy AR: Markerless Inspection of Augmented Reality Objects Using Fingertip Tracking, Proceedings of the 11th IEEE International Symposium on Wearable Computers, pp. 83–90.
35.Y. Li, S. Gong, J. Sherrah, & H. Liddell, (2004), Support vector machine based multi-view face detection and recognition, Image and Vision Computing, 22(5), pp.2004.
36.B. D. Lucas, & Kanade, T.(1986), An Investigation of Smoothness Constraints for the Estimation of Displacement Vector Fields from Image Sequences, IEEE Transactions on Pattern Analysis and Machine Intelligence, 8, pp. 565–593.
37.H. Moon & M. Miller, Estimating facial pose from a sparse representation, (2004), Proceedings of Int’l. Conf. Image Processing, pp. 75–78.
38.L.P. Morency, A. Rahimi, N. Checka, & T. Darrell, (2002), Fast stereo-based head tracking for interactive environments, in Proceedings of Int’l. Conf. Automatic Face and Gesture Recognition, pp. 375–380.
39.L.P. Morency, P. Sundberg, & T. Darrell, (2003), Pose estimation using 3d view-based eigenspaces, in Proceedings of IEEE Int’l. Workshop Analysis and Modeling of Faces and Gestures, pp. 45–52.
40.E. Murphy-Chutorian, & M.M. Trivedi, (2008), Head Pose Estimation in Computer Vision: A Survey, IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(4) , pp.607–626.
41.S. Niyogi, & W. Freeman, (1996), Example-based Head Tracking, Proceedings of Automatic Face and Gesture Recognition, pp. 374-378.
42.K. Oka, Y. Sato, Y. Nakanishi, & H. Koike, (2005), Head pose estimation system based on particle filtering with adaptive diffusion control, Proceedings of IAPR Conf. Machine Vision Applications, pp. 586–589.
43.R. Pappu and P. Beardsley, (1998), A qualitative approach to classifying gaze direction, Proceedings of IEEE Int’l. Conf. Automatic Face and Gesture Recognition, pp. 160–165.
44.R. Rae & H. Ritter, (1998), Recognition of human head orientation based on artificial neural networks, IEEE Trans. Neural Networks, 9(2),pp. 257–265.
45.B. Raytchev, I. Yoda, & K. Sakaue, (2004), Head pose estimation by nonlinear manifold learning, Proceedings of Int’l. Conf. Pattern Recognition, pp. 462–466.
46.S. Roweis & L. Saul, (2000), Nonlinear dimensionality reduction by locally linear embedding, Science, 290 (5500) , pp. 2323–2326.
47.H. Rowley, S. Baluja, & T. Kanade, (1998), Rotation invariant neural network-based face detection, Proceedings of IEEE Conf. Computer Vision and Pattern Recognition, pp. 38–44.
48.A. Sch¨odl, A. Haro, & I. Essa, (1998), Head tracking using a textured polygonal model, Proceedings of Workshop Perceptual User Interfaces.
49.E. Seemann, K. Nickel, & R. Stiefelhagen, (2004), Head pose estimation using stereo vision for human-robot interaction, Proceedings of IEEE Int’l. Conf. Automatic Face and Gesture Recognition, pp. 626–631.
50.O. Shay, & Ehud, R. (2006), Robust 3D Head Tracking Using Camera Pose estimation, Proceedings of International Conference in Pattern Recognition, pp. 1063–1066.
51.J. Sherrah, S. Gong, E.J. Ong, (2001), Face distributions in similarity space under varying head pose, Image and Vision Computing, 19(12), pp. 807–819.
52.J. Sherrah & S. Gong, (2001), Fusion of perceptual cues for robust tracking of head pose and position, Pattern Recognition, 34 (8) , pp. 1565–1572.
53.M. Soriano, , Martinkauppi, B., Huovinen, S. & Laaksonen, M. (2000), Using the Skin Locus to Cope with Changing Illumination Conditions in Color-Based Face Tracking, IEEE Nordic Signal Processing Symposium, pp. 383–386.
54.S. Srinivasan & K. Boyer, (2002), Head pose estimation using view based eigen spaces, Proceedings of Int’l. Conf. Pattern Recognition, pp. 302–305.
55.M. Soriano, , Martinkauppi, B., Huovinen, S. & Laaksonen, M. (2000), Skin Detection in Video under Changing Illumination Conditions, Proceedings of International Conference in Pattern Recognition, 1, pp. 839–842.
56.R. Stiefelhagen, J. Yang, and A. Waibel, (2002), Modeling focus of attention for meeting indexing based on multiple cues, IEEE Trans. Neural Networks, 13(4) , pp. 928–938.
57.V. Teodor, B. Maren,, & B. Sven, (2007) , Feature-based Head Pose Estimation from Images, Proceedings of the 7th IEEE-RAS International Conference on 2007 , pp. 330–335.
58.P. Viola, & M. J. Jones, (2001), Fast and Robust Classification Using Asymmetric AdaBoost and a Detector Cascade, Neural Information Processing Systems, 14, pp. 1311–1318.
59.P. Viola, & M. J. Jones, (2001), Rapid Object Detection Using a Boosted Cascade of Simple Features, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 511–518.
60.P. Viola, & M. J. Jones, (2004), Robust Real-Time Face Detection, International Journal of Computer Vision, 57(2), pp. 137–154.
61.J.G. Wang & E. Sung, (2007), EM enhancement of 3D head pose estimated by point at infinity, Image and Vision Computing, 25,(12) ,pp.1864–1874.
62.J. Wu & M. Trivedi, (2008), A two-stage head pose estimation framework and evaluation, Pattern Recognition, 41(3), pp. 1138–1158.
63.Y. Wu & K. Toyama, (2000), Wide-range, person- and illumination insensitive head orientation estimation, Proceedings of IEEE Int’l. Conf. Automatic Face and Gesture Recognition, pp. 183–188.
64.J. Xiao, T. Moriyama, T. Kanade, & J. Cohn, (2003), Robust full-motion recovery of head by dynamic templates and reregistration techniques, Int’l. J. Imaging Systems and Technology, 13(1) , pp. 85–94.
65.J. Xiao, S. Baker, I. Matthews, & T. Kanade, (2004), Real-time combined 2D+3D active appearance models, Proceedings of IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 535–542.
66.L. Xiaoguang, & A. K. Jain, (2006), Automatic Feature Extraction for Multiview 3D Face Recognition, Proceedings of the 7th IEEE International Conference on Automatic Face and Gesture Recognition, pp. 585–590.
67.S. Yan, Z. Zhang, Y. Fu, Y. Hu, J. Tu, , & T. Huang, (2007), Learning a person-independent representation for precise 3d pose estimation, Proceedings of Int’l. Workshop Classification of Events Activities and Relationships.
68.P. Yao, G. Evans, & A. Calway, (2001), Using affine correspondence to estimate 3-d facial pose, Proceedings of Int’l. Conf. Image Processing, pp. 919–922.
69.R. Yang & Z. Zhang, (2002), Model-based head pose tracking with stereovision, Proceedings of Int’l. Conf. Automatic Face and Gesture Recognition, pp. 242–247.
70.G. Zhao, L. Chen, J. Song, & G. Chen, (2007), “Large head movement tracking using SIFT-based registration,” Proceedings of Int’l. Conf. Multimedia, pp. 807–810.
71.T. Zhichao,& Q. Huabiao, (2005), Real-time Driver's Eye State Detection, Proceedings of IEEE International Conference on Vehicular Electronics and Safety, pp. 285–289.
72.J. Zhong, , Zhen, L., Jingyu, Y. & Quansen S. (2005), Face Detection Using Template Matching and Skin Color Information, International Conference on Intelligent Computing, 70(4), pp.636–645.