在傳統瀏覽虛擬實境系統中，對於瀏覽環境的視覺資訊是由一些硬體設備來控制，例如滑鼠或是軌跡球。而這些設備是由使用者的手部來控制，這相當不方便與不自然。在本論文中，我們想要發展一套單眼視覺式人臉追蹤系統，將安置單部攝影機位於人臉之正前方，使之能夠拍攝人臉的運動情形。我們假設人臉上的雙眼、嘴巴在3D空間中形成一個平面，我們推導出一個數學運算來估測兩張不同的臉部影像，進而從分解此數學運算所得的轉換矩陣來得到人臉的運動參數。當獲得這些人臉運動參數後，我們將其傳送到虛擬實境瀏覽系統中達到瀏覽的效果。所以，使用者藉由頭部自然地運動,便可控制瀏覽畫面，而不需手去操控任何設備。
我們所使用的方法是，利用單台攝影機，架設在人臉的正前方並做好攝影機內部參數校正，接下來，拍攝人臉左右旋轉、上下擺動、前後移動之運動情況。我們首先選擇一張人臉參考影像(reference face image)，而這張人臉參考影像盡量是正對於攝影機的姿態(neutral pose)，之後，我們可以運用所推導的數學公式，對後續的人臉運動影像與此人臉參考影像做運算以估測出運動參數R3x3 (Rotation) T3x1 (Translation)，最後，將所估算出的運動參數R3x3與T3x1傳送到虛擬實境平台，完成整個畫面瀏覽動作。

In the conventional virtual reality navigation system, the viewing of the scene information is controlled by some peripherals, e.g. a mouse or a trackball. Users have to use their hands to control the device; this is quite inconvenient and unnatural. In this thesis, we will develop a facial tracking system using only one camera. We mount a calibrated TV camera in the front of the human face to properly capture the images of the human facial motions. We assume that the eyes and the mouth on the human face are coplanar in 3-D space, so a mathematical transformation can be derived to estimate the relationship between two different images of the same face. Furthermore, the facial motion parameter can be decomposed from the derived 3x3 transformation matrix. When the facial motion parameter are obtained, they can be transmitted to the VR navigation system to change the viewing specification. Hence, without using hands to control any input device, users can use head movement to browse the virtual world naturally.
In our methods, we use a single camera mounted in the front of the human face, and carry out camera intrinsic calibration in advance. Then any rotation (mainly in the left-right or the up-down direction) and any translation (mainly in the front-back direction) of the head movement can be captured by the camera. We first select a reference face image from the captured images; this reference face image is generally taken when the human face is in a neutral pose with the face looking toward the camera. After that, we can use the derived formulation to determine the motion parameters of the head rotation (R3x3) and translation (T3x1) by making use of the reference face image and the currently captured facial image. Finally, the motion parameters are transmitted to the VR platform to do the viewing control.

第一章 緒論 …………………………………………………………1
1.1 研究動機與目標……..……..…………………………………………..1
1.2 相關研究………………………………………………………………..2
1.3 研究方法介紹…………..……………………..………………………..3
1.4 論文組織………..…………………..………………….……………….4
第二章 電腦視覺定位技術……………………………………………..5
2.1 電腦視覺定位理論…………...…………………………………….…..5
2.2 攝影機參數調校之方法………...…………….………………………..6
2.3 攝影機內部參數矩陣的求取……………….………………………….8
2.3.1利用牛頓-拉弗森非線性疊代法求取攝影機內部參數矩陣……….8
2.3.2利用矩陣乘法關係求取攝影機內部參數矩陣…………….……...14
第三章 平面景物之投影轉換理論與參數估測……………………..17
3.1 平面景物之運動影像間的數學關係…………………………………17
3.2 影像座標系統平面投影轉換矩陣估測之非線性最佳化問題……....21
3.2.1 影像座標系統平面投影轉換矩陣(N matrix)估測…..………..…21
3.2.2 影像座標系統平面投影轉換矩陣(N matrix)的修正…………….22
第四章 平面景物運動參數之估測……………………..……………27
4.1 平面景物之參考平面方程式的估測…………………………………27
4.1.1 參考平面方程式法向量參數n3x3的估算…………………...…….27
4.1.2 參考平面方程式距離參數d的估算…………………………..….34
4.2 平面景物之旋轉矩陣R及平移矩陣T的估測………..………………36
4.3 平面景物間的運動軌跡…...………………………………………….38
第五章 實驗與結果…………………………………………………..39
5.1 實驗之軟硬體設備……………………………………………………39
5.2 攝影機參數校正與攝影機內部參數之實驗數據……………………40
5.3 實驗設計…………..…………………………………………………..41
5.3.1 估測及正確性評估………………………..…..41
5.3.2 引入L-M Method 的實驗結果…..………..………………………43
5.3.3 真實人臉的運動估測……...……………...…………………….56
第六章 結論與未來發展方向……………………………………..69
參考文獻………………………………………………………………..70

1. S. B. Kang, "Hands-Free Navigation in VR Environment by Tracking the Head," International Journal of Human-Computer Studies, pp247-266, 1998.
2. R. Szeliski, "Video Mosaics for Environments," IEEE Computer Graphics and Applications, pp.22-30, 1996.
3. K. Mase, "Human Reader : A Vision-Based Man-Machine Interface," Computer Vision for Human-Machine Interaction, Combridge University Press, Combridge, pp.53-82, 1998.
4. R. Sharma, V. I. Pavlovic and T. S. Huang, "Toward Multimodal Human-Computer Interface," Proceedings of the IEEE Vol. 86, No. 5 , May , pp.853-869, 1998.
5. A. Gee and R. Cipolla, "Fast Visual Tracking by Temporal Consensus," Image and Vision Computing, Vol.14, pp.105-114, 1996.
6. Q. Chen , H. Wu , T. Shioyama , T. Shimada and K. Chihara, "A Robust Algorithm for 3D Head Pose Estimation," Proceedings of the International Conference on Pattern Recognition, pp.1356-1359, 1998.
7. Q. Chen , H. Wu ,T. Shioyama and M. Yachida, ?D Head Pose Estimation without Feature Tracking," Proceedings of the International Conference on Pattern Recognition, pp.128-132, 1998.
8. R. Takamatsu, T. Taniguchi and M. Sato, "Space Browsing Interface Based on Head posture information," Proceedings of the 3rd Asia Pacific Conference on Computer Human Interaction, Proceedings, pp.298 -303, 1998.
9. M. J. Maron and R. J. Lopez, Numerical Analysis : a Practical Approach, Wadsworth Publishing Company, California,1994.
10. W. H. Press, S. A, Teukolsky, W. T. Vetterling and B. P. Flannery, Numerical Recipes in C, Combridge University Press, Combridge, 1992.
11. R. C. Gonzales and R. E. Woods, Digital Image Processing, Addison Wesley, New York, 1992.
12. Matrox Imaging Library and Command Reference, Version 5.1, Matrox Electronic Systems Ltd., February 10, 1998.
13. Matrox Imaging Library User Guide, Version 5.1, Matrox Electronic Systems Ltd., February 10, 1998.
14. 謝耀賢, "環場全景製作技術及其在虛擬實境應用", 國立交通大學, 資訊工程研究所碩士論文, 民國86年。
15. 柯廷潔, "利用虛擬實境技術進行個人電腦組裝訓練", 國立交通大學, 資訊工程研究所碩士論文, 民國87年。
16. C. Y. Tang, Y. P. Hung and Z. Chen, "Automatic Detection and Tracking of Human Heads Using an Active Stereo Vision Systems," Proceedings of the 3rd Asian Conference on Computer Vision, Vol. 1, pp.623-639, 1998.
17. T. Horprasert, Y. Yacoob and L. S. Davis, "Computing 3-D Head Orientation from a Monocular Image Sequence," Proceedings of the 2nd International Conference on Automatic Face-and Gesture-Recognition, Killington, Vermont, pp.242-247, 1996.
18. C. Maggioni and B. Kammerer, "GestureComputer-History,Design and Applications," Computer Vision for Human-Machine Interaction, Combridge University Press, Combridge , pp.23-52, 1998.
19. G. Burdea and P. Coiffect, Virtual Reality Technology, John Wiley & Sons, New York, N.Y., 1994
20. J. Gradecki, The Virtual Reality Construction Kit, John Wiley & Sons, New York, N.Y., 19 94.