以影像處理技術為基礎的駕駛者精神狀態偵測(Driver Mental Detection)在先進車輛控制及安全服務(Advanced Vehicle Control and Safety Services, AVCSS)中是一個很重要的課題。其主要的功能是當駕駛者有疲勞或是打瞌睡的情況時，會發出警告提醒駕駛者，避免駕駛者因為長時間駕駛引發之疲勞而導致意外。考量此類型之研究在運算效率與系統即時性的需求，本論文使用個人電腦架構(PC-Based)來實現此系統。
在變化的背景及光源條件下偵測駕駛者及其臉部特徵，並根據眼睛特徵判斷其生理狀態是本系統之主要功能。如何充分的運用個人電腦之系統資源，並使用有效率的影像處理演算法是本論文中最重要的研究工作。本論文的硬體平台使用個人電腦做為開發平台，需搭配的周邊設備有(1)USB Web Camera：影像擷取設備；(2)觸控式TFT-LCD：使用者操作與顯示之介面；(3)音源輸出喇叭：語音警告之輸出設備。本論文所使用之軟體包含Fedora Core 5、Linux中的Frame Buffer qvfb、Web Camera的讀取與控制用之Video For Linux Two(V4L2)函式庫、GUI介面設計用之MiniGUI的函式庫、語音播放用之XMMS應用程式。
在駕駛者疲勞偵測的部分，使用C語言撰寫多層次的影像處理演算法，包含(1)駕駛者偵測：找出畫面中駕駛者之臉部位置，並利用此位置對臉部特徵進行偵測；(2)特徵偵測：偵測駕駛者之眼睛特徵及嘴唇特徵，並判斷眼睛特徵之狀態；(3)疲勞偵測：利用眼睛特徵之狀態結合模糊邏輯對駕駛者進行疲勞偵測；(4)語音輸出：當駕駛者出現疲勞或打瞌睡的情形時，以語音提醒駕駛人。
本論文之駕駛者疲勞偵測系統經由測試，在駕駛者眼睛特徵判斷的部分其正確率可達百分之九十七，以此判斷的數據統計為基礎，能有效地提升本系統進行疲勞偵測時之準確性。由實驗結果顯示，本系統在不同光源條件及背景下之多位不同駕駛者均能偵測正確。並且能根據駕駛者之疲勞程度，即時地給予相對應之警示。

Driver Mental Detection which based on image processing technique is very important in Advanced Vehicle Control and Safety Services, AVCSS. When drivers are drowsy or doze off, the system will issue warnings to remind drivers in order to avoid accidents due to drowsiness. We consider the research of this type in demand of operational efficiency and real-time system, so we use personal computer to accomplish this system in this thesis.
This system is to detect driver’s facial features and then determine his physiological conditions based on eye feature in various backgrounds and light sources. How to utilize resource of personal computer and image processing algorithm efficiently is the most significant topic in this research work. In this thesis, we use personal computer as the development platform. Its ancillary equipments are USB Web Camera – image capturing equipment, touch screen TFT-LCD – user’s operating and displaying interface, and audio sandbox – audio warning output equipment. Then, the software includes Fedora Core 5, Frame Buffer qvfb of Linux, Video for Linux Two (V4L2) function library of Web Camera, MiniGUI function library for GUI interface design, and XMMS application for playing sounds.
In the aspect of driver’s drowsiness detection, we use C Language to write multilayer image processing algorithm. First step is driver detection. We would find out the face location of driver in the image. After finding out face location, we could detect features of eyes as well as mouth and determine the condition of eye feature. Third, we would combine the condition of eye feature with fuzzy logic to determine drowsiness. Final step is to play sound. When the driver is drowsy or dozes off, the system would warn the driver.
After testing, In the part of distinguishing driver’s eye feature, its rate of accuracy is 97%. Using these statistics as foundation, we could promote the system’s accuracy of detecting fatigue effectively. From the results, the system works quite well with different drivers in various light sources and backgrounds. It also issues proper warnings based on levels of driver’s drowsiness.

摘要 I
Abstract III
誌謝 V
章節目錄 VI
圖目錄 IX
表目錄 XIII
第一章 緒論 1
1.1 研究背景 1
1.2 研究動機與目的 4
1.3 系統功能 5
1.4 系統架構 7
1.4.1 硬體架構 7
1.4.2 軟體架構 8
1.4.3 開發工具 9
1.4.3.1 GNU Make 9
1.4.3.2 GNU 偵錯器(GNU Debugger, GDB) 9
1.4.3.3 GNU 編譯器套裝(Compiler Collection, GCC) 10
1.4.3.4 GNU Binutils 10
1.4.3.5 圖形化使用者介面(Graphical User Interface, GUI) 10
1.4.4 系統開發 12
1.5 論文貢獻 13
1.6 論文架構 14
第二章 相關研究 15
2.1 研究技術回顧 15
2.2 人臉偵測之文獻探討 16
2.2.1 經驗基礎( Knowledge-Based) 17
2.2.2 特徵基礎(Feature-Based) 19
2.2.2.1 皮膚顏色(Skin Color) 19
2.2.2.2 臉部特徵(Facial Features) 20
2.2.2.3 多樣特徵(Multiple Features) 21
2.2.2.4 人臉紋理(Texture) 23
2.2.3 樣板比對(Template Matching) 24
2.2.3.1 預設臉部樣板(Predefined Face Templates) 24
2.2.3.2 變形樣板(Deformable) 27
2.2.4 外觀基礎(Appearance-Based) 28
2.2.4.1 類神經網路(Neural Network) 29
2.2.4.2 支援向量機(Support Vector Machines, SVMs) 31
2.2.4.3 資訊理論方法(Information-Theoretical Approach) 33
2.3 疲勞偵測之文獻探討 35
第三章 疲勞偵測 41
3.1 膚色分割 42
3.1.1 色彩空間轉換 42
3.1.1.1 RGB色彩空間正規化(RGB Normalized) 43
3.1.1.2 HSV色彩空間(HSV Color Space) 43
3.1.1.3 HSI色彩空間(HSI Color Space) 45
3.1.1.4 YCbCr色彩空間(YCbCr Color Space) 47
3.1.2 擷取膚色區域 47
3.2 形態學影像處理 48
3.2.1 侵蝕運算(Erosion Operation) 49
3.2.2 膨脹運算(Dilation Operation) 51
3.3 人臉候選區域 54
3.3.1 連通成分標記(Connected Component Labeling) 54
3.3.2 可能之人臉區域 58
3.4 人臉區域確認 59
3.5 追蹤臉部特徵 62
3.5.1 嘴唇特徵擷取 62
3.5.1.1 嘴唇特徵之感興趣區域 62
3.5.1.2 嘴唇特徵位置標示 64
3.5.2 眼睛特徵擷取 67
3.5.2.1 眼睛特徵之感興趣區域 67
3.5.2.2 直方圖等化(Histogram equalization) 69
3.5.2.3 眼睛特徵位置標示與狀態判別 71
3.6 疲勞辨識 73
3.6.1 模糊邏輯(Fuzzy logic) 74
3.6.2 模糊化(Fuzzification) 75
3.6.3 知識庫與規則庫的建立 75
3.6.4 模糊推論(Fuzzy Inference) 79
3.6.5 解模糊化(Defuzzification) 80
第四章 實驗結果與分析 84
4.1 實驗結果 84
4.1.1 疲勞偵測實驗結果 84
4.1.1.1 光源條件變化下之偵測結果 84
4.1.1.2 背景變動時之偵測結果 86
4.1.1.3 不同距離下之偵測結果 87
4.1.2 連續影像序列之疲勞偵測實驗結果 88
4.1.3 膚色問題之錯誤提示 90
4.2 實驗結果分析 92
第五章 結論 94
5.1 結論 94
5.2 未來研究方向 95
參考文獻 97

[1] 交通部運輸研究所。
[2] National Highway Traffic Safety Administration (NHTSA). (April.7th,2009). [Online]. Available:
[3] Awake Consortium (IST 2000-28062), System for effective assessment of driver vigilance and warning according to traffic risk estimation (AWAKE), Sep. 2001–2004. [Online].Available: http://www.awake-eu.org
[4] Microwindows’s ScreenShot http://www.microwindow.org/SSPixil.html
[5] Qtopia’s ScreenShot http://www.qtopia.net/themes/Trolltech/screens.php
[6] 李彥鋒，以視覺為基礎之嵌入式車輛偵測系統，國立勤益科技大學電子研究所碩士論文，2009七月。
[7] W. W. Wierwille, S. S. Wreggit, C. L. Kirn, L. A. Ellsworth, and R. J. Fairbanks III, “Research on vehicle-based driver status/performance monitoring: development, validation, and refinement of algorithms for detection of driver drowsiness,” U.S. DOT Tech Report No. DOT HS 808 247, National Highway Traffic Safety Administration. 1994.
[8] Q. Ji, Z. Zhu, and P. Lan, “Real-time nonintrusive monitoring and prediction of driver fatigue,” IEEE Transactions on Vehicular Technology, vol.53, pp.1052-1069, 2004.
[9] C. T. Lin, R. C. Wu, S. F. Liang, W. H. Chao, Y. J. Chen, and T. P. Jung, “EEG-based drowsiness estimation for safety driving using independent component analysis,” IEEE Transaction on Circuits and Systems, vol.52, pp.2726-2738, 2005.
[10] S. K. L. Lal, A. Craig, P. Boord, L. Kirkup, and H. Nguyen, “Development of an algorithm for an EEG-based driver fatigue countermeasure,” Journal of Safety Research, vol.34, pp.321-328, 2003.
[11] M. H. Yang, D. J. Kriegman and N. Ahuja, “Detecting Faces in Images: A Survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.24, pp.34-58, 2002.
[12] J. Miao, B. Yin, K. Wang, L. Shen and X. Chen, “A hierarchical multiscale and multiangle system for human face detection in a complex background using gravity-center template,” Pattern Recognition, vol. 32, no. 7, pp. 1237-1248, 1999.
[13] J. Yang and A. Waibel, “A Real-Time Face Tracker,” Proc. Third Workshop Applications of Computer Vision, pp. 142-147, 1996.
[14] R. L. Hsu, M. M. Abdel, A. K. Jain, “Face Detection in Color images,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, Issue 5, pp. 696 – 706, May, 2002.
[15] R. Kjeldsen and J. Kender, “Finding Skin in Color Images,” Proc. Second Int’l Conf. Automatic Face and Gesture Recognition, pp. 312-317, 1996.
[16] Y. Dai and Y. Nakano, “Face-Texture Model Based on SGLD and Its Application in Face Detection in a Color Scene,” Pattern Recognition, vol. 29, no. 6, pp. 1007-1017, 1996.
[17] Y. Yokoo and M. Hagiwara, “Human Faces Detection Method using Genetic Algorithm,” Proceedings of IEEE International Conference, on Evolutionary Computation, pp. 113-118, 1996.
[18] R. C. Gonzalez and R. E. Woods, Digital Image Processing 3/e, Prentice-Hall, Inc., 2009.
[19] A. Lanitis, C.J. Taylor, and T.F. Cootes, “An Automatic Face Identification System Using Flexible Appearance Models,” Image and Vision Computing, vol. 13, no. 5, pp. 393-401, 1995.
[20] H. Rowley, S. Baluja, and T. Kanade, “Neural Network-Based Face Detection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 1, pp. 23-38, Jan. 1998.
[21] Wikipedia Encyclopedia. “Support vector machine, the free encyclopedia,” Jul. 2008, from http://en.wikipedia.org/wiki/Support_vector_machine.
[22] E. Osuna, R. Freund, and F. Girosi, “Training Support Vector Machines: An Application to Face Detection,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 130-136, 1997.
[23] M.S. Lew, “Information Theoretic View-Based and Modular Face Detection,” Proc. Second Int’l Conf. Automatic Face and Gesture Recognition, pp. 198-203, 1996.
[24] P. Smith, M. Shah and N.d.V. Lobo, “Determining driver visual attention with one camera,” IEEE Transactions on Intelligent Transportation Systems, vol. 4, no. 4, pp. 205-218, Dec. 2003.
[25] L. Yunqi, Y. Meiling, S. Xiaobing, L. Xiuxia and O. Jiangfan, “Recognition of Eye States in Real Time Video,” Proc. IEEE Conf on Computer Engineering and Technology, pp. 554-559, Feb. 2009.
[26] Wikipedia Encyclopedia. “HSV color space (hue, saturation, value),” March 2010, from http://en.wikipedia.org/wiki/HSV.
[27] Design Studio. “HSI color space (hue, saturation, intensity),Nov. 2009, from http://nomadlibra.blogspot.com/2009_11_01_archive.html.
[28] P. Viola and M. Jones, “Rapid Object Dectetion Using a Boosted Cascade of Simple Features,” Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 2001.
[29] 王文俊, “認識Fuzzy-第三版,” 全華圖書, June 2007.
[30] 王進德, “類神經網路與模糊控制理論入門與應用,” 全華圖書, April 2008.
[31] P. Caffier, U. Erdmann and P. Ullsperger, “Experimental evaluation of eye-blink parameters as a drowsiness measure,” Eur J Appl Physiol.vol. 83, no. 1, pp.319-325 May. 2003.
[32] P. Simonov and M. Frolov, “Blink reflex as a parameter of human operator's functional state,” Aviat Space Environ Med. vol. 56, no. 8, pp.783-785, Aug. 1985.
[33] J.Stern, D. Boyer and D. Schroeder, “Blink rate: a possible measure of fatigue,” Hum Factors. vol. 36, no. 2, pp285-297, Jun. 1994.