臺灣博碩士論文加值系統

本論文提出幾種應用於閉眼警示之眼睛狀態偵測演算法系統的設計，這些系統將追蹤眼睛之狀態(張眼與閉眼)，並且以所追蹤到的眼睛狀態判斷是否發出警訊，如果偵測到的狀態為閉眼，系統能即時發出警告訊息。此系統可應用於拍照時的閉眼偵測，當受測者閉眼時系統能即時告知。

因此系統可分為兩個子系統:眼睛偵測及眼睛狀態判斷。一開始是由所取得之受測者影像進行色彩空間轉換、平均濾波器做平滑濾波、Sobel濾波器做邊緣偵測、自商影像作光源變化濾除等影像前處理，處理完後的影像利用不同大小的樣板進行眼睛特徵點偵測，找到眼睛區域後，接著使用以多重閥值的Cr偵測膚色、以正規化RGB 色彩為基礎偵測膚色及設定搜尋視窗大小在灰階影像尋找瞳孔、ㄇ字型濾波器找出瞳孔特徵的方式進行眼睛區域膚色或灰階值的變化，藉由動態影像中第一張畫面與現在影像畫面眼睛區域的色彩變化，來偵測目前眼睛的狀況。。

實驗結果顯示，使用四核心的個人電腦（2.66GHz）去測試演算法偵測狀態，在第一張畫面為張眼狀態的前提下，在正面拍攝及仰角拍攝受測者皆未戴眼鏡的情況下使用灰階影像搜尋視窗偵測眼睛狀態方法的正確率分別為96%及94%，其正確率較其他三種方法正確率高，而在受測者戴粗框或細框眼鏡的情況下使用十字節濾波器偵測眼睛狀態的正確率分別為比較82%及72%，其正確率為四種方法中最高的。

In this thesis, we discuss several methods of eye status detection for close eye alert systems. The proposed close eye alert system based on tracking eye states was implemented. If the system detects the close eye state, it will show a warning message to alert.

The proposed system contains two parts: the eye tracking and the eye state detection. Firstly, we pre-process original facial images for future easier processing due to some noises in images. We use several methods to do the pre-process, which are described as follows: 1. Convert the color space to the YCbCr format, 2. Use the mean filter to smooth the images, 3. The Sobel filter is used to get the edges of images, 4.Use “Self Quotient Image” algorithm to remove influences of different light conditions, and then we use different sizes of templates to detect the possible candidate area of eyes. Next, we propose four methods to detect the eye states, which are described as follows: 1. Use the skin color base on the threshold of the Cr color, 2. Use the skin color base on the normalized RGB pixels, 3. Use the search window to find out the minimum gray-level value, 4. Use the cross filter to find out the minimum gray-level value. Thus, we can get the close eye states by comparing the color variations of the first image with that of the present image.

In our experiments, we use a personal computer with 2.66GHz Quad-core CPUs for simulations. There are up to 15 video clips in the video database for four different individuals, which includes: the clips for frontal view without glasses, the clips with frontal view and wearing thin rim glasses, the clips for frontal view and black frame glasses, and the clips with upward view without glasses.

The experimental results show that our methods can detect the eye states in various situations, such as wearing glasses. Under the premise that eyes are open at the first frame, compared with the other methods, the proposed method by using the search window has better detection accuracy for the eye state detection on non-glasses situations, and the proposed method by using the cross filter has better detection accuracy for the eye state detection on glasses situations.

致謝 i
論文摘要 ii
Abstract iii
目錄 iv
第一章 緒論 1
1.1研究動機與目的 1
1.2 相關研究 1
1.2.1眼睛特徵點擷取 1
1.2.2眼睛狀態偵測 2
1.3系統概述 3
1.4 論文架構 3
第二章 先前相關研究 4
2.1 眼睛特徵點偵測 4
2.1.1 眼睛特徵點偵測流程 4
2.1.2 快速自商影像(Fast Self Quotient Image) 4
2.1.3 降取樣(Downscale) 4
2.1.4 眼睛濾波器(Eye filter) 5
2.1.5 辨別眼睛[11] 6
2.1.6 偵測出眼睛特徵點 7
2.2眼睛狀態偵測 7
2.2.1訓練分類器的方法 7
2.2.2平均密度變化法 8
2.2.3百分比法 8
2.2.4角度法 8
第三章 我們的方法 9
3.1使用膚色偵測的方法偵測眼睛狀態 9
3.1.1 方法一:以多重閥值的Cr偵測膚色 10
3.1.1.1眼睛特徵點偵測流程 10
3.1.1.2色彩空間轉換(RGB 轉 YCbCr) 10
3.1.1.3平滑影像(Smooth image) 11
3.1.1.4邊緣偵測(Edge detection) 13
3.1.1.5 T字節濾波器(T filter) 14
3.1.1.6取出眼睛區域 17
3.1.1.7偵測出眼睛特徵點 17
3.1.1.8眼睛修正(Eye Correction) 20
3.1.1.9眼睛追蹤 21
3.1.1.10眼睛狀態偵測 23
3.1.1.10.1多重閥值Cr膚色偵測 24
3.1.1.10.2眼睛狀態偵測流程 25
3.1.1.10.3 計算眼睛區域膚色像素數量 26
3.1.1.10.4 眼睛狀態偵測演算法 26
3.1.1.11眼睛狀態偵測結果 27
3.1.2 方法二:以正規化 RGB 色彩為基礎偵測膚色 28
3.1.2.1眼睛特徵點偵測流程 28
3.1.2.2色彩空間轉換(RGB 轉 YCbCr 28
3.1.2.3平滑影像(Smooth image) 28
3.1.2.4 邊緣偵測(Edge detection) 28
3.1.2.5 T字節濾波器(T filter) 28
3.1.2.6取出眼睛區域 28
3.1.2.7偵測出眼睛特徵點 28
3.1.2.8眼睛修正 29
3.1.2.9眼睛追蹤 29
3.1.2.10眼睛狀態偵測 29
3.1.2.10.1眼睛狀態偵測流程 29
3.1.2.10.2正規化RGB(Normalized RGB)偵測膚色[28][36] 30
3.1.2.10.3計算眼睛區域膚色像素數量 31
3.1.2.10.4 眼睛狀態偵測演算法 31
3.1.2.11眼睛狀態偵測結果 31
3.2使用灰階影像偵測眼睛狀態 33
3.2.1方法三:使用搜尋視窗找出瞳孔灰階值 33
3.2.1.1眼睛特徵點偵測流程 33
3.2.2.2色彩空間轉換(RGB 轉 YCbCr) 33
3.2.1.3平滑影像(Smooth image) 33
3.2.1.4自商影像(Self Quotient Image) 34
3.2.1.5 邊緣偵測(Edge detection) 36
3.2.1.6 T字節濾波器 37
3.2.1.7 取出眼睛的區域 39
3.2.1.8 偵測出眼睛特徵點 40
3.2.1.9 眼睛修正 42
3.2.1.10眼睛狀態偵測 42
3.2.1.10.1眼睛狀態偵測的流程 43
3.2.1.10.2 計算眼睛區域搜尋視窗的灰階值 44
3.2.1.10.3眼睛狀態偵測演算法 45
3.2.1.11 眼睛狀態偵測結果 46
3.2.2方法四:使用十字節濾波器找出瞳孔特徵 47
3.2.2.1眼睛特徵點偵測流程 47
3.1.2.2色彩空間轉換(RGB 轉 YCbCr) 48
3.2.2.3平滑影像(Smooth image) 48
3.2.2.4自商影像(Self Quotient Image) 48
3.2.2.5邊緣偵測(Edge detection) 48
3.2.2.6ㄇ字型濾波器 48
3.2.2.7取出眼睛的區域 51
3.2.2.8偵測出眼睛特徵點 51
3.2.2.9眼睛修正 51
3.2.2.10影像直方圖等化 51
3.2.2.11眼睛狀態偵測 54
3.2.2.11.1眼睛狀態偵測流程圖 54
3.2.2.11.2濾出瞳孔特徵點 54
3.2.2.11.3眼睛狀態偵測演算法 56
3.2.2.12眼睛狀態偵測結果 57
第四章模擬與比較 58
4.1實驗環境 58
4.2實驗結果 60
4.2.1使用膚色判斷眼睛狀態偵測的結果 60
4.2.2 使用灰階判斷眼睛狀態偵測的結果 61
4.3 實驗結果比較 61
第五章結論與未來工作 63
5.1結論 63
5.2未來研究方向 63
參考文獻 64

[1] M. Abdel-Mottaleb, Rein-Lien Hsu, A. K. Jain, “ Face Detection In Color Images,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, pp.696-706,2002.
[2] T. D. Orazio, M. Leo, C. Guaragnella, and A. Distante, "A visual approach for driver inattention detection," Pattern Recognition, vol. 40, pp. 2341-2355, 2007.
[3]T. Zhichao and Q. Huabiao, "Real-time driver's eye state detection, " IEEE International Conference on Vehicular Electronics and Safety, pp.285-289,
2005.
[4] J. W. Wang and W. Y. Chen, "Eye detection based on head contour
geometry and wavelet subband projection," Optical Engineering, vol. 45.
057001-1-057001-12, 2006.
[5] Z. Zhiwei and J. Qiang, "Robust real-time eye detection and tracking under variable lighting conditions and various face orientations," Computer Vision and Image Understanding, vol. 98, pp. 124-154, 2005.
[6] J. Qiang, Z. Zhiwei, and P. Lan, "Real-time nonintrusive monitoring and prediction of driver fatigue," IEEE Transaction on Vehicular Technology, vol.53, pp. 1052-1068, 2004.
[7] W.B. Horng and C.Y. Chen, "A Real-Time Driver Fatigue Detection System Based on Eye Tracking and Dynamic Template Matching," Tamkang Journal of Science and Engineering, vol. 11, pp. 65-72, 2008.
[8] J. Wu and M. M. Trivedi, "Simultaneous eye tracking and blink detection with interactive particle filters," EURASIP Journal on Advance in Signal Process, vol. 2008, pp. 1-17, 2008.
[9] J. H. Hu, "Design and Implement of Facial Features Detection and Facial Expression Recognition Algorithm for Baby Watch and Care System", National Chung Hsing University master thesis, Taichung, Taiwan, 2009.
[10] J. H. Hu, “Design and Embedded System Implementation of Colorless Foreign Object Detection Algorithm and Facial Expression Recognition for Baby Watch and Care System”, National Chung Hsing University master thesis, Taichung, Taiwan, 2011.
[11] 范志鵬,游聖民，”應用於嬰幼兒監護系統之臉部表情辨識與異物偵測演算法設計與實作”, 國立中興大學電機工程學系碩士論文, 2010年.
[12] Gang Pan, L. Sun, Z.H. Wu, S.H. Lao “Eyeblink-based Anti-spoofing in Face Recognition from a Generic Webcamera,” International Conference on Computer Vision - ICCV , pp. 1-8, 2007
[13]T. Moriyama, T. Kanade, J. F. Cohn, J. Xiao, Z. Ambadar, J. Gao, H. Imamura, “Automatic Recognition of Eye Blinking in Spontaneously Occurring Behavior,” International Conference on Methods and Techniques in Behavioral Research, vol. 35, pp. 420-428, 2003
[14]1994年Nissan 車輛研究實驗室
[15] I. Park, J. H. Ahn, and H. Byun, “Efficient Measurement of Eye Blinking under Various Illumination Conditions for Drowsiness Detection Systems”, 18th International Conference on Pattern Recognition, pp383 - 386,2006.
[16] Z. Xin, X. Yanjun, and D. Limin, “Locating facial features with color information”, Fourth International Conference on Signal Processing Proceedings, vol. 2, pp. 889-892, 1998.
[17] J. Xiong, Z. Jiang, J. Liu, and H. Feng, "Multiple states and joint objects particle filter for eye tracking," in Automatic Target Recognition and Image Analysis; and Multispectral Image Acquisition, 2007.
[18]張家銓，”以追蹤眼睛狀態為基礎之駕駛者疲勞偵測系統”，國立成功大學工程科學系碩士論文,2008年.
[19] 范志鵬,余學翰，”應用於駕駛疲勞警示系統之臉部與頭部狀態偵測技術”, 國立中興大學電機工程學系碩士論文, 2013年.
[20] L. Gan, B. Cui and W. Wang “Driver Fatigue Detection Based on Eye Tracking”, Proceedings of the 6th World Congress on Intelligent Control and Automation, June 21 - 23, 2006, Dalian, China.
[21]Y. Guan, “Robust Eye Detection from Facial Image based on Multi-cue Facial Information”, IEEE International Conference on Control and Automation, pp. 1775 - 1778, 2007.
[22]P. Viola and M. J. Jones, ”Robust Real-Time Face Detection”, International Journal of Computer Vision, vol. 57(2), pp.137–154, 2004.
[23] P. R. Tabrizi and R. A. Zoroofi “Open/Closed Eye Analysis for Drowsiness Detection”, IPTA, 2008
[24] Y. Tian,T. Kanade and J. F. Cohn,” Dual State Parametric Eye Tracking”, Fourth IEEE International Conference,pp.110-115,2000.
[25] J. Park, J. Seo, D. An, S. Chung, “Detection of Human Faces Using Skin Color and Eyes,” IEEE International Conference on Multimedia and Expo, pp. 133-136, 2000.
[26] L. Hong, W. Yuwen, and Z. Hongbin, "Eye state detection from color facial image sequence," Second International Conference on Image and Graphics, pp. 693-698, 2002.
[27] P. Viola and M. J. Jones, “Rapid Object Detection using a Boosted Cascade of Simple Features,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp.511-518, 2001
[28] Soriano, M., B. Martinkauppi, S. Huovinen, and M. Laaksonen, “Using
the skin locus to cope with changing illumination conditions in color-based face tracking,” in Proc. IEEE Nordic Signal Proc. Symp.,Kolmarden, Sweden, pp.383-386, July 13-15, 2000.
[28] Hsu, R.-L., M. A. Mottaleb, and A. K. Jain, “Face detection in color
Images ,” IEEE Trans. Pattern Anal. Mach. Intell., vol.24, no.5,pp.696-706, 2002.
[29] Berbar, M. A., H. M. Kelash, and A. A. Kandeel, “Faces and facial features
detection in color images,” Geometric Modeling and Imaging -New Trends,
pp. 209-214, July 2006.
[30] X. Cui, Z. Ying, and W. Zengfu, "Efficient eye states detection in real-time for drowsy driving monitoring system," International Conference on Information and Automation, pp. 170-174 , 2008.
[31] Horng, W.-B., C.-Y. Chen, Y. Chang, and C.-H. Fan, “Driver fatigue
detection based on eye tracking and dynamic template matching,” in Proc.
IEEE Intl. Conf. on Networking, Sensing and Cont., Taipei,Taiwan, pp.7-12, Mar. 21-23, 2004.
[32] R. C. Gonzalez and R. E. Woods, Digital Image Processing, 2nd ed. New Jersey:Prentice Hall, 2002.
[33] T. Huachun and Y.-J. Zhang, "Detecting eye blink states by tracking iris and
eyelids," Pattern Recognition Letters, vol. 27, pp. 667-675, 2006.
[34] T. Zhichao and Q. Huabiao, "Real-time driver's eye state detection," in IEEE International Conference on Vehicular Electronics and Safety, pp. 285-289, 2005.
[35] 范志鵬,林哲立，” 應用臉部狀態偵測之臥床病人視訊監護系統設計與嵌入式系統實現”, 國立中興大學電機工程學系碩士論文, 2012年.
[36] Siana, L., A Study of Human Tracking and Face Detection on A
Pantilt-zoom Camera, Inst. of Electrical and Control Engineering
National Chiao-Tung Univ., Master Thesis, 2005.
[37] L.R.Rabiner, A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE, pp.257-286, 1989.