臺灣博碩士論文加值系統

受到神經元疾病（MND）影響的患者急切的需要一個系統，通過這個系統他們可以隨時透過他們僅能發出的眼球動作與護理人員進行交流。這篇論文提出了一種基於眼球動作的遠端溝通系統。通過對病床上的運動神經元病患進行眨眼偵測、注視方向的判別和頭部姿態的估計，來達成與護理人員進行簡單溝通的目的。此系統可以減少在休息時間護理人員對運動神經元病患進行無間斷檢查的需要，也可以使患者在有需要的時候能夠及時傳遞訊息並得到護理。

Patients affected by Motor Neuron Disease (MND) call for the need of a system for they to communicate with care providers any time they want with the final motion they can perform, the eye motion. This thesis proposes an eye-motion based communication system, to detect eye blink, gaze estimation and head pose estimation on MND patients on the bed and allow them to conduct simple communication with care providers. This eye-motion based communication system can reduce the need of constant checking on the MND patients by care providers during the bed time, and enable the patients to reach out and receive the care whenever they have the need.

Contents
Abstract I
摘要 II
Contents III
List of Figures V
List of Tables VI
Chapter 1 1
Introduction 1
1.1 Motor Neuron Disease 1
1.2 Background of Eye-Motion Based System 2
1.2.1 Eye-Gaze-Controlled Wheelchair System 2
1.2.2 The Eye Gaze Edge System 3
1.2.3 Wearable Eye-gaze System 4
1.2.4 ERICA Eye Gaze System 4
1.3 Overview of the Thesis 5
Chapter 2 6
Visual and Tracking System 6
2.1 Face Detection 6
2.1.1 Haar Feature-based Cascade Classifier 6
2.1.2 HD Face Detection of Kinect v2 SDK 7
2.1.3 Facial Landmarks Detection with Dlib 8
2.1.4 CLM-framework 8
2.2 Eye Center Localization 9
Chapter 3 10
Eye-Motion Based Communication System 10
3.1 Hardware 10
3.1.1 Infrared (IR) Camera 10
3.1.2 Face Tracking Mechanism 11
3.2 Mathematical Model 13
3.2.1 Blink Detection 13
3.2.2 Initialization Process of EAR 14
3.2.3 Head Pose Estimation 15
3.2.4 Perspective Transformation 17
3.2.5 Eye Center Localization 18
3.2.6 Gaze Estimation 19
3.3 The Finite-state Machine of the System 20
Chapter 4 22
Experiments and Results 22
4.1 Experiments 22
4.1.1 Blink Detection 23
4.1.2 Gaze Estimation 25
4.1.3 Tests Under Different Experimental Conditions are Carried Out by Combining Blink Detection and Gaze Estimation 26
4.1.4 Face Tracking Mechanism 27
4.1.5 IR Camera 29
4.2 Statistical Results 31
4.2.1 Blink Detection with Different Distances 31
4.2.2 Blink Detection with Different Angles 31
4.2.3 EAR Value Changes Under Different Distances when open and close eyes 33
4.2.4 Integration of Blink Detection and Gaze Estimation 36
4.2.5 The Error of Face Tracking System 37
Chapter 5 38
Conclusions 38
Chapter 6 39
Future Work 39
6.1 Eye Controlled Typing 39
6.2 Speech Synthesis 39
6.3 Human-Machine Interface 40
6.4 Embedded System 40
References 42

References
[1] C. J. McDermott and P. J. Shaw, “Diagnosis and Management of Motor Neurone Disease,” BMJ, vol. 336(7645), pp. 658-662, 2008.
[2] M. A. Eid, N. Giakoumidis and A. E. Saddik, “A Novel Eye-Gaze-Controlled Wheelchair System for Navigating Unknown Environments: Case Study with a Person with ALS,” IEEE Access, vol. 4, pp. 558-573, 2017.
[3] S. E. Lokhande and S. D. Dharkar, “Eye Gaze Technology,” IJCSA, vol. 6, 2013.
[4] C. M. Weaver, R. Martinez, C. A. Maier, C. Cerqueira and R. A. Foulds, “Design of a Wearable Eye-gaze Communication System for People with Severe Neuromuscular Impairment,” Proceedings of the IEEE 31st Annual Northeast Bioengineering Conference, pp. 26-27, 2005.
[5] D. Harris and M. Goren, “The ERICA Eye Gaze System Versus Manual Letter Board to Aid Communication in ALS/MND,” BJNN, vol. 5(5), pp. 227-230, 2009.
[6] P. Viola and M. Jones, “Rapid Object Detection using a Boosted Cascade of Simple Features,” CVPR, 2001.
[7] D. E. King, “Dlib-ml: A Machine Learning Toolkit,” JMLR, vol. 10, pp. 1755-1758, 2009.
[8] Z. H. Zhou and X. Geng, “Projection Functions for Eye Detection,” Pattern Recognition, vol. 37(5), pp. 1049-1056, 2004.
[9] Y. Zheng and Z. F. Wang, “Minial Neighborhood Mean Projection Function and Its Application to Eye Location,” J Softw, vol. 19(9), pp. 2322-2328, 2008.
[10] N. Cherabit, F. Z. Chelali and A. Djeradi, “Circular Hough Transform for Iris Localization,” Science and Technology, vol. 2(5), pp. 114-121, 2012.
[11] S. Z. Li and Z. Q. Zhang, “FloatBoost Learning and Statistical Face Detection,” IEEE Trans Pattern Anal Mach Intell, vol. 26(9), pp. 1112-1123, 2004.
[12] D. Cristinacce and T. Cootes, “Feature Detection and Tracking with Constrained Local Models,” BMVC, vol. 3, pp. 929-938, 2006.
[13] T. Baltrusaitis, P. Robinson and L. P. Morency, “Constrained Local Neural Fields for Robust Facial Landmark Detection in the Wild,” 2013 IEEE International Conference on Computer Vision Workshops, pp. 354-361, 2013.
[14] J. G. Daugman, “High Confidence Visual Recognition of Persons by a test of statistical independence,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 15(11), pp. 1148-1161, 1993.
[15] K. Nishino and S. K. Nayar, “Eyes for Relighting,” ACM Transactions on Graphics (TOG), vol. 23(3), pp. 704-711, 2003.
[16] F. Timm and E. Barth, “Accurate Eye Centre Localisation by Means of Gradients,” VISAPP, vol. 1, pp. 125-130, 2011.
[17] C. Y. Lin, L. T. Son, Y. L. Chang and Y. S. Shiue, “Image-Sensor-Based Fast Industrial-Robot Positioning System for Assembly Implementation,” Sensors and Materials, vol. 29(7), pp. 935-945, 2017.
[18] T. Soukupová and J. Čech, “Real-Time Eye Blink Detection using Facial Landmarks,” Computer Vison Winter Workshop, 2016.