臺灣博碩士論文加值系統

摘要
在現今的智慧車輛系統中，駕駛專注力監控是先進駕駛輔助系統(Advanced Driver Assistance Systems, ADAS)設計之重要配備，此項技術可以藉由對駕駛的臉部追蹤來決定其目前是否處於分心狀態。標準的感測過程包含了複雜的特徵擷取以及圖形辨識程序，因此經常需要仰賴體積小，高性能，以及低功耗之運算系統，方能有效地實現在狹小的車內空間。而為了克服傳統電腦的高功耗及耐震度不佳等缺點，本研究專注在如何將駕駛視線感測演算法實現在單晶片上。
在此，個人利用模糊語意影像描述(Semantics-based Vague Image Representation, SVIR)搭配全連接類神經網路(Fully Connection Neural network, FCN)，有效的在單一現場可程式邏輯閘陣列(Field Programmable Gate Array, FPGA)晶片上以最精簡之硬體電路達到即時及高效率臉部追蹤，同時對眼睛注視方向進行分類，並利用分離度來驗證系統對不同注視方向眼睛之辨別程度。另外，由於本研究僅使用簡單邏輯元件以及少量記憶體，且不需使用額外DSP資源來輔助運算，因此特別適用於小型嵌入式系統(Embedded system)中。
根據本研究之實驗結果，在使用640*480影像解析度以及每秒80張畫面情形下，本研究僅需0.52 us即可完成一次臉部追蹤。此外，由分離度分析結果中顯示，在採用門檻值為70%情況下，與其它不同注視方向之眼睛皆至少保持20%以上之分離度，由此可知，本研究能有效地降低偵測錯誤率。最後，由於本研究之硬體設備只使用單一小型數位相機搭配FPGA運算，加上演算法也僅使用簡單的邏輯規則以及基本的加乘法器來運算，因此其體積與功耗程度皆特別適用於未來先進駕駛輔助系統。

關鍵字: 人眼監測、模糊語意影像特徵表示法、全連接類神經網路、即時處理系統、先進駕駛輔助系統

Abstract
For a smart vehicle system design, driver’s attention monitoring system is essential to advanced driver assistance system (ADAS). Such technology can be achieved by using face tracking to detect distraction of driver. The computing process involves complicated feature extraction and pattern recognition so that design concepts of small dimension, high computing performance, and low power consumption are required in order to be implement in a vehicle. For this, this study focuses on the way to realize an efficient driving eyes tracking algorithm on a single chip.
In this research, the algorithm of Semantics-based Vague Image Representation(SVIR) with a Fully Connection Neural network(FCN) are implemented on a single Field Programmable Gate Array(FPGA) chip to track driver’s eye direction in real-time with economical hardware resources usage. The reliability for eyes direction detection can be verified by a high separation from FCN. Furthermore, the proposed system also avoids the use of DSP with low logic elements and memory usages. Therefore, it is especially suitable for small embedded systems.
According to the experimental results of this study, with 640*480 image resolution and 80 image frames per second, it only consumes 0.52 us to finish eyes tracking. In addition, high reliability can be found from 20% separation to the other eyes directions with 70% similarity threshold at the output of FCN. These saliences demonstrate the feasibility for advanced driver assistance systems in the future.

Keywords: Human Eyes Monitoring, Semantics-based Vague Image Representation, Fully Connected Neural Network, Real-time Processing System, Advanced Driver Assistance System

目錄
摘要 i
Abstract ii
致謝 iii
目錄 iv
圖目錄 vi
表目錄 viii
縮寫說明 ix
第一章 緒論 1
1.1 研究動機 1
1.2 研究目的 1
1.3 研究方法 2
1.4 論文貢獻 4
1.5 文獻探討 4
1.6 論文架構 7
第二章 基於模糊語意影像特徵表達法擷取影像特徵 8
2.1 前言 8
2.2 膚色偵測 12
2.2.1 膚色偵測 12
2.2.2 形態學(Morphology) 15
2.3 模糊語意影像特徵表達法 17
2.3.1 雙極性編碼 17
2.3.2 垂直重疊演化 19
2.3.3 反向編碼 22
2.3.4 中空編碼 24
2.4 實驗結果與討論 26
2.5 結論 32
第三章 基於全連接類神經網路分類法辨識眼睛注視方向 33
3.1 前言 33
3.2 全連接類神經網路(FCN)分類法 36
3.2.1 FCN訓練方法 36
3.2.2 FCN晶片架構 40
3.3 基於人眼特徵與分類結果之注視方向辨識規則 43
3.4 實驗結果與討論 46
3.5 結論 53
第四章 總結與未來展望 54
4.1 前言 54
4.2 論文貢獻回顧 54
4.2.1 即時處理 55
4.2.2 高解析度及高畫面張數 55
4.2.3 低體積 56
4.3 未來展望 56
4.4 結論 56
參考文獻 58

Abuzneid M. and Mahmood A., "Performance improvement for 2-D face recognition using multi-classifier and BPN", IEEE Long Island Systems, Applications and Technology Conference (LISAT), Farmingdale, NY, USA, 2016.

Ai M., Zhao L., Wang J., "Region based inter-color intra prediction for RGB signals", Int. Conf. on Computer Science and Information Technology(CSII), Chengdu, China, pp. 41-44,2010.

Asundi, P. K. and Geetha, K. S., "Raindrop detection algorithm for ADAS", IEEE Int. Conf. on Recent Trends in Electronics, Information & Communication Technology (RTEICT), Bangalore, India, pp. 1646-1650, 2017.

Bougharriou S., Hamdaoui F. and Mtibaa A., "Linear SVM classifier based HOG car detection", Int. Conf. on Sciences and Techniques of Automatic Control and Computer Engineering (STACCE), Monastir, Tunisia, pp. 241-245, 2017.

Choi I. H., Hong S. K. and Kim Y. G., "Real-time categorization of driver's gaze zone using the deep learning techniques", Int. Conf. on Big Data and Smart Computing (BigComp), Hong Kong, China, pp. 143-148, 2016.

Devadethan S., Titus G., and Purushothaman S., “Face detection and facial feature extraction based on a fusion of knowledge based method and morphological image processing”, Annual International Conference on Emerging Research Areas: Magnetics, Machines and Drives (AICERA/iCMMD), Kottayam, India, 2014.
Ding Z., Zhao F., Wang T., Shu W., and Wu M., “Hecto-Scale Frame Rate Face Detection System for SVGA Source on FPGA Board”, IEEE 19th Annual International Symposium on Field-Programmable Custom Computing Machines, Salt Lake City, UT, USA, pp. 37-40, 2011.

Du X., Ang M. H. and Rus D., "Car detection for autonomous vehicle: LIDAR and vision fusion approach through deep learning framework", IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada, pp. 749-754, 2017.

Fersch T., Buhmann A., Koelpin A. and Weigel R., "The influence of rain on small aperture LiDAR sensors", German Microwave Conference (GeMiC), Bochum, Germany, pp. 84-87, 2016.

Fu Y., Li J., and Xiang R., “Robust Eye Localization on Multi-View Face in Complex Background Based on SVM Algorithm”, Int. Symposium on Information Engineering and Electronic Commerce, Ternopil, Ukraine, 2010.

Guo K., Sui L.,Qiu J., Yu J., Wang J., Yao S., Han S., and Wang Y., “Angel-Eye: A Complete Design Flow for Mapping CNN Onto Embedded FPGA”, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 37,no. 1, pp. 35-47, 2018.

Hao G., Min L. and Feng H., "Improved Self-Adaptive Edge Detection Method Based on Canny", Int. Conf. on Intelligent Human-Machine Systems and Cybernetics (IHMSC), Hangzhou, China, vol. 2, pp. 527-530, 2015.

Huang H. and Guo J., "Application of multiple order morphology transformation on edge detection for liver CT image", IEEE 19th Int. Conf. Information Technology and Artificial Intelligence Conference (ITAIC), Chongqing, China, pp. 107-110, 2014.

Jaswal G., Kaul A., Nath R. and Nigam A., "Finger knuckle image ROI extraction using watershed transformation for person recognition", Int. Conf. on Image Information Processing (ICIIP), Shimla, India, pp. 137-142, 2017.

Kumar A. and Malhotra S., "Real-time human skin color detection algorithm using skin color map", Int. Conf. on Computing for Sustainable Global Development (INDIACom), New Delhi, India, pp. 2002-2006, 2015.

Kocić O., Simić A., Bjelica M. Z. and Maruna T., “Optimization of driver monitoring ADAS algorithm for heterogeneous platform”, Telecommunications Forum (TELFOR), Belgrade, Serbia, 2016.

Kovács L. and Terstyánszky G. Z., “Diagnosising faults by supervised and unsupervised learning”, European Control Conference (ECC), Karlsruhe, Germany, pp. 4238-4242, 1999.

Lee M. L., Howard M. E., Horrey W. J., Liang Y., Anderson C., Shreeve M. S., O’Brien C. S. and Czeisler C. A., "High risk of near-crash driving events following night-shift work", Proceedings of the National Academy of Sciences(PNAS), vol. 113, no. 1, pp. 176-181, 2015.


Lee S., Kwak. Y., Kim Y. J., Park S. and Kim J., “Contrast-preserved chroma enhancement technique using YCbCr color space”, IEEE Transactions on Consumer Electronics(TCE), vol. 58, no. 2, pp. 641-645, 2012.

Li G., Huang Z., Xia J., Lu C., “Mining human mitochondrial and mitochondrial associated proteins based on SVM and neural network”, IEEE Int. Conf. on Granular Computing, Hangzhou, China, 2008

Liang H. and Zhang H., “Identification of slope stability based on the contrast of BP neural network and SVM”, Int. Conf. on Computer Science and Information Technology(CSIT), Chengdu, China, pp. 347-350, 2010.

Liu X. and Che M., "A parallel architecture of AdaBoost-based face detection for gaze estimation", Int. Conf. on Multimedia Technology, Hangzhou, China, pp. 2888-2891, 2011.

Mavely A. G., Judith J. E., Sahal P. A. and Kuruvilla S. A., "Eye gaze tracking based driver monitoring system", IEEE Int. Conf. on Circuits and Systems (ICCS), Thiruvananthapuram, India, pp. 364-367, 2017.

Mima H., Ikeda K., Shibata T., Fukaya N., Hitomi K. and Bando T., "A rear-end collision warning system for drivers with support vector machines", IEEE/SP 15th Workshop on Statistical Signal Processing, Cardiff, UK, pp. 650-653, 2009.

Na S., Xumin L., and Yong G., “Research on k-means Clustering Algorithm: An Improved k-means Clustering Algorithm”, Int. Symposium on Intelligent Information Technology and Security Informatics, Jinggangshan, China, pp. 63-67, 2010.

Qi Z. and Zhang J., “Design of fuzzy SVM multi-category classifier model and application in engine fault diagnosis”, Int. Congress on Image and Signal Processing, Chongqing, China, pp. 1782-1785,2012

Qin H., Xu X., Hu Z. and Zhang D., "Eye tracking system based on SOPC", IEEE Int. Conf. on Electronics, Circuits and Systems (ICECS), Marseille, France, pp. 171-174, 2014

Rezaei M. and Klette R., "3D Cascade of Classifiers for Open and Closed Eye Detection in Driver Distraction Monitoring", Computer Analysis of Images and Patterns (CAIP), Berlin, pp. 171-179, 2011.

Ryu J. B., Yang H. S. and Seo Y., “Real Time Eye Blinking Detection Using Local Ternary Pattern and SVM”, Int. Conf. on Broadband and Wireless Computing, Communication and Applications, Compiegne, France, pp. 598-601, 2013.

Sandnes F. E., Neyse L. and Huang Y. P., "Simple and practical skin detection with static RGB-color lookup tables: A visualization-based study", IEEE Int. Conf. on Systems, Man, and Cybernetics (SMC), Budapest, Hungary, pp. 2370-2375, 2016.

Shanmugavadivu P. and Kumar A., 2016. "Rapid face detection and annotation with loosely face geometry", Int. Conf. on Contemporary Computing and Informatics (IC3I), Noida, India, pp. 594-597, 2016.

Sharkas M., “A neural network based approach for iris recognition based on both eyes”, SAI Computing Conference (SAI), London, UK, pp. 253-258,2016

Sieber M. and Farber B., "Driver perception and reaction in collision avoidance: Implications for ADAS development and testing", IEEE Intelligent Vehicles Symposium (IV), Gothenburg, Sweden, pp. 239-245, 2016.

Sooksatra S., Kondo T. and Bunnun P., 2015. "A drowsiness detection method using distance and gradient vectors", Int. Conf. of Information and Communication Technology for Embedded Systems (IC-ICTES), Hua-Hin, Thailand, pp. 1-5, 2015.

Su C. H., Chiu H. S. and Hsieh T. M., 2011. "An efficient image retrieval based on HSV color space", Int. Conf. on Electrical and Control Engineering, Yichang, China, pp.5746-5749, 2011.

Vijayalaxmi and Rao P. S., "Eye detection using Gabor Filter and SVM", Int. Conf. on Intelligent Systems Design and Applications (ISDA), Kochi, India, pp. 880-883, 2012.

Vanlalhruaia, Singh Y. K. and Singh N. D., "Binary face image recognition using logistic regression and neural network", Int. Conf. on Energy, Communication, Data Analytics and Soft Computing (ICECDS), Chennai, India, pp. 3883-3888, 2017.

Xu B. and Li Q., "A bounded multi-dimensional modal logic for autonomous cars based on local traffic and estimation", Int. Symposium on Theoretical Aspects of Software Engineering (TASE), Sophia Antipolis, France, pp. 1-8, 2017.

Xu J., Liu B., Lin H. and Li J., 2011. "A new method for realizing LOG filter in image edge detection", Proceedings of 2011 6th International Forum on Strategic Technology, vol. 2, Harbin, Heilongjiang, Harbin, pp. 733-737, 2011.

Xu J., Min J. and Hu J., "Real-time eye tracking for the assessment of driver fatigue", Healthcare Technology Letters, vol. 5, no. 2, pp. 54-58, 2018.

Xu X., Wang S., Li F., Wu Z. and Sun W., “Research on Fault Diagnosis Based on Wavelet Packet Multi-class Classification SVM”, Int. Conf. on Manufacturing Automation, Hong Kong, China, pp. 164-167, 2010.

Xuan L. and Hong Z., "An improved canny edge detection algorithm", Int. Conf. on Software Engineering and Service Science (ICSESS), Harbin, China, pp. 275-278, 2017.

Yu Y. H., Lee T. T., Chen P. Y. and Kwok N., "On-chip real-time feature extraction using semantic annotations for object recognition", Journal of Real-Time Image Processing, vol. 15, no. 2, pp. 249-264, 2014.

Zhang F., Clarke D. and Knoll A., "Vehicle detection based on LiDAR and camera fusion", Int. Conf. on Intelligent Transportation Systems (ITSC), Berlin, Germany, pp. 1620-1625, 2014.

Zhang H., Zhu Q. and Guan X., “Probe into Image Segmentation Based on Sobel Operator and Maximum Entropy Algorithm”, Int. Conf. on Computer Science and Service System(CSSS), Nanjing, China, pp. 238-241, 2012.
Zhang Q. and Kamata S., “A novel color space based on RGB color barycenter”, IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China, pp. 1601-1605, 2016.