臺灣博碩士論文加值系統

近年來視覺追蹤的應用逐漸普遍，從實驗室環境逐步擴展至生活中的應用，如醫學及認知心理學、行車安全監控、廣告效益分析及學習歷程分析等等生活應用面。但目前研究實驗環境或是應用場域，大多侷限於單人使用，其原因在於背景干擾及多人視覺落點不易利用單一化模組進行估算，而為解決此問題，本研究提出利用PTZ攝影機與Kinect結合取得觀看者位置資訊及眼睛影像，並依設備條件建構實驗環境，為了收集實驗環境內所有觀看資訊，但其需耗費大量時間，故提出參考點建立方法，由於特徵群聚效應，相近的參考點其觀看資訊也會相近，故提出將參考點進行分群，並依照所屬群組經由CNN建構出多樣性視覺評估模組，可以有效降低建構關注模型所耗費的時間。由於實驗環境自行建置，並無開放的資料庫，所以眼睛影像收集不易，故本研究透過資料擴增方法增加訓練樣本，建置多人視覺落點追蹤系統且可應用於開放式場域。

In recent years, the application of gaze tracking has become more and more popular from the laboratory environment to the application in life, such as in medical and cognitive psychology, in analyzing advertising effectiveness and learning process. Most of the experimental environments are limited for single user because gaze points are difficult to estimate when multi-users are involved and environment background might cause the interference. To solve this problem, the research constructed an experimental condition and proposed using a PTZ camera combined with Kinect to obtain eye images and position information of the audience. Collecting all the focused information is time consuming. Therefore, the method of computing reference points is proposed. Based on the feature clustering effect, neared reference points have similar focused information. Therefore, grouping the clusters of the reference points is proposed. CNN then is used to construct an estimated gaze module according to the grouping outcomes. In the meantime, lacking the open database makes it difficult on collecting eye images as the training samples. The research used data augmentation method to increase the number of training samples and built a Multiple-user Eye Tracking system for open fields.

摘要 i
Abstract ii
致謝 iii
目錄 iv
圖目錄 vi
表目錄 ix
第一章 緒論 1
1.1研究背景與動機 1
1.2研究目的 1
1.3論文架構 2
第二章 文獻探討 3
2.1視覺追蹤 3
2.2深度學習 4
2.3卷積神經網路 5
2.4資料擴增(Data Augmentation) 8
2.5臉部與眼部定位方法 13
第三章 研究方法 17
3.1單人關注區域問題探討 17
3.2多人關注區域問題探討 21
3.3蒐集觀看資訊方法 26
3.3.1參考點計算 27
3.3.2位置分群 28
3.4多人關注區域追蹤系統架構說明 31
3.5觀看者影像擷取程序 32
3.5.1 PTZ控制程序 33
3.5.2取得正臉影像和眼睛影像 34
3.6關注區域模型 (FAM) 36
3.6.1單眼關注區域模型 37
3.6.2雙眼關注區域模型 38
3.6.3臉部關注區域角度分類模型 39
3.6.4多人關注區域模型 39
第四章 實驗結果 41
4.1實驗環境 41
4.2資料擴增 42
4.3模型評估 42
4.3.1單眼關注區域模型評估(FASEM) 44
4.3.2雙眼關注區域模型評估(FAEM) 47
4.3.3臉部關注區域角度分類模型評估(FAFRCM) 49
4.3.4多人關注區域模型設計(FAMAM) 51
4.4多人關注區域模型 52
4.4.1雙眼關注區域模型評估 52
4.4.2多人關注區域模型評估 55
4.5資料擴增方法比較 59
第五章 結論與未來展望 64
參考文獻 65

[1]D.E. Broadbent, “Perception and communication,” Elsevier, pp. 75－80, 2013.
[2]A.L. Yarbus, “Eye movements during perception of complex objects,” Eye movements and vision, pp.171－211, 1967.
[3]C.W. Kao, Y.J. Huang, K.C. Fan, H.H. Chen, W.C. Chung, B.J. Hwan and C.H. Hsieh, “The integrated gaze, web and object tracking techniques for the web-based e-learning platform,” IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE), pp. 720—724, 2013.
[4]Y. Sugano, Y. Matsushita and Y. Sato, “Appearance-Based Gaze Estimation Using Visual Saliency,” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 329－341, 2013.
[5]S.V. Sheela and P.A. Vijaya, “An Appearance based Method for Eye Gaze Tracking,” Journal of Computer Science, pp.1194－1203, 2011.
[6]H.C. Lu, G.L. Fang, C. Wang and Y.W. Chen, “A novel method for gaze tracking by local pattern model and support vector regressor,” Signal Processing, pp. 1290－1299, 2010.
[7]C.W. Kao, B.J. Hwang, C.W. Yang, K.C. Fan and C.P. Huang, “A Novel with Low Complexity Gaze Point Estimation Algorithm,” Proceedings of the International MultiConference of Engineers and Computer Scientists., vol. 1, 2012.
[8]K. Nayan,“EyeX Controller: SteelSeries and Tobii Technology team up to take on Kinect with eye-tracking controller,” SIGGRAPH ASIA 2016 Courses, pp. 17, 2016.
[9]K. Krafka, A. Khosla, P. Kellnhofer, H. Kannan, S. Bhandarkar, W. Matusik and A. Torralba, “Eye tracking for everyone,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2176－2184, 2016.
[10]E. Wood, T. Baltrusaitis, X. Zhang, Y. Sugano, P. Robinson and A. Bulling, “Rendering of eyes for eye-shape registration and gaze estimation,” Proceedings of the IEEE International Conference on Computer Vision, pp. 3756－3764, 2015.
[11]R. Jafari and D. Ziou, “Gaze estimation using Kinect/PTZ camera,” Robotic and Sensors Environments (ROSE), 2012.
[12]K. A. Funes Mora and J. M. Odobez, “Gaze estimation from multimodal kinect data,” IEEE Conference in Computer Vision and Pattern Recognition, Workshop on Gesture Recognition(CVPRW), pp. 25－30, 2012.
[13]J. Markoff, “How many computers to identify a cat? 16,000,” New York Times, pp. 06－25, 2012.
[14]S. Draghici, “A neural network based artificial vision system for licence plate recognition,” International Journal of Neural Systems, pp. 113－126, 1997.
[15]C. Meng and X. Zhao, “Webcam-Based Eye Movement Analysis Using CNN,” IEEE Access 5, pp. 19581－19587, 2017.
[16]S. Hoppe and A. Bulling, “End-to-end eye movement detection using convolutional neural networks,” arXiv preprint arXiv:1609.02452, 2016.
[17]C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke and A. Rabinovich, “Going deeper with convolutions,” Proceedings of the IEEE conference on computer vision and pattern recognition, 2015.
[18]K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
[19]F. Chollet, “Keras,” Keras, pp. 153－157, 2015.
[20]T. Tran and T. Pham, “A Bayesian Data Augmentation Approach for Learning Deep Models,” Advances in Neural Information Processing Systems, pp. 2794－2803, 2017.
[21]H. Furukawa, “Deep Learning for Target Classification from SAR Imagery: Data Augmentation and Translation Invariance,” IEICE Technical Report, pp. 13－17, 2017.
[22]K. Nguyen, T. Le, V. Nguyen, T. Nguyen, and D. Phung, “Multiple Kernel Learning with Data Augmentation,” Asian Conference on Machine Learning, pp. 49－64, 2016.
[23]D. Komura, and S. Ishikawa, “Machine learning methods for histopathological image analysis,” arXiv preprint arXiv:1709.00786, 2017.
[24]R. Kohavi, “A study of cross-validation and bootstrap for accuracy estimation and model selection,” Ijcai., vol. 14. no. 2, pp.1137－1145, 1995.
[25]B. F. Klare, B. Klein, E. Taborsky, A. Blanton, J. Cheney, K. Allen, P. Grother, A. Mah and A. K. Jain, “Pushing the frontiers of unconstrained face detection and recognition: Iarpa janus benchmark a,” Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1931－1939, 2015.
[26]D. E. King, “Dlib-ml: A machine learning toolkit,” Journal of Machine Learning Research, pp. 1755－1758, 2009
[27]A. Jana, “Kinect for windows SDK programming guide,” Packt Publishing Ltd, 2012.
[28]K. K. Shieh and D. S. Lee, “Preferred viewing distance and screen angle of electronic paper displays,” Applied Ergonomics, pp. 601－608, 2007.
[29]D. Birant and A. Kut, “ST-DBSCAN: An algorithm for clustering spatial–temporal data,” Data & Knowledge Engineering, vol. 60. no.1, pp. 208－221, 2007.
[30]K. He, X. Zhang, S. Ren and J. Sun, “Identity mappings in deep residual networks,” European conference on computer vision, pp. 630－645, 2016.