臺灣博碩士論文加值系統

本研究提出以演算法模型調適深度類神經網路來實現眼動追蹤，利用六個難度不同的連續性實驗收集眼球資訊以後續眼動追蹤。在注視熱區方面，則利用三個不同的校正板並分為屏幕顯示以及顯示器顯示收集10個受測者的眼動資訊。
於眼動追蹤部份，本研究採用基於Android系統架構之智慧眼鏡平台來擷取眼動資訊，分為小圖片176x144 pixels以及1280x720 pixels，並以傳統影像處理將收集到的影片先截成圖片，並對圖片進行灰階、二值化，在得到二值化影像後再以高斯模糊及形態學方式消除雜訊，接著以霍夫轉換以及感興趣區域的條件限制擬合出最正確虹膜中心，並利用subject adaptive e機制挑出與全卷積網路結果誤差低於leave one 中10個人的RMSE以及RMSE+0.5SD以及RMSE+SD結果的影像並當成全卷積網路的輸入，176x144 pixels的結果比原本只有單純全卷積網路誤差降低了2.6% pixels，1280x720 pixels做leave one的瞳孔中心預測結果比176x144 pixels更準確，但是其睜眼準確率較低。
於注視熱區部份，本研究利用三種校正板進行校正計算視角誤差，分為兩種模式，一種為屏幕顯示另一種為利用智慧眼鏡的顯示器顯示，利用屏幕顯示也分為錄影以及拍照模式，像素分別為176x144 pixels以及640x480 pixels，針對兩個不同pixel差做正規化之後，錄影模式視角誤差為2.02度，拍照模式誤差視角為1.98度。利用顯示器顯示則是利用Record Video Background軟體錄製影片收集眼動資訊，其視角誤差為2.13度。本研究之最大創新點與貢獻在於利用演算法模型搭配深度類神經網路達到較好的效果。

This research proposes to use an algorithm model to adapt a deep neural network to achieve eye tracking, and use six consecutive experiments with different difficulties to collect eye information for subsequent eye tracking. In terms of gazing at the hot zone, three different calibration boards are used and divided into screen display and monitor display to collect the eye movement information of 10 people. For eye tracking, this research uses a smart glasses platform based on the Android system architecture to capture eye movement information, which is divided into small pictures of 176x144 pixels and 1280x720 pixels. First ,using traditional image processing to cut the collected videos into pictures. The image will do the gray-scale and binary. After obtaining the image, Gaussian blur and morphological methods are used to eliminate noise, and then the most correct iris center is fitted by Hough transform and the conditions of the region of interest. And use the subject adaptive mechanism to pick out the images with the results of the full convolutional network that are lower than the RMSE, RMSE+0.5SD and RMSE+SD results of the 10 people in leave one and use them as the input of the full convolutional network, 176x144 pixels The result is 2.6% lower than the original pure full convolutional network error. The pupil center prediction result of 1280x720 pixels leave one is more accurate than 176x144 pixels, but the accuracy of eye opening is lower. In the hot spot of gaze, this study uses three calibration boards to correct and calculate the viewing angle error, which is divided into two modes, one is screen display and the other is display display using smart glasses. The screen display is also divided into video and photo modes. The pixels are 176x144 pixels and 640x480 pixels. After normalizing the difference between the two different pixels, the viewing angle error in the recording mode is 2.02 degrees, and the viewing angle error in the photographing mode is 1.98 degrees. Using the monitor to display is to use the Record Video Background software to record video to collect eye movement information, and the viewing angle error is 2.13 degrees. The biggest innovation and contribution of this research lies in the use of algorithmic models with deep neural networks to achieve better results.

致謝辭 II
中文摘要 III
Abstract IV
目錄 I
圖目錄 III
表目錄 IV
第一章 序論 1
1.1 研究動機與目的 1
1.2 研究架構 1
第二章 研究背景 3
2.1 前言 3
2.2VR、AR介紹 3
2.3 眼動追蹤 5
2.3.1 眼球構造 6
2.3.2 眼球運動 7
2.3.3 眼動儀 9
2.4 深度學習探討 11
第三章 眼動追蹤系統 12
3.1 硬體架構 12
3.1.1 硬體設備與規格 13
3.2 眼動資訊實驗 15
3.2.1 眼動實驗介紹以及流程 15
3.3 眼動資訊擷取 17
3.3.1 虹膜中心檢測 19
3.3.2 全卷積神經網路 25
3.3.3 自動編碼器 27
3.3.4 視線投射實驗介紹以及流程 28
第四章 實驗結果與討論 29
4.1眼動追蹤 29
4.1.1 實驗結果 30
4.1.2 相關文獻比較 33
4.2視線投射 35
4.2.1 實驗結果 35
4.2.2 相關文獻比較 37
第五章 結論與未來展望 38
5.1 結論 38
5.2 未來展望 39
參考文獻 40

[1]https://ictjournal.itri.org.tw/Content/Messagess/contents.aspx?&MmmID=654304432061644411&MSID=745170540670271640
[2] https://vr.tobii.com/foveated-rendering/
[3] M. Köles, B. P. Hámornik, E. Lógó, K. Hercegfi and S. Tóvölgyi, "Experiences of a combined psychophysiology and eye-tracking study in VR," 2014 5th IEEE Conference on Cognitive Infocommunications (CogInfoCom), Vietri sul Mare, 2014, pp. 391-395.
[4] E. Whitmire, "High-Fidelity Interaction for Virtual and Augmented Reality," 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Reutlingen, 2018, pp. 796-798.
[5] http://140.134.32.129/bioassist/eyetrack/paper2/2-8.htm
[6] T. Kuo and C. Fan, "Design and Implementation of Deep Learning Based Pupil Tracking Technology for Application of Visible-Light Wearable Eye Tracker," 2020 IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, NV, USA, 2020, pp. 1-2, doi: 10.1109/ICCE46568.2020.9043149.
[7] N. Kan, N. Kondo, W. Chinsatit and T. Saitoh, "Effectiveness of Data Augmentation for CNN-Based Pupil Center Point Detection," 2018 57th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE), Nara, 2018, pp. 41-46, doi: 10.23919/SICE.2018.8492593.
[8] N. Kondo, W. Chinsatit and T. Saitoh, "Pupil center detection for infrared irradiation eye image using CNN," 2017 56th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE), Kanazawa, 2017, pp. 100-105, doi: 10.23919/SICE.2017.8105630.
[9] Z. Zhao, Y. Zhang and Q. Zheng, "Fast Localization Algorithm of Eye Centers Based on Improved Hough Transform," 2019 IEEE 7th International Conference on Bioinformatics and Computational Biology ( ICBCB), Hangzhou, China, 2019, pp. 185-189, doi: 10.1109/ICBCB.2019.8854635.
[10] A. Liaghatdar, K. Kangarloo and F. Farokhi, "Pupil localizing in video images the first step toward eye monitoring," 2011 International Conference on Multimedia Technology, Hangzhou, 2011, pp. 3163-3166, doi: 10.1109/ICMT.2011.6001859.
[11] M. T. Setiawan, S. Wibirama and N. A. Setiawan, "Robust Pupil Localization Algorithm Based on Circular Hough Transform for Extreme Pupil Occlusion," 2018 4th International Conference on Science and Technology (ICST), Yogyakarta, 2018, pp. 1-5, doi: 10.1109/ICSTC.2018.8528286.
[12] H. Qin, X. Wang, M. Liang and W. Yan, "A novel pupil detection algorithm for infrared eye image," 2013 IEEE International Conference on Signal Processing, Communication and Computing (ICSPCC 2013), KunMing, 2013, pp. 1-5, doi: 10.1109/ICSPCC.2013.6664050.
[13] Y. Li and Z. Xue, "Fast Iris Boundary Location Based on Window Mapping Method," 2011 Seventh International Conference on Computational Intelligence and Security, Hainan, 2011, pp. 1127-1130, doi: 10.1109/CIS.2011.250.
[14] A. S. Al-Waisy, R. Qahwaji, S. Ipson and S. Al-Fahdawi, "A Fast and Accurate Iris Localization Technique for Healthcare Security System," 2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing, Liverpool, 2015, pp. 1028-1034, doi: 10.1109/CIT/IUCC/DASC/PICOM.2015.156.
[15] S. Jamaludin, N. Zainal and W. M. D. W. Zaki, "Fast, accurate and memory efficient pupil localization based on pixels properties method," 2015 IEEE 3rd International Conference on Smart Instrumentation, Measurement and Applications (ICSIMA), Kuala Lumpur, 2015, pp. 1-6, doi: 10.1109/ICSIMA.2015.7559036.
[16]蔡宜庭，「藉由眼動資訊、九軸感測訊號及生理訊號估測專注度」，國立中正大學，民國一百零六年七月
[17] S. Baek, K. Choi, C. Ma, Y. Kim and S. Ko, "Eyeball model-based iris center localization for visible image-based eye-gaze tracking systems," in IEEE Transactions on Consumer Electronics, vol. 59, no. 2, pp. 415-421, May 2013.
[18] J. Wu, W. Ou and C. Fan, "NIR-based gaze tracking with fast pupil ellipse fitting for real-time wearable eye trackers," 2017 IEEE Conference on Dependable and Secure Computing, Taipei, 2017, pp. 93-97.
[19]陳品志，「結合智慧眼鏡上慣性元件與眼動資訊認知來估測專注度」，國立中正大學，民國一百零五年八月
[20] 陳品志，「結合智慧眼鏡上慣性元件與眼動資訊認知來估測專注度」，國立中正大學，民國一百零五年八月