臺灣博碩士論文加值系統

　　在智慧型手機盛行的時代，人機互動（Human Computer Interaction, HCI）之研究在這幾年間快速竄起，2D手勢辨識與解鎖技術愈發成熟，成為常見的操作方式及安全措施之一，然而3D電影、AR和VR的崛起，2D手勢的創造性與多樣性已無法滿足人們的需求，專家學者開始著力於3D手勢辨識之研究，在3D空間比起2D空間多了一個維度，其複雜度也大幅提升。本論文提出一套動態手勢鎖辨識方式，無需顧慮3D手勢起始點，只要手勢和原設定手勢密碼相近，即可順利辨識與解鎖。
　　在擁有三維座標數值的情況下，利用機器學習的方式，會直觀的將資料點當作訓練資料，進行深度學習，而此種訓練常伴隨著，起始點變更，整組資料被當成不同種類之手勢，且三維資料的各個維度，比重必須相同，造成某兩個維度資料相近，而另一維度相當不同時，現實情況差異大，但辨認影差異不大的問題，因此本論文提出另一訓練方式，不受限於起時位置，同時能辨別出各維度變化之影響的訓練模式，將3D手勢資料拆分成正視圖及側視圖，能夠同時接收到兩面向之變化，並應用影像處理之技巧，產生出深度學習需要的大量資料，使圖形訓練模型CNN達到較佳的訓練效果，完成手勢辨識解鎖。
　　本論文選用具有深度資訊的攝影設備Leap Motion取得手部位置與深度資訊，將三維度資訊正規化，以此組手勢做為樣板，進行資料擴展短時間建置手勢訓練集，並將訓練集之每組3D資訊拆分，分別對應組合成一張圖檔長100像素、寬50像素，最後帶入卷積類神經網路（CNN）訓練，完成一個密碼設定。欲解鎖者需對著本論文之手勢偵測設備，完成任一手勢，若經CNN模型預測，與已設手勢鎖相似度高於70%即可解鎖。

關鍵字：HCI、3D手勢、資料擴展、Leap Motion、CNN

※註1：「資料拆分與對應」於本論文定義為一組3D手勢資料，取兩組2D資料分別對應成一張圖，可得正視角圖和側視角圖，再將兩張圖合併成一張。

　　These days, mobile device can be seen everywhere, especially the smart phone which is a part of everyone’s life. Therefore, Human Computer Interaction (HCI), an interaction for human to computer, has been developed and researched in several years. It is important that a lot of technologies were designed in HCI, the recognition of 2D-gesture and security lock are cared more than twenty years ago, and then they play an considerable role in operating method and security for mobile device.
　　However, 3D-movies, augmented reality (AR), and virtual reality (VR) is rising rapidly, people are not satisfied with the creativity and diversity of 2D-technology, experts and scholars start to contributing to research on recognizing 3D-gesture, three dimensional space is direct information in real world, so it have really impact on people rather than the information of two dimensional space. Recognizing 3D-gesture is more complex than 2D’s it because adding a dimension lead to more combination and more possibility. The purpose of this study was to investigate a system of the recognition of moving gesture in three dimensional space for unlocking gesture; besides, the same of gesture with different origin was recognized the same pattern.
　　When getting the three dimensional information, it often is used to input data in the machine learning model, and doing a deep learning without data preprocessing. But data training with unprocessed data bring about the prediction that a different from the same gesture with the other origin. The purpose method of this study was another data training, using xy-plane and zy-plane of the 3D-gesture to be a front view and a side view, and then both view merge with a wider image. To increase the accuracy of prediction of CNN, some image processing and data augmentation were added.
　　Leap Motion is a great photo equipment, it not only have a good technology of hand tracking but also have high stability. Hence, it was selected in this paper. In gesture setting, getting information of user’s hand from Leap Motion, made normalizing, transforming it into two view image, data augmentation, and data training, each step is necessary. Finally, in unlocking mode, user waves his hand in front of the purposed device, if this gesture is similar to the anything of correct gesture, the lock will be unlocked.
Keywords：HCI、3D-gesture、data augmentation、Leap Motion、CNN

摘要．．．．．．．．．．．．．．．．．．．．．．．．．．．．．I
英文摘要．．．．．．．．．．．．．．．．．．．．．．．．．．．II
致謝．．．．．．．．．．．．．．．．．．．．．．．．．．．．III
目錄．．．．．．．．．．．．．．．．．．．．．．．．．．．．．IV
表目錄 ．．．．．．．．．．．．．．．．．．．．．．．．．．．VI
圖目錄 ．．．．．．．．．．．．．．．．．．．．．．．．．．．VII
第一章、緒論．．．．．．．．．．．．．．．．．．．．．．．．．1
1.1 研究動機．．．．．．．．．．．．．．．．．．．．．．．1
1.2 研究目的．．．．．．．．．．．．．．．．．．．．．．．2
1.3 論文大綱．．．．．．．．．．．．．．．．．．．．．．．2
第二章、文獻探討．．．．．．．．．．．．．．．．．．．．．．．3
2.1手部位置捕捉及追蹤 ．．．．．．．．．．．．．．．3
2.1.1 實時影像捕捉 ．．．．．．．．．．．．3
2.1.2 穿戴式感測器捕捉 ．．．．．．．．．．3
2.2 Leap Motion感測器 ．．．．．．．．．．．．．．4
2.3 道格拉斯－普克演算法（Douglas-Peucker Algorithm）．．．．．．．．4
2.4 資料擴增 ．．．．．．．．．．．．．．．．．．．．6
第三章、研究方法．．．．．．．．．．．．．．．．．．．．．．．7
3.1 整體架構 ．．．．．．．．．．．．．．．．．．．．7
3.2 實驗流程 ．．．．．．．．．．．．．．．．．．．．8
3.3掌心偵測與點座標蒐集 ．．．．．．．．．．．．．．11
3.4點座標正規化 ．．．．．．．．．．．．．．．．．．11
3.4.1邊界正規化 ．．．．．．．．．．．．．．．．．．．．．．13
3.5多餘點過濾與軌跡簡化 ．．．．．．．．．．．．．．14
3.5.1雜訊點過濾 ．．．．．．．．．．．．．．．．．．．．．．14
3.5.2 3D道格拉斯－普克演算法（3D Douglas–Peucker Algorithm）．14
3.6資料擴增（Data Augmentation）．．．．．．．．．．．．．．．．．16
3.6.1 3D道格拉斯－普克演算法之交叉配對．．．．．．．．．．．16
3.6.2差異對應變形法 ．．．．．．．．．．．．．．．．．．．．17
3.6.3 點座標三軸旋轉 ．．．．．．．．．．．．．．．．．．．．18
3.7資料拆分與對應．．．．．．．．．．．．．．．．．．．．．．19
3.8 CNN模型建置、訓練及預測．．．．．．．．．．．．．．．．．20
3.8.1 CNN預測結果穩定法．．．．．．．．．．．．．．．．．．20
第四章、實驗結果．．．．．．．．．．．．．．．．．．．．．．．21
第五章、結論與未來展望．．．．．．．．．．．．．．．．．．．．23
參考文獻．．．．．．．．．．．．．．．．．．．．．．．．．．．24

[1] David H Douglas and Thomas K Peucker, "Algorithms for the Reduction of the Number of Points Required to Represent a Digitized Line or its Caricature", Cartographica: The International Journal for Geographic Information and Geovisualization, pp.112-122, 1973.
[2] Ali A. Alani and Georgina Cosma, "Hand Gesture Recognition Using an Adapted Convolutional Neural Network with Data Augmentation", 2018 4th International Conference on Information Management (ICIM), pp.5-12, 2018.
[3] Mahmoud Elmezain, Ayoub Al-Hamadi, Jorg Appenrodt and Bernd Michaelis, "A Hidden Markov Model-Based Continuous Gesture Recognition System for Hand Motion Trajectory", 2008 19th International Conference on Pattern Recognition, pp.1-4 2008.
[4] V M Sethu Janaki, Satish Babu, and S S Sreekanth, "Real time recognition of 3D gestures in mobile devices", 2013 IEEE Recent Advances in Intelligent Computa-tional Systems (RAICS), pp.149-152, 2013.
[5] Qianqian Wang, Yuan-Rong Xu, Xiao Bai, Dan Xu, Yen-Lun Chen and Xinyu Wu, " Dynamic Gesture Recognition using 3D Trajectory Qianqian", 2014 4th IEEE International Conference on Information Science and Technology, pp.598-601, 2014.
[6] Jayash Kumar Sharma, Rajeev Gupta and Vinay Kumar Pathak, "Numeral Gesture Recognition using Leap Motion Sensor", 2015 International Conference on Computational Intelligence and Communication Networks, pp.411-414, 2015.
[7] Ti Chiang and Chih-Peng Fan, "3D Depth Information Based 2D Low-Complexity Hand Posture and Gesture Recognition Design for Human Computer Interact-ions", 2018 3rd International Conference on Computer and Communication Sys-tems, pp.233-238, 2018.
[8] S. Malassiotis, N. Aifanti and M.G. Strintzis, "A gesture recognition system using 3D data", Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission, pp.1-4, 2002.
[9] Ming-Hsuan Yang and Narendra Ahuja, "Recognizing Hand Gesture Using Motion Trajectories", Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149), pp.466-472 1999.