臺灣博碩士論文加值系統

隨著近幾年來，.人機介面與浮空手勢操作的發展，許多人投入相關的研究，並創造出更多更方便的應用。傳統的操作介面已逐漸轉變成更符合人類直覺的操作方式。本論文以三維浮空手勢手指互動介面為基礎，更深入探討手勢與手指之間的追蹤以及動作辨識的差異性，並各別分析其特性：手勢互動以掌心中心點為依據，利用其變化達到手勢的追蹤與辨識；而手指互動則是以指尖位置與手指骨架來作判斷，以分析手指的動作。接著將兩者特性做整合，藉此透過大幅度與小幅度動作的分類，在使用者介面上進行相關手勢手指動作的辨識，使得手勢手指互動的功能，對於使用者在使用上更加方便，實現出更完整的手勢手指互動技術。
本論文結合深度與彩色攝影機進行空間中物體的捕捉，利用所擷入的彩色資訊與深度資訊進行影像的分析，其過程包含了前處理的部分，透過手部的切割、偵測、追蹤以及手指的偵測，來獲取手部的資訊，並加入模糊控制理論，建立模糊控制器，定義其輸入輸出的模糊集合以及歸屬函數，經由所設計的演算法流程，得知當前手部的位置、手部的移動方向等資訊，在我們所建立的模糊規則執行模糊推論，最後進行解模糊化，就能推論出目前的動作是屬於大幅度的動作還是小幅度的動作，最後在動作辨識的部分，更進一步判斷出其相對應的動作功能，如手指的點擊、單手與雙手圖片的放大縮小、圖片的移動、圖片的旋轉和基於深度距離的圖片縮放，來進行使用者的互動操控，手指點擊準確度為91.6%，圖片移動偵測的準確率為92.4%，單手縮放的準確率為95.5%，而雙手縮放與旋轉偵測的準確率皆達100%，基於深度的圖片縮放偵測準確率達98.75%。此外，我們也將所提出的系統進行硬體架構設計，並根據數位積體電路設計流程實現為數位晶片，讀取手掌中座標位置的變化量之後，將分析出結果做為手勢手指互動功能的判斷依據。

With the human-machine interface and mid-air control have been developed in recent years, many people focus on related research, and create more and more convenient applications. The traditional user interface has been gradually transformed into more consistent with human intuitive operation manner.
In this thesis, the proposed method based on three-dimensional mid-air gesture and finger interactive interface, and more deep discussion the differences of tracking and actions recognition between gestures and fingers. We analyze their characteristics individually. The tracking and recognition of gesture is based on hand palm center changes. The analysis of finger movements is based on the position of the finger fingertip and finger skeleton. Then we integrate the two characteristics, and through the classification of big margin and small margin, to recognize gesture and finger movements on the user interface, to make more convenient for use about interact function of fingers and gestures, and realize more complete finger and gesture interactive technology.
In this thesis, we combine the depth and color cameras to capture objects in space, and use the color information and the depth information to analyze images. The process includes preprocessing through the hand segmentation, hand detection, hand tracking, and finger detection, to get the information of the hand, and add fuzzy control theory, establish fuzzy controller, define the fuzzy set and membership function of inputs and outputs. Through the design flow of algorithm, we can know the current hand position and hand moving direction information, and execute fuzzy inference in the fuzzy rule we have created. Finally, we can infer the current action belong with big margin or small margin after defuzzifier. In action recognition, it can determine the corresponding function such as finger clicking, image zooming by one hand/ two hands, image moving, image rotation depth distance-based zooming to operate with computer. The accuracy of finger clicking is 91.6%. The accuracy of image zooming is 92.4%. The accuracy of image zooming by one hand is 95.5%. The accuracy of image zooming by two hands and image rotation are both 100%, and the accuracy of depth distance-based zooming is 98.75%. Besides, we designed the hardware architecture design of the system we proposed, and implement a digital chip via the “Cell-based IC Design Flow.”

摘　要 i
ABSTRACT iii
誌 謝 v
目錄 vi
圖目錄 x
表目錄 xiv
第一章 介紹 1
1.1. 簡介 1
1.2. 發展與重要性 1
1.3. 研究動機 4
1.4. 論文章節組成 4
第二章 相關研究與文獻探討 5
2.1. 手部追蹤 5
2.1.1. 「CAMSHIFT」追蹤演算法 5
2.1.2. 「卡爾曼濾波器(Kalman filter)」的手勢追蹤技術 6
2.1.3. 手掌重心掃描追蹤(Palm Center Scanning) 8
2.2. 手勢辨識 10
2.2.1. 手勢偵測 10
2.2.2. 手指偵測 14
2.3. 人工智慧 17
2.3.1 模糊理論與模糊控制系統 17
2.3.2 人工類神經網路 22
2.3.3 類神經模糊系統 24
2.4 結論 26
第三章 研究方法 28
3.1 系統架構 28
3.2 前處理(Preprocessing) 29
3.2.1 手部切割 29
3.2.2 手部偵測 30
3.2.3 手部追蹤 31
3.2.4 手指偵測 31
3.3 模糊邏輯控制器(Fuzzy Logic Controller) 34
3.3.1 確定模糊控制器的架構 34
3.3.2 定義輸入、輸出模糊集合 35
3.3.3 定義輸入、輸出的歸屬函數 36
3.3.4 建立模糊控制規則 38
3.3.5 執行模糊推論 39
3.3.6 進行解模糊化 41
3.4 動作辨識(Action Recognition) 42
3.4.1 小幅度動作辨識 43
3.4.2 大幅度動作辨識 45
3.4.3 單輸入模糊控制器 48
3.5 結論 51
第四章 硬體架構設計 52
4.1 系統架構 52
4.2 模糊化機制 53
4.3 模糊推論引擎 57
4.4 解模糊化機制 60
4.5 資料流處理 62
4.6 結論 63
第五章 實驗方法與結果 64
5.1 實驗方法 64
5.2 實驗結果 71
5.3 實驗結果比較 80
5.3.1 手勢偵測準確率比較 80
5.3.2 動態手勢準確率比較 84
5.4 結論 88
第六章 晶片設計流程 89
6.1 數位電路設計流程 89
6.1.1 電路規格與輸出入腳位定義 89
6.1.2 RTL設計 91
6.1.3 邏輯合成設計 92
6.1.4 可測試性電路設計 93
6.1.5 自動佈局繞線 95
6.1.6 DRC與LVS驗證 95
6.1.7 實體佈局模擬驗證 96
6.2 晶片實體佈局與規格 97
6.3 數位晶片量測流程 99
6.4 結論 101
第七章 總結與未來展望 102
7.1 總結 102
7.2 未來展望 103
參考文獻 104
附錄A發表論文 109
附錄B獲獎榮譽 110
附錄C量測報告 111

[1] 人機互動技術, http://www.moneydj.com/KMDJ/wiki/WikiViewer.aspx?KeyID=38e2a33a-9a61-46c4-8bcd-ea64d51969c5
[2] Kinect, http://www.techbang.com/posts/2936-get-to-know-how-it-works-kinect
[3] Kinect | 台灣, http://www.xbox.com/zh-TW/Kinect
[4] Leap Motion | Mac & PC Motion Controller for Games, Design, Virtual Reality & More, https://www.leapmotion.com/
[5] Leap Motion 手勢互動裝置, http://www.powertw.com/#!leapmotion/c16bn
[6] 智慧臂環Myo, http://www.ithome.com.tw/news/94011
[7] S. M. Nadgeri, S. D. Sawarkar, and A. D. Gawande, “Hand gesture recognition using CAMshift algorithm,” International Conference on Emerging Trends in Engineering and Technology (ICETET) , 2010, pp. 37-41.
[8] Ryosuke Araki, Seiichi Gohshit, and Takeshi Ikenaga, “Real-time both hands tracking using CAMshift with motion mask and probability reduction by motion prediction,” Signal & Information Processing Association Annual Summit and Conference (APSIPA ASC), 2012, pp. 1-4.
[9] D. H. Yang and J. S. Xia, "Face Tracking Based on Camshift Algorithm and Motion Prediction," IEEE Workshop on Intelligent Systems and Applications, IEEE Press, May. 2009, pp.1-4.
[10] M. Asaari and S. Suandi, "Hand gesture tracking system using adaptive kalman filter," in International Conference on Intelligent Systems Design and Applications (ISDA), 2010 10th, Dec. 1, 2010, pp. 166 -171.
[11] Shu Mo, Shihai Cheng, and Xiaofen Xing, “Hand gesture segmentation based on improved kalman filter and TSL skin color model,” International Conference on Multimedia Technology (ICMT), 2011, pp. 3543-3546.
[12] Cheng Tang, Yongsheng Ou, Guolai Jiang, Qunqun Xie, and Yangsheng Xu, “Hand tracking and pose recognition via depth and color information,” IEEE International Conference on Robotics and Biomimetics (ROBIO), Dec.11-14, 2012, pp. 1104 – 1109.
[13] M. S. M. Asaari, and S. A. Suandi, “Hand gesture tracking system using adaptive Kalman filter,” International Conference on Intelligent Systems Design and Applications, Nov. 29-Dec. 1, 2010, pp. 166-171.
[14] H. Wang, L. Huo, and J. Zhang “Target tracking algorithm based on dynamic template and Kalman filter,” IEEE International Conference on Communication Software and Networks (ICCSN), 2011, pp. 330-333.
[15] Junhong Wu, Jun Cheng, and Wei Feng, “3D Dynamic Gesture Recognition Based on Improved HMMs with Entropy,” IEEE International Conference on Information and Automation (ICIA), July. 28-30, 2014, pp. 213-218.
[16] M. Van den Bergh, and L. Van Gool, “Combining RGB and ToF Cameras for Real-time 3D Hand Gesture Interaction,” IEEE Workshop on Applications of Computer Vision (WACV), Jan.5-7, 2011, pp. 66-72.
[17] Zhou Ren, Junsong Yuan, Jingjing Meng, and Zhengyou Zhang, “Robust Part-Based Hand Gesture Recognition Using Kinect Sensor,” IEEE Trans. Multimedia, vol. 15, no. 5, pp. 1110-1120, Aug. 2013.
[18] Lae-Kyoung Lee, Su-Yong An, and Se-Young Oh, “Robust fingertip extraction with improved skin color segmentation for finger gesture recognition in human-robot interaction,” IEEE Congress on Evolutionary Computation (CEC), 2012, pp. 1-7.
[19] G. Simion, V. Gui, and M. Otesteanu. “Finger detection based on hand contour and colour information,” in IEEE International Symposium on Applied Computational Intelligence and Informatics (SACI), may 2011,pp. 97-100.
[20] Z. Meng, J. Pan, k. Tseng, et al., "Dominant points based hand finger counting for recognition under skin color extraction in hand gesture control system, " in Genetic and Evolutionary Computing (ICGEC), 2012, pp. 364-367.
[21] J. Lambrecht, H. Walzel, and J. Kruger, ”Robust finger gesture recognition on handheld devices for spatial programming of industrial robots,” IEEE Int. Workshop on Robots and Human Interactive Communications. Aug, 2013, pp. 26-29.
[22] M. Zabri Abu Bakar, R. Samad, D. Pebrianti and N. L. Y. Aan “Real-time rotation invariant hand tracking using 3D data,” IEEE International Conference on Control System Computing and Engineering(ICCSCE), Nov. 28 – 30, 2014, pp. 490-495.
[23] 周世穎，基於深度與色彩資訊之三維自然使用者介面設計與晶片實現，國立台北科技大學電腦與通訊研究所，台北，2014。
[24] 黃信榮，基於立體視覺手勢辨識的人機互動系統，國立中央大學資訊工程研究所，台北，2009。
[25] Y. Li "Multi-scenario gesture recognition using Kinect," International Conference Computer Games (CGAMES), July, 2012, pp. 126-130.
[26] A. Kulshreshth, C. Zorn and J.J. LaViola, "Poster: Real-time markerless Kinect based finger tracking and hand gesture recognition for HCI,"IEEE Symposium on 3D User Interfaces (3DUI), Mar. 16-17, 2013, pp. 187-188
[27] S. Poularakis and I. Katsavounidis, “Finger detection and hand posture recognition based on depth information,” IEEE International Conference on Acoustic Speech and Signal Processing (ICASSP), May. 4-9, 2014, pp. 4329 - 4333.
[28] A. Chinu, Chhabra, “A hybrid approach for color based image edge detection,” International Conference on Advances in Computing Communications and Informatics (ICACCI), Sep. 24-27, 2014, pp. 2443 -2448.
[29] N.Q. Qureshi and I. Akhter, “Interactive Control Through Hand Gestures,” International Conference Frontiers of Information Technology (FIT), Dec. 16-18, 2013, pp. 83 - 88.
[30] R. Hartanto, A. Susanto and P. I. Santosa, “Real time hand gesture movements tracking and recognizing system,” Electrical Power Electronics, Communications Controls and Informatics Seminar (EECCIS), Aug. 27-28, 2014, pp. 137-141.
[31] 模糊控制理論, http://wiki.mbalib.com/zh-tw/
[32] 張斐章、張麗秋，類神經網路，東華書局，2005。
[33] 陶雨龍，改良式模糊類神經網路與其應用，私立中原大學，桃園，2004。
[34] 李允中、王小璠、蘇木春，模糊理論及其應用(精裝本)(第三版) ，全華圖書股份有限公司，2002。
[35] 劉韋亨，基於區間第二型模糊理論之跌倒偵測系統設計，國立台北科技大學，台北，2013。
[36] 王進德，類神經網路與模糊控制理論入門與應用，全華圖書股份有限公司，2006。
[37] 王文派、邱俊智 (2010)，自適應性類神經模糊推論系統於客製化生產環境之預測應用，資訊科技國際研討會論文集。
[38] 人工神經網路, https://zh.wikipedia.org/wiki/%E4%BA%BA%E5%B7%A5%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C
[39] Xingang Fu, Ting Zhang, H. Chadwell Bonair, L. Marvin and Jiang Lu, “Wavelet Enhanced Image Preprocessing and Neural Networks for Hand Gesture Recognition,” IEEE International Conference on Smart City/SocialCom/SustainCom together with DataCom and SC2, Dec. 19-21, 2015, pp. 838 – 843.
[40] Xiying Wang and Guozhong Dai “A Novel Method to Recognize Complex Dynamic Gesture by Combining HMM and FNN Models,” IEEE Symposium on Computational Intelligence in Image and Signal Processing (CIISP 20), Apr. 1-5, 2007, pp. 13-18.
[41] 郭日良、陳順智、林詩文，"高速可參數化之模糊邏輯控制器晶片"，第三屆微電子技術發展與應用研討會，2005 。
[42] 林恆瑤，以低電壓電流式電路設計之語意修辭模糊邏輯控制器，國立成功大學電機工程學系碩博士班，台南，2003。
[43] 黃聖翔，雙手自然使用者介面設計與晶片實現，碩士論文，國立台北科技大學，台北，2013。
[44] 王竣弘，基於複雜背景資訊之手勢互動介面設計與晶片實現，國立台北科技大學電腦與通訊研究所，台北，2014。
[45] R. Annamária, Várkonyi-Kóczy and Balázs Tusor. “Human–Computer Interaction for Smart Environment Applications Using Fuzzy Hand Posture and Gesture Models,” IEEE Trans on instrumentation and measurementation and measurement, vol. 60, no. 5, pp. 1505-1514, May, 2011.
[46] Design for Testability and Scan, http://www.csee.umbc.edu/~cpatel2/links/418/lectures/chap14_lect13_scan.pdf
[47] 財團法人國家實驗研究院國家晶片系統設計中心電子報第153期, https://www.cic.org.tw/CommonUtilServlet?type=2.4&file=1444.pdf
[48] Giuseppe Ascia and Vincenzo Catania, “A pipeline parallel architecture for a fuzzy inference processor.” IEEE International Conference on Fuzzy Systems, vol.1, May. 7-10, 2000, pp. 257-262
[49] A. Faizal Samman and Y. Eniman Syarnsuddin, “PROGRAMMABLE FUZZY LOGIC CONTROLLER CIRCUIT ON CPLD CHIP.”, Asia-Pacific Conference on Circuits and Systems, vol.2, 2002, pp.561 – 564