臺灣博碩士論文加值系統

揮動手掌是人類肢體語言中最具便捷性、直觀性，且最常被使用的動作，而透過手掌遠端操作器械在已成為未來發展的趨勢，相比於手持搖桿，利用手掌的揮動軌跡遠距離控制裝置不但跨越了空間的限制，同時人類手掌的靈活度也能做出更加精密細緻的動作，使需要繁瑣操作的工程簡單化。因此，本研究欲開發一個動態手掌追蹤系統，以期在未來能夠結合遠端連線及動態手勢辨識技術，達到遠距離使用硬體設備的功能。
本手掌追蹤系統的功能模組分為：手掌偵測、手掌追蹤及軌跡留存三個部分。手掌偵測模組使用MobileNetv2 SSD與非最大抑制偵測畫面中的手掌，並得出手掌邊界框的座標；手掌追蹤模組使用轉換色彩空間、二值化、反向投影、CamShift等方法實現手掌的追蹤；軌跡留存模組透過計算並儲存手掌的中心點座標，將手掌移動的軌跡顯示在螢幕。

Waving hands is the most convenient, intuitive, and commonly used gesture in human body language. Through the use of a hand-held remote control device, remote control through hand movements has become a trend in future development. Compared with handheld joysticks, the use of hand waving trajectories to control devices over long distances not only overcomes spatial limitations, but also allows for more precise and delicate movements of the human hand, simplifying the cumbersome operation of engineering. Therefore, this study aims to develop a dynamic hand tracking system to combine remote connection and dynamic gesture recognition technology to achieve the function of using hardware devices over long distances.
The functional modules of this dynamic hand tracking system are divided into three parts: hand detection, hand tracking, and trajectory retention. The hand detection module uses MobileNetv2 SSD and non-maximum suppression to detect hands in the screen and obtain the coordinates of the hand boundary box. The hand tracking module uses methods such as Color Space Conversion, Binarization, Back Projection, and CamShift to realize hand tracking. The trajectory retention module calculates and stores the coordinates of the hand's center point, displaying the trajectory of hand movement on the screen.

第一章 緒論 1
1.1 前言 1
1.2 研究動機與目的 1
1.3 問題與陳述 2
1.4 研究方法與貢獻 2
1.5 論文架構 3
第二章 文獻回顧與相關技術 4
2.1 影像處理概念 4
2.2 物件偵測技術 4
2.2.1 YOLO 5
2.2.2 SSD 7
2.2.3 注意力機制 9
2.3 物件追蹤技術 11
2.3.1 CamShift 11
2.3.2 KCF 12
2.3.3 Deep Sort 12
2.4 手掌偵測之應用 13
第三章 系統方法 14
3.1 手掌追蹤系統 14
3.2 手掌偵測模組 15
3.2.1 影像預處理子模組 16
3.2.2 手部偵測子模組 18
3.2.3 非最大值抑制 34
3.3 CamShift子模組 35
3.3.1 轉換色彩空間子模組 36
3.3.2 二質化子模組 38
3.3.3 計算直方圖子模組 39
3.3.4 反向投影子模組 40
3.3.5 計算中心點子模組 41
3.4 軌跡留存模組 42
第四章 實驗 44
4.1 實驗環境 44
4.1.1 軟體環境 44
4.1.2 桌上型電腦 45
4.1.3 筆記型電腦 45
4.1.4 感光元件 46
4.1.5 實驗架構圖 48
4.2 實驗摘要 49
4.3 實驗內容 50
4.4 實驗結果 51
4.4.1 資料集介紹 51
4.4.2 訓練與驗證方法 52
4.4.3 資料集訓練結果 57
4.4.4 CamShift效能測試 62
第五章 結論與未來展望 65
5.1 結論 65
5.2 未來展望 65
參考文獻 67

[1]J. S. Sheu, C. K. Tsai, and P. T. Wang, “Driving Assistance System with Lane Change Detection,” Advances in Technology Innovation, vol. 6, no. 3, pp. 137-145, 2021.
[2]J. S. Sheu and K. C. Wang, “Implementation of Instant Numeral Recognition in Space Trajectory,” Scientia Iranica Transations B: Mechanical Engineering, vol. 22, issue 4, pp. 1501-1509, Aug. 2015.
[3]C. Cortes and V. Vapnik, “Support-Vector Networks,” Machine Learning, vol. 20, pp. 273-297, Sep. 1995.
[4]I. Mesecan and I. O. Bucak, “Searching The Effects of Image Scaling for Underground Object Detection Using KMeans and KNN,” UkSim-AMSS 8th European Modeling Symposium, Pisa, Italy, pp. 180-184, Oct. 21st-23rd, 2014.
[5]P. Viola and M. Jones, “Rapid Object Detection Using a Boosted Cascade of Simple Features,” Proceeding of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Press, Kauai, HI, USA, pp. 511-518, Dec. 8th-14th, 2001.
[6]K. He, X. Zhang, S. Ren, and J.Sun, “Deep Residual Learning for Image Recognition,” Proceeding of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Press, Las Vegas, NV, USA, pp. 770-778, Jun. 27th-30th, 2016.
[7]P. T. Wang, S. Y. Lin, and J. S. Sheu, “Vehicle Path Planning with Multicloud Computation Services,” Advances in Technology Innovation, vol. 6, no. 4, pp. 213-221, 2021.
[8]J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection,” Proceeding of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Press, Las Vegas, NV, USA, pp. 779-788, Jun. 27th-30th, 2016.
[9]J. Redomn and A. Farhadi, “YOLO9000: Better, Faster, Stronger,” Proceeding of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Press, Honolulu, HI, USA, pp. 7263-7271, Jul. 21st-26th, 2017.
[10]S. REn, K. He, R. Grishick, and J. Sun, “Fast R-CNN: Towards Real-Time Object Detection with Region Proposal Networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, issue 6, pp. 1137-1149, Jun. 2017.
[11]J. Redmon and A. Farhadi, “YOLOv3: An Incremental Improvement,” arXiv preprint arXiv:1804.02767, 2018. (Access on Apr. 8th, 2018)
[12]A. Bochkovskiy, C. Y. Wang, and H. Y. M. Liao, “Yolov4: Optimal Speed and Accuracy of Object Detection,” arXiv preprint arXiv:2004.10934, 2020. (Access on Apr. 23rd, 2020)
[13]C. Y. Wang, H. Y. Mark Liao, Y. H. Wu, P. Y. Chen, J. W. Hsieh, and I. H. Yeh, “CSPNet: A New Backbone that Can Enhance Learning Capability of CNN,” Proceeding of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), IEEE Press, Seattle, WA, USA, pp. 390-391, Jun. 14th-19th, 2020.
[14]K. He, X. Zhang, S. Ren, and J.Sun, “Spatial Pyramidal Pooling in Deep Convolution Networks fof Visual Recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, issue 9, pp. 1904-1906, Sep, 2015.
[15]H. Li, P. Xiong, J. An, and L. Wang, “Pyramid Attention Network for Semantic Segmentation,” arXiv preprint arXiv:1805.10180, 2018. (Access on May. 25th, 2018)
[16]W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. H. Fu, and A. Berg, “SSD: Single Shot Multibox Detector,” Proceeding of Conference on European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands, pp. 21-37, Oct.11th-14th, 2016
[17]X. Liu, X. Kang, S. Nishide, and F. Ren, “Object Detection based on SSD-ResNet,” Proceeding of IEEE Conference on Cloud Computing and Intelligence and Systems (CCIS), IEEE Press, Singapore, pp. 89-92, Dec. 19th-21st, 2019.
[18]A. G. Howard, M. Zhu, B. Chen D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “MobileNets: Efficient Convoulutional Neural Networks for Mobile Vision Applications,” arXiv preprint arXiv:1704.04861, 2017. (Access on Apr. 17th, 2017)
[19]Y. C. Chiu, C. Y. Tsai, M. D. Ruan, G. Y. Shen, and T. T. Lee, “Mobilenet-SSDv2: An Improved Object Detection Model for Embedded Systems,” Proceeding of IEEE Conference on International Conference on System Science and Engineering (ICSSE), IEEE Press, pp. 1-5, Sep. Aug. 31st-Sep. 3rd, 2020
[20]C. Ning, H. Zhou, Y. Song, and J. Tang, “Inception Single Shot MultiBox Detector for Object Detection,” Proceeding of IEEE Conference on International Conference on Multimedia and Expo Workshop (ICMEW), IEEE Press, Hong Kong, China, pp. 10-14, Jul. 10th-14th, 2017.
[21]C. Y. Fu, W. Liu, A. Ranga, A. Tyagi, and A. Berg, “DSSD: Deconvolutional Single Shot Detector,” arXiv preprint arXiv:1701.06659, 2017. (Access on Jan. 23rd, 2017)
[22]Z. Li and F. Zhou, "FSSD: Feature Fusion Single Shot Multibox Detector," arXiv preprint arXiv:1712.00090, 2018. (Access on May. 17th, 2018)
[23]T. Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature Pyramid Networks for Object Detection,” Proceeding of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Press, Honolulu, HI, USA, pp. 2117-2125, Jul. 21st-26th, 2017.
[24]R. j. Wang, X. li, and C. X. Ling, “Pelee: A Real-Time Object Detection System on Mobile Devices,” Proceeding of Conference on Advances in Neural Information Processing Systems 31(NIPS 18), Montreal, Canada, pp. 1963-1972, Dec. 3rd-8th, 2018.
[25]J. Hao, F. Jiang, R. Zhang, X. Lin, B. Leng, and G. Song, “Scale Pyramid Attention for Single Shot MultiBox Detector,” IEEE Access, vol. 7, pp.138816-138824, Sep, 2019.
[26]Y. Liu and Y. Zhan, “Small Object Detection and Application Based on Improved SSD Algorithm,” Computer Science and Application, vol. 11, no. 4, pp. 1061-1069, Apr, 2021. (in Chinese)
[27]T. T. Feng and H. Y. Ge, “Pedestrian Detection Based on Attention Mechanism and Feature Enhancement with SSD,” Proceeding of Conference on International Conference on Communication, Image and Signal Proceeding (CCISP), Chengdu, China, pp. 145-148, Nov. 13th-15th, 2020.
[28]J. Hu, L. Shen, and G. Sun, “Squeeze-and-Excitation Networks,” Proceeding on IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Press, Salt Lake City, UT, USA, pp. 7132-7141, Jun. 18th-23rd, 2018.
[29]Q. wang, B. Wu, P. Zhu, P. Li, W. Zuo, and Q. Hu, “ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks,” Proceeding on IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Press, Seattle, WA, USA, pp. 11531-11539, Jun. 13th-19th, 2020.
[30]S. Woo, J. Park, J. Y. Lee, and I. S. Kwen, “CBAM: Convolutional Block Attention Module,” Proceeding of Conference on European Conference on Computer Vision (ECCV), Munich, Germany, pp. 3-19, Sep. 8th-14th, 2018.
[31] Q. Hou, D. Zhou, and J. Feng, “Coordinate Attention for Efficient Mobile Network Design,” Proceeding of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Press, Nashville, TN, USA, pp. 13713-13722, Jun. 20th-25th, 2021.
[32]Y. Cheng, “Mean Shift, Mode Seeking, and Clustering,” IEEE Transactions on pattern Analysis and Machine Intelligence, vol. 17, issue 8, pp. 790-799, Aug, 1995.
[33]J. Henriquesm R. Caseiro, P. Martins, and J. Batista, “High-Speed Tracking with Kernelized Correlation Filters,” IEEE Transacitions on pattern Analysis and Machine Intelligence, vol. 37, issue 3, pp. 583-596, Aug, 2014.
[34]L. Bertinetto, J. Valmadre, J. Henriques, and A. Vedaldi and P. Torr, “Fully-Convolutional Siamese Networks for Object Tracking,” Proceeding of Conference on European Conference on Computer Vision (ECCV), Amsterdam, Netherlands, pp. 850-865, Oct. 8th-10th, 2016.
[35]N. Wojke, A. Bewley, and D. Paulus, “Simple Online and Realtime Tracking with a Deep Association Metric,” Proceeding of IEEE Conference on International Conference on Image Processing (ICIP), IEEE Press, Beijing, China pp. 3645-3649, Sep. 17th-20th, 2017.
[36]G. R. Bradski, “Real Time Face and Object Tracking as a Component of a Perceptual User Interface,” Proceedings on IEEE Conference of Workshop on Applications of Computer Vision (WACV), IEEE Press, Princeton, NJ, USA, pp. 214-219, Oct. 19th-21st, 1998.
[37]A. Bewley, Z. Ge, L. Ott, F. Ramos, and B. Upcroft, “Simple Online and Realtime Tracking,” Proceeding of IEEE Conference on International Conference on Image Processing (ICIP), IEEE Press, Phoenix, AZ, USA, pp. 3464-3468, Sep. 25th-28th, 2016.
[38]H. Y. Lai and H. J. Lai, “Real-Time Dynamic Hand Gesture Recognition,” Proceeding of IEEE Conference on International Symposium on Computer, Consumer and Control, IEEE Press, Taichung, Taiwan, pp. 658-661, Jun. 10th-12th, 2014.
[39]C. Miron, A. Pasarica, H. Costin, V. Manta, R. Timofte, and R. Ciucu, “Hand Gesture Recognition based on SVM Classification,” Proceeding of IEEE Conference on E-Health and Bioengineering Conference (EHB), IEEE Press, Iasi, Romaina, pp. 1-6, Nov. 21st-23rd, 2019.
[40]J. Guo, J. Cheng, Y. Guo, and J. Pang, “A Real-Time Dynamic Gesture Recognition System,” Mechanics and Materials, vol. 3, pp. 849-855, Jul, 2013.
[41]P. Liu, X. Li, H. Cui, S. Li, and Y. Yuan, “Hand Gesture Recognition Based on Single-Shot Multibox Detector Deep Learning,” Mobile Information Systems, vol. 2019, pp. 1-7, Dec. 2019.
[42]F. Zhang, V. Bazarevsky, A. Vakunov, A. Tkachenka, G. Sung, C. L. Chang, and M. Grundmann, “MediaPipe Hands: On-Device Real-Time Hand Tracking,” arXiv preprint arXiv: 2006.10214, 2020. (Access on Jun. 18th, 2020)