臺灣博碩士論文加值系統

行政院國家發展委員會的報告指出臺灣正處在高齡社會，更推估於2025年邁入超高齡社會，這意味著六十五歲以上的銀髮族比例逐年增加，而銀髮族經常面臨身體老化或是相關疾病導致肌力方面出現退化的情況，症狀輕則肌肉萎縮，重則臥病在床甚至永久癱瘓，對於復健醫療的需求日漸增大，為了保全醫療體系，完備醫療量能，良好的居家復健顯得格外重要。
在疫情肆虐全球的情況下，世界各地的經濟與社會問題面臨著相當大的衝擊，使得人們不得不改變目前的生活習慣，許多長期在醫院接受復健治療的病患也須改為居家復健，但目前居家復健仍然面臨不少挑戰。因此本研究提出一套結合人工智慧與人型機器人的復健輔助系統。首先，人型機器人會扮演復健師的角色，示範復健動作並帶領患者復健。其次，使用OpenPose模型用於追蹤復健時的人體骨架。最後，將辨識結果與事先設定的標準動作比較，當動作不標準時發出提示，再將其動作的標準程度量化成評分，以利後續記錄與追蹤。
本研究所提出的方法在COCO 2017資料集與自行建立的LSVT Dataset上得到驗證，在骨架偵測準確率方面的Average Precision達到85.5%，並且邀請9位受測者參與實驗，實驗結果顯示當動作越不標準時，系統的評分也會越低，證明這項研究可以為居家復健提供新的監督方法。

The report of the National Development Committee of the Executive Yuan pointed out that Taiwan is in an aging society, and it is estimated that it will enter a super-aged society in 2025, which means that the proportion of silver-haired people over the age of 65 is increasing year by year, and silver-haired people often face physical aging or degeneration of muscle strength caused by related diseases. The symptoms range from muscle atrophy to bedridden or even permanent paralysis. The demand for rehabilitation medicine is increasing day by day. In order to preserve the medical system, complete medical capacity and home rehabilitation is very important.
With the Covid-19 raging around the world, economic and social problems around the world are facing a considerable impact, making people have to change their current living habits, and many patients who have been receiving rehabilitation treatment in hospitals for a long time must also be rehabilitated at home. But home rehabilitation currently faces two key challenges.
One point is that there is no guidance from a physical therapist at home for rehabilitation activities, and patients may have non-standard but unconscious problems in rehabilitation activities. The other point is that there is no objective evaluation index for patients to refer to the progress after each home rehabilitation. In order to solve the above problems, this study proposes a rehabilitation assistance system combining artificial intelligence and humanoid robots. First, the humanoid robot can play the role of a physical therapist, demonstrating rehabilitation movements and leading patients to rehabilitate. Second, OpenPose model is used to track the human skeleton during rehabilitation. Finally, we compare with the pre-set standard actions, calculate their similarity, and then convert them into quantitative scores for subsequent recording and tracking.
The method proposed in this study has been verified on the COCO 2017 dataset and self-built LSVT dataset, and the average precision of skeleton detection reaches 85.5%. We invited 9 subjects to participate in the experiment. The results show that the more the actions are non-standard, the lower the score is. This proved that the presented work can provide a new monitoring method for home rehabilitation.

摘　要 i
ABSTRACT ii
致　謝 iv
目　錄 v
表目錄 viii
圖目錄 ix
第一章 緒論 1
1.1 研究背景 1
1.2 研究目的 2
1.3 研究方法 3
1.4 論文架構 4
第二章 相關技術與運用探討 5
2.1 巴金森氏症復健運動 5
2.2 姿態檢測技術 6
2.2.1 VICON 6
2.2.2 Azure Kinect 7
2.2.3 PoseNet 8
2.2.4 OpenPose 9
2.3 MobileNet 12
2.4 卡爾曼濾波 15
2.5 動態時間校正 17
2.6 Flask框架 19
第三章 軟硬體架構與設計 20
3.1 硬體設備 20
3.1.1 人型機器人 20
3.1.2 深度攝影機 24
3.1.3 Arduino 25
3.1.4 Raspberry Pi 26
3.2 開發環境 27
3.3 系統架構與流程 28
3.4 模型架構 31
3.5 復健動作分析方法 32
3.5.1 Floor to Ceiling 32
3.5.2 Side to Side 33
3.5.3 Forward Step 35
3.5.4 Backward Step 36
3.5.5 Forward Rock and Reach 37
3.5.6 Sideways Rock and Reach 38
3.5.7 Sideways Step 39
3.6 復健成效評估方法 40
3.6.1 雜訊抑制 41
3.6.2 動態時間校正 (Dynamic Time Warping, DTW) 42
3.6.3 骨架座標轉換 42
3.6.4 量化評分 44
3.7 圖形化使用者介面 44
第四章 實驗規劃與結果 47
4.1 資料集 47
4.1.1 COCO 2017 Dataset 47
4.1.2 LSVT Dataset 48
4.2 評估指標 48
4.2.1 PCK (Percentage of Correct Keypoints) 48
4.2.2 OKS (Object Keypoint Similarity) 49
4.2.3 AP (Average Precision) 49
4.3 基於OpenPose偵測骨架 49
4.3.1 資料標記 49
4.3.2 訓練流程 50
4.3.3 模型比較 51
4.4 基於復健成效評估方法輸出分數 53
4.4.1 資料收集 53
4.4.2 實驗結果 54
4.5 影像串流速度 55
第五章 結論與未來展望 56
5.1 結論 56
5.2 未來展望 56
參考文獻 57

[1]健保大數據看巴金森（6-2）／高齡社會 病友年增2千人，
https://udn.com/news/story/7266/6194101.
[2]D. Roetenberg, C. Baten and P. Veltink, “Estimating body segment orientation by applying inertial and magnetic sensing near ferromagnetic materials,” IEEE Trans. on Neural Systems and Rehabilitation Engineering, vol. 15, no. 3, pp.469-471, Sep. 2007.
[3]VICON，
https://www.vicon.com/hardware/cameras/.
[4]VICON製品，
https://www.crescentinc.co.jp/product/vicon/p_syosai/camera/vantage/.
[5]N. DiFilippo and M. Jouaneh, “Characterization of different Microsoft Kinect sensor models,” IEEE Sensors Journal, vol. 15, no. 8, pp.4554-4564, Aug. 2015.
[6]A. Bari and M. Gavrilova, “Artificial neural network based gait recognition using Kinect sensor,” IEEE Access, vol. 7, pp.162708-162722, Nov. 2019.
[7]微軟宣佈初代 Kinect 將在 2015 年退休，
https://chinese.engadget.com/chinese-2014-12-31-oroginal-kinect-discontinued.html.
[8]Microsoft to debut new Kinect for Windows this summer，
https://www.geekwire.com/2014/kinect-windows-arriving-summer/.
[9]快速入門：設定 Azure Kinect 全身追蹤，
https://docs.microsoft.com/zh-tw/azure/kinect-dk/body-sdk-setup.
[10]D. Roetenberg, M. Baten and P. Veltink, “Estimating body segment orientation by applying inertial and magnetic sensing near ferromagnetic materials,” IEEE Trans. on Neural Systems and Rehabilitation Engineering, vol. 15, no. 3, pp.469-471, Sep. 2007.
[11]B. Yang, Z. Wang, W. Ran, H. Lu and Y. Chen, “Multi-directional convolution networks with spatial-temporal feature pyramid module for action recognition,” in Proc. of ICASSP 2021 - 2021 IEEE Int. Conf. on Acoustics, Speech and Signal Processing, Toronto, Canada, pp.2440-2444, Jun. 2021.
[12]J. Jun, J. Lee, C. Lee and C. Kim, “Monocular human depth estimation via pose estimation,” IEEE Access, vol. 9, pp.151444-151457, Nov. 2021.
[13]GitHub - google-coral/project-posenet: Human Pose Detection on EdgeTPU，
https://github.com/google-coral/project-posenet
[14]Z. Cao, G. Hidalgo, T. Simon, S. Wei and Y. Sheikh, “OpenPose: realtime multi-person 2d pose estimation using part affinity fields,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 43, no. 1, pp.172-186, Jan. 2021.
[15]Y. Li, D. Yang, Y. Chen, C. Peng, Z. Sun and L. Jiao, “A lightweight top-down multi-person pose estimation method based on symmetric transformation and global matching,” IEEE Access, vol. 10, pp.22112-22122, Feb. 2022.
[16]J. Wu and H. J. Lee, “Partitioned centerpose network for bottom-up multi-person pose estimation,” in Proc. of IEEE Int. Conf. on Image Processing (ICIP), Anchorage, AK, USA, pp.914-918, Sep. 2021.
[17]N. C. Beaulieu and Q. Guo, “Novel estimator for the location parameter of the generalized Gaussian distribution,” IEEE Communications Letters, vol. 16, no. 12, pp.2064-2067, Dec. 2012.
[18]Z. Huang, W. Li, X. Xia and R. Tao, “A general Gaussian heatmap label assignment for arbitrary-oriented object detection,” IEEE Trans. on Image Processing, vol. 31, pp.1895-1910, Feb. 2022.
[19]論文閱讀筆記 — 人體姿態辨識 OpenPose，
https://williamchiu0127.medium.com/%E8%AB%96%E6%96%87%E9%96%B1%E8%AE%80%E7%AD%86%E8%A8%98-%E4%BA%BA%E9%AB%94%E5%A7%BF%E6%85%8B%E8%BE%A8%E8%AD%98-openpose-f8c2d5d651e2.
[20]S. Hsiao and B. Tsai, “Efficient computation of depthwise separable convolution in MoblieNet deep neural network models,” in Proc. of IEEE Int. Conf. on Consumer Electronics-Taiwan (ICCE-TW), Penghu, Taiwan, pp.1-2, Sep. 2021.
[21]Y. Jia, W. Miao, C. Jiang and W. Ye, “An improved pointwise convolutional block for efficient model compression,” in Proc. of IEEE 10th Int. Conf. on Software Engineering and Service Science (ICSESS), Beijing, China, pp.24-28, Oct. 2019.
[22]X. Zheng, S. Zhang, X. Li, G. Li and X. Li, “Lightweight bridge crack detection method based on SegNet and bottleneck depth-separable convolution with residuals,” IEEE Access, vol. 9, pp.161649-161668, Dec. 2021.
[23]M. A. Tahir, S. Wiesler, R. Schlüter and H. Ney, “Investigation of mixture splitting concept for training linear bottlenecks of deep neural network acoustic models,” in Proc. of IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), Brisbane, Australia, pp.4614-4618, Apr. 2015.
[24]Y. Li, Y. Li, W. He, W. Shi, T. Wang and Y. Li, “SE-OHFM: a surgical phase recognition network with SE attention module,” in Proc. of Int. Conf. on Electronic Information Engineering and Computer Science (EIECS), Changchun, China, pp.608-611, Sep. 2021.
[25]MobileNet-V3 | GitHub，
https://ustccoder.github.io/2020/03/26/feature_extraction%20MobileNet_V3/.
[26]W. Muhammad and A. Ahsan, “Airship aerodynamic model estimation using unscented Kalman filter,” Journal of Systems Engineering and Electronics, vol. 31, no. 6, pp.1318-1329, Dec. 2020.
[27]T. Sun, H. Liu, H. Yu and C. Chen, “Degree-pruning dynamic programming approaches to central time series minimizing dynamic time warping distance,” IEEE Trans. on Cybernetics, vol. 47, no. 7, pp.1719-1729, Jul. 2017.
[28]Q. Wei, D. Liu and H. Lin, “Value iteration adaptive dynamic programming for optimal control of discrete-time nonlinear systems,” IEEE Trans. on Cybernetics, vol. 46, no. 3, pp.840-853, Mar. 2016.
[29]Dynamic Time Warping，
http://mirlab.org/jang/books/dcpr/dpDtw.asp?title=84%20Dynamic%20Time%20Warping&language=chinese.
[30]A. Yaganteeswarudu, “Multi disease prediction model by using machine learning and Flask API,” in Proc. of 5th Int. Conf. on Communication and Electronics Systems (ICCES), Coimbatore, India, pp.1242-1246, Jun. 2020.
[31]G. Parisi, E. Strahl and S. Wermter, “Robust fall detection with an assistive humanoid robot,” in Proc. of IEEE-RAS Int. Conf. on Humanoid Robots, Madrid, Spain, pp.1013-1013, Nov. 2014.
[32]Flashforge Dreamer 3D Printer – FlashForge，
https://www.flashforge.com/product-detail/flashforge-dreamer-3d-printer.
[33]SKYMAKER 8系列3D列印機，
http://www.skymaker.com.cn/8_cn.php.
[34]Intel® RealSense™ Depth Camera D435i，
https://www.intelrealsense.com/depth-camera-d435i/.
[35]Y. Lai and Y. Lin, “Assessment of pulse-width modulation techniques for brushless DC motor drives,” in Proc. of Conf. Record of the IEEE Industry Applications Conference Forty-First IAS Annual Meeting, Tampa, FL, USA, pp.1629-1636, Oct. 2006.
[36]Arduino Mega 2560 概述，
https://www.arrow.com/zh-cn/research-and-events/articles/arduino-mega-2560-overview.
[37]Ö. Kafadar, “RaspMI: raspberry pi assisted embedded system for monitoring and recording of seismic ambient noise,” IEEE Sensors Journal, vol. 21, no. 5, pp.6306-6313, Mar. 2021.
[38]Day13 認識樹莓派的GPIO接腳，
https://ithelp.ithome.com.tw/articles/10215294.
[39]D. Osokin, “Real-time 2D multi-person pose estimation on CPU: lightweight OpenPose,” arXiv:1811.12004, 2018.
[40]C. Chen, T. Su, G. Meng, Z. Xing and Y. Liu, “From UI design image to GUI skeleton: a neural machine translator to bootstrap mobile GUI implementation,” in Proc. of IEEE/ACM 40th Int. Conf. on Software Engineering (ICSE), Gothenburg, Sweden, pp.665-676, Jun. 2018.
[41]Z. Zhang, J. Tang and G. Wu, “Lightweight human pose estimation under resource-limited scenes,” in Proc. of ICASSP 2021 IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), Toronto, Canada, pp.2170-2174, Jun. 2021.
[42]L. Shao, F. Zhu and X. Li, “Transfer learning for visual categorization: a survey,” IEEE Trans. on Neural Networks and Learning Systems, vol. 26, no. 5, pp.1019-1034, May 2015.
[43]M. Li, F. Wei, Y. Li, S. Zhang and G. Xu, “Three-dimensional pose estimation of infants lying supine using data from a Kinect sensor with low training cost,” IEEE Sensors Journal, vol. 21, no. 5, pp.6904-6913, Mar. 2021
[44]L. Zhao, J. Xu, C. Gong, J. Yang, W. Zuo and X. Gao, “Learning to acquire the quality of human pose estimation,” IEEE Trans. on Circuits and Systems for Video Technology, vol. 31, no. 4, pp.1555-1568, Apr. 2021.
[45]S. Wen and W. Dang, “Research on base64 encoding algorithm and PHP implementation,” in Proc. of 26th Int. Conf. on Geoinformatics, Kunming, China, pp.1-5, Jun. 2018.