臺灣博碩士論文加值系統

跌倒一直是老年人死亡的首要原因，在全球人口高齡化和出生率下降的時 代，護理人員的短缺嚴重影響了老年人的健康。如果可以利用資訊和通信技 術，對老年人跌倒進行自動檢測和識別，我們相信它可以減少老年人因跌倒 造成的傷害。本文提出了一種有別於以往穿戴式感測器的方法，是基於圖像 中人體相對位置的位移參數來識別人體墜落的發生。我們實現了一個基於 OpenPose 的系統，並結合具有時間序列深度學習神經網路模型 LSTM 進行 影像識別，透過影像中人體墜落和跌倒的關節位移參數，並對識別出的參數 進行簡單過濾，然後將過濾後的參數用於模型訓練，再將過濾後的參數用於 模型的訓練，最後將模型部署到邊緣設備。此外，還會對模型的效能及邊緣 運算進行可行性評估。

Falls are consistently the top cause of death among seniors. At a time when the global population is getting older and fewer births. The shortage of nursing staff seriously affects the health care of the elderly. If information and communication technology can be used, automatic detection and identification the elderly fall, we believe it can reduce the injury of the elderly due to falls. This paper proposes a method different from the previous wearable sensing device, which is based on the displacement of human relative positional parameters in the image to identify the occurrence of human fall. We implemented a system based on OpenPose and combined with the deep learning neural network model LSTM with time series, the image recognition is carried out, the human joint parameters of human posture falling and falling in the image are captured, and the identified parameters are simply filtered, and then the filtered parameters are used for model training then used the filtered parameters for model training, and finally deploy the model to the edge device. Also, reliability evaluation will be made for the performance of model and edge computing.

摘要 i
Abstract ii
目錄 iii
表目錄 v
圖目錄 vi
第一章、緒論 1
1.1 研究背景 1
1.2 研究動機 3
1.3 研究目的 4
1.4 論文架構 5
第二章、相關研究 6
2.1 卷積神經網路 6
2.2 遞迴神經網路 7
2.3 長短期記憶 7
2.4 OpenPose人體骨架識別 11
2.5 TensorRT 12
2.6 NVIDIA JETSON 13
2.7 Pytorch 13
第三章、系統設計與方法 14
3.1 模型訓練流程 14
3.2 資料集蒐集與製作 16
3.3 OpenPose人體骨架及關節點取得之方法 17
3.4 關節點的數據預處理 18
3.4.1 關節點的過濾 19
3.4.2 關節點正規化 20
3.4.3 關節點缺失值補償 21
3.4.4 資料集擴增 23
3.4.5 增加資料集雜訊 24
3.5 TensorRT對模型最佳化策略 24
第四、實驗結果與分析 26
4.1 模型架構及資料集選用 27
4.1.1 訓練環境及參數設定 29
4.1.2 訓練模型結構參數 30
4.2資料集訓練效能比較 32
4.2.1 根據影像長寬對關節點位置正規化比較 32
4.2.2 使用關節點遺失補償比較 35
4.2.3 使用關節點增加雜訊比較 39
4.3實驗成果與分析 43
4.3.1 模型選用LSTM 43
4.3.2 識別參數設定 44
4.3.3 模型轉換TensorRT 44
4.3.4 邊緣裝置效能及模型參數的選擇 45
第五章、結論與未來展望 48
參考文獻 49

中文部分:
國家發展委員會，中華民國人口推估(2020 至 2070年)，國家發展委員會https://pop-proj.ndc.gov.tw/download.aspx?uid=70&pid=70 , (2020).
李美慧。從世界人口展望探討日本情勢及其因應政策。國家實驗研究院科技政策研究與資訊中心, (2019).
英文部分:
United Nations, Department of Economic and Social Affairs, Population Division. World Population Prospects 2019: Highlights, (ST/ESA/SER.A/423), (2019).
C. Jay, et al. "Wearable sensors for reliable fall detection." 2005 IEEE engineering in medicine and biology 27th annual conference, (2006).
L. Dima, Y. Zigel, I. Gannot. "Fall detection of elderly through floor vibrations and sound." 2008 30th annual international conference of the IEEE engineering in medicine and biology society, (2008).
M. Georgios, D. Makris. "Fall detection system using Kinect’s infrared sensor." Journal of Real-Time Image Processing, (2014).
Y. L. Cun, Y. Bengio. "Convolutional Networks for Images, Speech, and Time-Series. Cambridge.", MA: MIT Press, Pages 255-258 (1995).
Krizhevsky, I. Sutskever, G. Hinton. "ImageNet Classification with Deep Convolutional Neural Networks.", Advances in Neural Information Processing Systems, Volume 1, Pages 1097-1105 (2012).
K. He, X. Zhang, S. Ren, J. Sun. "Deep Residual Learning for Image Recognition.", IEEE Conference on Computer Vision and Pattern Recognition, Pages 770-778(2016).
M. Tomas, et al. "Recurrent Neural Network Based Language Model." Eleventh annual conference of the international speech communication association, Volume 2, No. 3, Pages 1045-1048 (2010).
K. Jun, D. Lee, K. Lee, S. Lee, M. S. Kim, "Feature Extraction Using an RNN Autoencoder for Skeleton-Based Abnormal Gait Recognition.", IEEE Access, Volume 8, Pages 19196-19207 (2020).
S. Hochreiter, J. Schmidhuber. "Long Short-Term Memory Neural Computation", Volume 9, Issue 8, Pages 1735-1780 (1997).
S. Selvin, R. Vinayakumar, E. A. Gopalakrishnan, V. K. Menon, K. P. Soman. "Stock price prediction using LSTM, RNN and CNN-sliding window model.", 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI), Pages 1643-1647 (2017).
A. Graves, N. Jaitly and A. Mohamed, "Hybrid speech recognition with Deep Bidirectional LSTM.", 2013 IEEE Workshop on Automatic Speech Recognition and Understanding, Pages 273-278 (2013).
Z. Cao, G. Hidalgo, T. Simon, S. E. Wei, Y. Sheikh, "OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields.", in IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 43, no 1, Pages 172-186 (2021).
Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, T. Darrell. "Caffe: Convolutional Architecture for Fast Feature Embedding.", arXiv preprint arXiv:1408.5093 (2014).
K. Simonyan and A. Zisserman. "Very deep convolutional networks for large-scale image recognition" arXiv preprint arXiv:1409.1556 (2014).
NVIDIA TensorRT. 2021. https://developer.Nvidia.com/tensorrt (2021)
S. Yegulalp, "Facebook brings GPU-powered machine learning to Python.", InfoWorld 19 (2017).
A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, A. Lerer. "Automatic differentiation in PyTorch.". In NIPS Workshop (2017) .
W. H. Chen. "Data Augmentation for Human Activity Recognition.", in Bulletin of College of Engineering National Ilan University, No.13, Pages 12-29 (2019).
E. Insafutdinov, L. Pishchulin, B. Andres, M. Andriluka, B. Schiele, " A deeper, stronger, and faster Multi-Person Pose Estimation Model.", in European Conference on Computer Vision. Springer, Pages 34–50 (2016).
P. C. Chen, C. H. Chang, Y. W. Chan, Y. T. Tasi, W. C. Chu, " An Approach to Real-Time Fall Detection based on OpenPose and LSTM.", CompSAC 2022, Pages 1573-1578 (2022).
L. C. Bi, et al. "A Framework for Fall Detection Based on OpenPose Skeleton and LSTM/GRU Models.", Applied Sciences (2021).
P. Svarny, Z. Straka, M. Hoffmann, "Toward Safe Separation Distance Monitoring From RGB-D Sensors In Human-Robot Interaction.", International PhD Conference on Safe and Social Robotics (SSR2018), Pages 11-14 (2018).
T. Kohei, M. Kono, J. Rekimoto, "Post-data augmentation to improve deep pose estimation of extreme and wild motions.", 2019 IEEE Conference on Virtual Reality and 3D User Interfaces, (2019).
B. Kwolek, M. Kepski, "Human Fall Detection on Embedded Platform Using Depth Maps and Wireless Accelerometer.", Computer Methods and Programs in Biomedicine, Volume 117, Pages 489-501 (2014).
Onnx. [Online]. Available: https://github.com/onnx/onnx (2017).
H. Vanholder, "Efficient inference with tensorrt." GPU Technology Conference, Volume 1 (2016).
NVIDIA TensorRT Developer Guide demonstrates https://docs.Nvidia.com/deeplearning/tensorrt/developer-guide. (2022).