臺灣博碩士論文加值系統

隨著科技的進步，在人體監測技術上已從傳統的接觸式漸漸進化為非接觸式，除了遠程光電容積脈搏波描記(Remote Photoplethysmography)技術最近幾年在醫療監測和日常健康追蹤方面有了顯著的進展，另一種非侵入性的監測方法，影像心衝擊描記(Imaging Ballistocardiography)也發展的相當迅速。
因此本研究將利用上述兩種人體生理監測技術，盡可能地透過非接觸式取得病患的生理數據，架設數位攝影機擷取病患臉部影像，經由演算法預估患者心率，再搭配些許的生理監測儀器的數據，如: ECG與呼吸率。開發出一套自動評估患者疼痛系統，考量到重症患者可能因插管或是昏迷而處於無法說話的狀態，導致難以表達自身的疼痛感受，因此目標以重症加護病房病患為主要對象。使用非接觸式監測技術無須任何線材接觸患者，能有效的降低患者身體上的負擔，我們希望能以最少接觸的方式得到所需的生理數據。其中臉部影像輸入至專門訓練圖片的卷積神經網路(Residual Network)，使用殘差塊解決深度神經網絡中的梯度消失的問題，在此系統中扮演著相當重要的角色，而ECG與呼吸率則是透過分支網路訓練，再整合為最終的深度學習模型，而整合後的模型將做出疼痛等級的評估，在我們的實驗中的疼痛分類是使用CPOT分為三種等級。
本研究所設計的自動評估疼痛系統對患者或是醫療照護人員而言無疑帶來莫大的幫助，不僅能在第一時間幫助病患傳達病痛感，讓護理人員在第一時間給予照護，同時也減輕護理人員的壓力，也有助於解決現今護病比過度不均的難題。

With the advancement of technology, human monitoring techniques have gradually evolved from traditional contact methods to non-contact methods. In recent years, remote Photoplethysmography (rPPG) technology has significantly progressed in medical monitoring and daily health tracking. Another non-invasive monitoring method, imaging Ballistocardiography (iBCG), has also developed rapidly.
This study aims to use the above human physiological monitoring technologies to obtain patients' physiological data as much as possible through non-contact methods. By setting up digital cameras to capture facial images of patients and using algorithms to estimate their heart rates, supplemented with some physiological monitoring data such as ECG and respiratory rate, we aim to develop an automated pain assessment system. This system is primarily targeted at intensive care unit (ICU) patients, who may be unable to speak due to intubation or coma, making it difficult for them to express their pain. Using non-contact monitoring technology eliminates the need for wires to touch the patient, effectively reducing their physical burden. We hope to obtain the necessary physiological data with minimal contact. Our methodology feeds facial images into a Residual Network, a convolutional neural network trained explicitly on images. This network, designed to address the vanishing gradient problem in deep neural networks using residual blocks, is a crucial component of our system. ECG and respiratory rate data are processed through branch networks and then integrated into the final deep-learning model, which will evaluate the pain level. The pain classification in our experiment is divided into three levels using CPOT.
The automated pain assessment system proposed in this study has the potential to revolutionize patient care. Providing immediate and accurate pain assessment can significantly reduce the burden on nursing staff and ensure patients receive care when needed. It will also help solve the current problem of excessive unevenness in the ratio of care to patients.

中文摘要 i
Abstract ii
Contents iv
List of Figures vi
List of Tables viii
Chapter 1 Introduction 1
1.1 Remote photoplethysmography 1
1.2 Imaging ballistocardiography 2
1.3 Neural network 3
1.4 Motivation and objectives 4
1.5 Contributions 5
1.6 Brief overview 6
Chapter 2 Related Work 7
2.1 Organizing of datasets 7
2.2 Tracking region of interest 8
2.3 Extract pulse signals by using remote photoplethysmography 10
Chapter 3 Main Algorithm 15
3.1 ECG filtering algorithm 15
3.2 Extract pulse signals by using imaging ballistocardiography 21
3.2.1 Feature point detection and tracking 21
3.2.2 Adaptive filter 24
3.2.3 Heart rate signal estimation 26
3.3 Applications of deep learning neural networks 27
3.3.1 Neural network architecture for images 28
3.3.2 Neural network architecture for physiological information 33
3.3.3 Integration of models 36
Chapter 4 System Architecture 39
4.1 The architecture of iBCG 39
4.2 Multiple neural network model 41
4.3 Overall system architecture 43
Chapter 5 Experimental Result 46
5.1 Collecting data sets 46
5.2 Comparison of filters for filtering noisy ECG signals 46
5.3 Heart rate estimation with iBCG and rPPG 47
5.4 Training and testing results 49
Chapter 6 Conclusions 54
References 56

[1]Bin Huang, Weihai Chen, Chun-Liang Lin, Chia-Feng Juang, Yuanping Xing, Yanting Wang, Jianhua Wang, "A neonatal dataset and benchmark for non-contact neonatal heart rate monitoring based on spatio-temporal neural networks, " Engineering Applications of Artificial Intelligence. vol. 106, 2021.
[2]Poh, M. Z., McDuff, D. J., & Picard, R. W. (2010). "Non-contact, Automated Cardiac Pulse Measurements Using Video Imaging and Blind Source Separation". Optics express, 18(10), 10762-10774.
[3]Xu, S., Sun, L., & Rohde, G. K. (2014). "Robust efficient estimation of heart rate pulse from video". Biomedical optics express, 5(4), 1124-1135.
[4]Wu Chieh-Liang , Liu Shu-Fang , Yu Tian-Li , Shih Sou-Jen , Chang Chih-Hung , Yang Mao Shih-Fang , Li Yueh-Se , Chen Hui-Jiun , Chen Chia-Chen , Chao Wen-Cheng "Deep Learning-Based Pain Classifier Based on the Facial Expression in Critically Ill Patients" Frontiers in Medicine, 2296-858X, 2022.
[5]T. Adjei, W. Von Rosenberg, V. Goverdovsky, K. Powezka, U. Jaffer and D. P. Mandic, "Pain Prediction From ECG in Vascular Surgery," IEEE Journal of Translational Engineering in Health and Medicine, vol. 5, pp. 1-10, Art no. 2800310, 2017.
[6]M. Jiang, R. Mieronkoski, A. M. Rahmani, N. Hagelberg, S. Salanterä and P. Liljeberg, "Ultra-short-term analysis of heart rate variability for real-time acute pain monitoring with wearable electronics", IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Kansas City, MO, USA, pp. 1025-1032, 2017.
[7]R. Logier, M. Jeanne, B. Tavernier and J. De jonckheere, "Pain / Analgesia evaluation using heart rate variability analysis," International Conference of the IEEE Engineering in Medicine and Biology Society, New York, USA, pp. 4303-4306, 2006.
[8]R. Cao, S. A. H. Aqajari, E. Kasaeyan Naeini and A. M. Rahmani, "Objective Pain Assessment Using Wrist-based PPG Signals: A Respiratory Rate Based Method," Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Mexico, pp. 1164-1167, 2021.
[9]Google, "Mediapipe," 2021. [Online]. Available: https://mediapipe.dev. [Accessed: 26-Jun-2024].
[10]W. Wang, A. C. den Brinker, S. Stuijk and G. de Haan, "Algorithmic Principles of Remote PPG," in IEEE Transactions on Biomedical Engineering, vol. 64, no. 7, pp. 1479-1491, July 2017.
[11]Nitzan M, Noach S, Tobal E, Adar Y, Miller Y, Shalom E, Engelberg S. “Calibration-free pulse oximetry based on two wavelengths in the infrared - a preliminary study.” Sensors (Basel). 2014 Apr 23;14(4):7420-34.
[12]Prahl S. "The absorption spectra of the oxygenated and deoxygenated hemoglobin molecules". graph. [(accessed on 21 January 2014)]. Available online: http://omlc.ogi.edu/spectra/hemoglobin/index.html
[13]Wang, “Robust and automatic remote photoplethysmography,” Ph.D. dissertation, Electrical Engineering, Oct. 2017, proefschrift.
[14]G. de Haan and V. Jeanne, "Robust Pulse Rate From Chrominance-Based rPPG," IEEE Transactions on Biomedical Engineering, vol. 60, no. 10, pp. 2878-2886. Oct, 2013.
[15]X. Niu, H. Han, S. Shan and X. Chen, "Continuous heart rate measurement from face: A robust rPPG approach with distribution learning," 2017 IEEE International Joint Conference on Biometrics (IJCB), Denver, CO, USA, 2017, pp. 642-650.
[16]Á. Fehér, "Denoising ECG signals by applying discrete wavelet transform," 2017 International Conference on Optimization of Electrical and Electronic Equipment (OPTIM) & 2017 Intl Aegean Conference on Electrical Machines and Power Electronics (ACEMP), Brasov, Romania, 2017, pp. 863-868, doi: 10.1109/OPTIM.2017.7975078.
[17]Birle, A., Malviya, S. & Malviya, D (2015) 'Noise Removal in ECG Signal Using Savitzky-Golay Filter', IJARECE, 4(5), pp. 2278 909X.
[18]M. T. Almalchy, V. Ciobanu and N. Popescu, "Noise Removal from ECG Signal Based on Filtering Techniques," 2019 22nd International Conference on Control Systems and Computer Science (CSCS), Bucharest, Romania, 2019, pp.
[19]M. A. Hassan, A. S. Malik, D. Fofi, N. M. Saad, Y. S. Ali and F. Meriaudeau, "Video-Based Heartbeat Rate Measuring Method Using Ballistocardiography," IEEE Sensors Journal, vol. 17, no. 14, pp. 4544-4557, 15 July15, 2017.
[20]M. A. Haque, R. Irani, K. Nasrollahi and T. B. Moeslund, "Heartbeat Rate Measurement from Facial Video," IEEE Intelligent Systems, vol. 31, no. 3, pp. 40-48, May-June 2016.
[21]R. Song, J. Li, J. Cheng, C. Li, Y. Liu and X. Chen, "Motion Robust Imaging Ballistocardiography Through a Two-Step Canonical Correlation Analysis," IEEE Transactions on Instrumentation and Measurement, vol. 70, pp. 1-10, Art no. 4001710, 2021.
[22]Juan Cheng, Bicheng Yue, Rencheng Song, Yu Liu, Chang Li, Xun Chen, "Motion-robust anterior–posterior imaging ballistocardiography for non-contact heart rate measurements", Biomedical Signal Processing and Control, Volume 86, Part C, 2023.
[23]B. T. Susam et al., "Automated Pain Assessment in Children Using Electrodermal Activity and Video Data Fusion via Machine Learning," in IEEE Transactions on Biomedical Engineering, vol. 69, no. 1, pp. 422-431, Jan. 2022.
[24]T. N. V. S. Praveen, D. Sivathmika, G. Jahnavi and J. Bolledu, "An In-depth Exploration of ResNet-50 for Complex Emotion Recognition to Unraveling Emotional States," 2023 International Conference on Advancement in Computation & Computer Technologies (InCACCT), Gharuan, India, 2023.
[25]E. D and N. P. G. Bhavani, "An Effective DNN Based ResNet Approach for Satellite Image Classification," 2023 4th International Conference on Smart Electronics and Communication (ICOSEC), Trichy, India, 2023, pp. 1055-1062,
[26]S. Azhaginiyan, M. Anish, M. K. Shivaranjan and M. Ganesan, "Denoising of BCG Signal using Multi Resolution Analysis," 2019 5th International Conference on Advanced Computing & Communication Systems (ICACCS), Coimbatore, India, 2019, pp. 1005-1008, doi: 10.1109/ICACCS.2019.8728549.
[27]S. Reznichenko and S. Zhou, "Optimization of Arrhythmia-based ECG-lead Selection for Computer-interpreted Heart Rhythm Classification," 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Sydney, Australia, 2023, pp. 1-4, doi: 10.1109/EMBC40787.2023.10340738.
[28]X. Liu et al., "Right precordial leads V4R and V5R in ECG detection of acute ST elevation mi associated with proximal right coronary artery occlusion," Computers in Cardiology, Lyon, France, 2005, pp. 651-654, doi: 10.1109/CIC.2005.1588185.
[29]S. Saxena, R. Jais and M. K. Hota, "Removal of Powerline Interference from ECG Signal using FIR, IIR, DWT and NLMS Adaptive Filter," 2019 International Conference on Communication and Signal Processing (ICCSP), Chennai, India, 2019, pp. 0012-0016, doi: 10.1109/ICCSP.2019.8698112.
[30]Z. Shang, Y. Zhao and Y. Lian, "Removal of Baseline Wander Noise in ECG Signal Using Asymmetrical Frequency-Response Masking Bandpass Filters," 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu, HI, USA, 2018, pp. 6002-6005, doi: 10.1109/EMBC.2018.8513611.
[31]N. Sultana and Y. Kamatham, "Mitigation of noise and interference in ECG signals with Savitzky-Golay least squares polynomials and Discrete Wavelet Transform," 2015 IEEE International Conference on Electrical, Computer and Communication Technologies (ICECCT), Coimbatore, India, 2015, pp. 1-5, doi: 10.1109/ICECCT.2015.7226125.