跳到主要內容

臺灣博碩士論文加值系統

(44.200.194.255) 您好!臺灣時間:2024/07/24 05:21
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果 :::

詳目顯示

我願授權國圖
: 
twitterline
研究生:顏伯融
研究生(外文):Yan, Bo-Rong
論文名稱:基於深度學習的遠程光電容積脈搏波(rPPG)開發平台
論文名稱(外文):DLPrPPG-Development and Design of Deep Learning Platform for rPPG
指導教授:賴伯承
指導教授(外文):Lai, Bo-Cheng
口試委員:李文賢帥宏翰賴伯承
口試委員(外文):LEE, Wen-HsienSHUAI, HONG-HANLai, Bo-Cheng
口試日期:2021-12-07
學位類別:碩士
校院名稱:國立陽明交通大學
系所名稱:電子研究所
學門:工程學門
學類:電資工程學類
論文種類:學術論文
論文出版年:2021
畢業學年度:110
語文別:中文
論文頁數:52
中文關鍵詞:生理訊號深度學習機器學習訊號轉換心律估計
外文關鍵詞:rPPGBio-signalDeep learningMachine learningSignal transformHeart rate estimation
相關次數:
  • 被引用被引用:0
  • 點閱點閱:859
  • 評分評分:
  • 下載下載:0
  • 收藏至我的研究室書目清單書目收藏:0
rPPG 是一個不斷發展和流行的研究領域,特別是隨著深度學習方法的引入可以顯著提高其信號質量和心率預測可靠性。然而,目前研究中的實驗方法還存在很多問題,例如非標準化和私有數據,不同的預處理方法,和不完整或不可重複的實驗方法等等,這些問題讓相關方法無法公平的比較,導致所提出的實驗數據可靠性降低,阻礙了該領域的進展。
基於這些原因,本文提出了一個基於深度學習的遠程光電容積脈搏波(rPPG)開發平台。這是一個開放的平台框架,以促進基於 rPPG 的深度學習神經網路開發的設計和實驗。在我們的平台中提供現成的CNN Auto-Encoder, LSTM, GAN, 及Transformer等神經網路,並固定神經網路之前的信號處理部分,讓所有的實驗都能有一個公平的參考點,避免之前研究中不同實驗方法和數據帶來的誤差,針對不同的參數及架構優化不同神經網路,從而克服某些網路必然比其他網路更好的盲點,避免產生一些毫無根據和未經驗證的結論。
通過我們的平台可以快速且合理地選擇最佳的參數組合。例如,基於 CNN 的
Auto-Encoder,我們比較不同的kernel width,結果Signal MAE 的差異從 0.209 到
0.190,HR MAE 則為 33.127 到 2.405。這在其他研究中很少提及,或者只比較了幾個特定的項目,且較難去比較。但是通過我們的平台,可以很容易地觀察到各種參數組合所造成的差異性。我們也在我們的平台中證實,若不同神經網路皆有做參數優化,較舊的架構其效能可以與新架構相提並論,甚至優於新架構。
This paper presents a comprehensive neural network-based development platform for remote photoplethysmography (rPPG). rPPG is a growing and popular research area, especially with the introduction of deep learning methods that can significantly improve its signal quality and heart rate prediction reliability. However, there are still many problems with the experimental methods in current studies, such as non-standardized and private data, different pre-processing methods, and incomplete or irreproducible experiment methodologies, among others. These problems prevent methods from being compared fairly and lead to lower reliability of the proposed experimental results, hindering progress in this area.

For these reasons, we propose an open-source framework to facilitate the design and experimentation of deep learning based rPPG development, and it’s made freely available on GitHub. We provide ready to-use implementations of CNN-AE, LSTM, GAN, and Transformer models in our platform, and we fix the signal processing part before the neural network, so that all experiments can have a fair reference point, avoid errors caused by different experimental methods and data in the previous research, and optimize different neural networks for different parameters and architectures. It can overcome the blind spots of certain networks is necessarily better than others, leading to unfounded and unproven conclusions.

Through the platform, the best parameter combination can be selected quickly and reasonably. For example, comparing different kernel widths for CNN based Auto-Encoder, the difference of resulting signal MAE is from 0.209 to 0.190, and the HR MAE is from 33.127 to 2.405. This is rarely mentioned in other studies, or only a few specific items are compared, but with our platform, the differences in various combinations can be easily observed. We also show that if the parameters of different neural networks are optimized, the performance of older architectures can be on par or even outperform newer ones.
摘要 i
Abstract ii
致謝 iii
Contents iv
List of Figures v
List of Table vi
Chapter 1 Introduction 1
Chapter 2 Related Work 4
2.1 Traditional rPPG Algorithm 4
2.2 DL-based Methods 5
Chapter 3 Framework and Architecture 9
3.1 Face Detection and ROI Selection 9
3.2 Signal Preprocessing and rPPG Extraction 11
3.3 DL Network 13
3.3.1 Denoising Auto-Encoder 14
3.3.2 LSTM 15
3.3.3 Generative Adversarial Networks 17
3.3.4 Transformer 20
3.4 Loss Functions and Normalization Methods 22
Chapter 4 Experiment Methodology 24
4.1 Database 24
4.2 Experiment Setup 24
4.2.1 Raw Data Setting 24
4.2.2 Network Training Setup 24
4.3 Metrics 25
4.4 Experimental Results 26
4.4.1 Parametric Study 26
4.4.2 Model Comparison 33
Chapter 5 Conclusion 38
Reference 39
Appendix 1 44
Appendix 2 48
[1]G. Cennini et al., “Heart rate monitoring via remote photoplethysmography with motion artifacts reduction,” Opt. Exp., vol. 18, no. 5, pp. 4867–4875, 2010.
[2]Y. Sun and N. Thakor, “Photoplethysmography revisited: From contact to noncontact, from point to imaging,” IEEE Trans. Biomed. Eng., vol. PP, no. 99, p. 1, 2015, doi: 10.1109/TBME.2015.2476337.
[3]] W. Verkruysse, L. O. Svaasand, and J. S. Nelson, “Remote plethysmographic imaging using ambient light,” Opt. Express, vol. 16, no. 26, pp. 21434–21445, 2008.
[4] W. Wang et al., “Exploiting spatial redundancy of image sensor for motion robust rPPG,” IEEE Trans. Biomed. Eng., vol. 62, no. 2, pp. 415– 425, Feb. 2015.
[5]W. Chen and D. Mcduff, “DeepPhys: Video-based physiological measurement using convolutional attention networks,” Proc. ECCV, 2018, pp. 349–365.
[6]D. Biswas et al., “CorNET: Deep learning framework for PPG-based heart rate estimation and biometric identification in ambulant environment,” IEEE Trans. Biomed. Circuits Syst., vol. 13, no. 2, pp. 282–291, Apr. 2019.
[7]R. Song, H. Chen, J. Cheng, C. Li, Y. Liu, and X. Chen, “PulseGAN: Learning to generate realistic pulse waveforms in remote photoplethysmography,” IEEE J. Biomed. Health Informat., early access, Jan. 12, 2021, doi: 10.1109/JBHI.2021.3051176.
[8]R. Song, S. Zhang, C. Li, Y. Zhang, J. Cheng, and X. Chen, “Heart rate estimation from facial videos using a spatiotemporal representation with convolutional neural networks,” IEEE Trans. Instrum. Meas., vol. 69, no. 10, pp. 7411–7421, Oct. 2020.
[9]G. Slapniˇcar, E. Dovgan, P.Cuk, and M. Luštrek, “Contact-free monitoring of physiological parameters in people with profound intellectual and multiple disabilities,” in Proc. IEEE Int. Conf.Comput. Vis. Workshops, 2019, pp. 1–9.
[10]Botina-Monsalve, D., et al.: Long short-term memory deep-filter in remote photoplethysmography. In: IEEE CVPRW, pp. 306–307 (2020)
[11]Zhan, Q.; Wang, W.; de Haan, G. Analysis of CNN-based remote-PPG to understand limitations and sensitivities. Biomed. Opt. Express 2020, 11, 1268–1283.
[12]Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. arXiv preprint arXiv:1706.03762, 2017.
[13]Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient Transformer for long sequence time-series forecasting. arXiv preprint arXiv:2012.07436, 2020.
[14]Shiyang Li, Xiaoyong Jin, Yao Xuan, Xiyou Zhou, Wenhu Chen, YuXiang Wang, and Xifeng Yan. Enhancing the locality and breaking the memory bottleneck of Transformer on time series forecasting. In Advances in Neural Information Processing Systems 32, pages 5244– 5254. Curran Associates, Inc., 2019.
[15]Rewon Child, Scott Gray, Alec Radford, and Ilya Sutskever. Generating long sequences with sparse Transformers. arXiv preprint arXiv:1904.10509, 2019.
[16]Bobbia, S.; Macwan, R.; Benezeth, Y.; Mansouri, A.; Dubois, J. Unsupervised skin tissue segmentation for remote photoplethysmography. Pattern Recognit. Lett. 2017.
[17]W. Wang, A. den Brinker, S. Stuijk, and G. de Haan, ‘‘Algorithmic principles of remote-PPG,’’ IEEE Trans. Biomed. Eng., vol. 64, no. 7, pp. 1479–1491, Jul. 2016.
[18]E. A. Rios, C. -C. Lai, B. -R. Yan and B. -C. Lai, "Parametric Study of Performance of Remote Photopletysmography System," 2021 IEEE International Symposium on Circuits and Systems (ISCAS), 2021, pp. 1-4, doi: 10.1109/ISCAS51556.2021.9401620.
[19]Boccignone, G., Conte, D., Cuculo, V., D’Amelio, A., Grossi, G., & Lanzarotti, R. (2020). An Open Framework for Remote-PPG Methods and Their Assessment. IEEE Access, 8,
216083–216103.
[20] M. Bian, B. Peng, W. Wang, and J. Dong, “An accurate LSTM based video heart rate estimation method,” in Pattern Recognit. Comput. Vis., 2019, pp. 409–417.
[21] M. Z. Poh, D. J. McDuff, and R. W. Picard, “Non-contact, automated cardiac pulse measurements using video imaging and blind source separation,” Opt. Expr., vol. 18, pp. 10762–10774, May 2010.
[22]M. Lewandowska, J. Ruminski, T. Kocejko, and J. Nowak, “Measuring pulse rate with a webcam—A non-contact method for evaluating cardiac activity,” in Proc. Federated Conf. Comput. Sci. Inform. Syst., 2011, pp. 405–410.
[23] G. De Haan and V. Jeanne, “Robust pulse rate from chrominance-based rPPG,” IEEE Trans. Biomed. Eng., vol. 60, no. 10, pp. 2878–2886, Oct. 2013.
[24]G. Heusch, A. Anjos, and S. Marcel. (2017). “A reproducible study on remote heart rate measurement.” [Online]. Available: https://arxiv. org/abs/1709.00962
[25] Q. Zhan et al., “Analysis of CNN-based remote-PPG to understand limitations and sensitivities,” Biomed. Opt. Express, vol. 11, no. 3, pp. 1268–1283, Mar. 2020.
[26]R. Spetlik et al., “Visual heart rate estimation with convolutional neural network,” in Proc. Brit. Mach. Vis. Conf., Aug. 2018, pp. 3–6.
[27] Z. Yu, W. Peng, X. Li, X. Hong, and G. Zhao, “Remote heart rate measurement from highly compressed facial videos: An end-to-end deep learning solution with video enhancement,” in Proc. IEEE Int. Conf. Comput. Vis., 2019, pp. 151–160.
[28]R. Song, S. Zhang, C. Li, Y. Zhang, J. Cheng, and X. Chen, “Heart rate estimation from facial videos using a spatiotemporal representation with convolutional neural networks,” IEEE Trans. Instrum. Meas., vol. 69, no. 10, pp. 7411–7421, Oct. 2020.
[29]Z. Yu, X. Li, and G. Zhao, “Remote photoplethysmography signal measurement from facial videos using spatio-temporal networks,” in Proc. British Mach. Vis. Conf., 2019, pp. 1–12.
[30]A. M. Unakafov, “Pulse rate estimation using imaging photoplethysmography: Generic framework and comparison of methods on a publicly available dataset,” Biomed. Phys. Eng. Exp., vol. 4, no. 4, Apr. 2018, Art. no. 045001.
[31]VIOLA, P. AND JONES, M. 2001. Rapid object detection using a boosted cascade of simple features. In Proceedings, IEEE Conference on Computer Vision and Pattern Recognition
[32] Z. Zhang, P. Luo, C. C. Loy, and X. Tang, “Facial landmark detection by deep multi-task learning,” in Proc. Eur. Conf. Comput. Vis., 2014, pp. 94– 108.
[33]Richard Macwan et al. “Periodic Variance Maximization Using Generalized Eigenvalue Decomposition Applied to Remote Photoplethysmography Estimation”. In: CVPRW. IEEE, 2018.
[34]C. S. Pilz, S. Zaunseder, J. Krajewski, and V. Blazek, ‘‘Local group invariance for heart rate estimation from face videos in the wild,’’ in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), Jun. 2018, pp. 1254–1262.
[35] R. Stricker, S. Mller, and H. Gross, “Non-contact video-based pulse rate measurement on a mobile service robot,” in Proc. 23rd IEEE Int. Symp. Robot Hum. Interactive Commun., 2014, pp. 1056–1062.
[36]D. J. McDuff, E. B. Blackford, and J. R. Estepp, “The impact of video compression on remote cardiac pulse measurement using imaging photoplethysmography,” in Proc. 12th IEEE Int. Conf. Autom. Face Gesture Recognit. (FG), Jun. 2017, pp. 63–70.
[37] M. Soleymani, J. Lichtenauer, T. Pun, and M. Pantic, “A multimodal database for affect recognition and implicit tagging,” IEEE Trans. Affect. Comput., vol. 3, no. 1, pp. 42–55, Jan.–Mar. 2012.
[38]R. Stricker, S. Miller, and H. Gross, “Non-contact video-based pulse rate measurement on a mobile service robot,” in Proc. 23rd IEEE Int. Symp. Robot Hum. Interactive Commun., 2014, pp. 1056–1062.
[39]https://github.com/prouast/heartbeat
[40]P. Rouast, M. Adam, V. Dorner, and E. Lux, “Remote photoplethysmography: Evaluation of contactless heart rate measurement in an information systems setting,” in Proc. Appl. Inform. Technol. Innov. Conf., 2016, p. 1.
[41] P. V. Rouast, M. T. P. Adam, D. J. Cornforth, E. Lux, and C. Weinhardt, “Using contactless heart rate measurements for real-time assessment of affective states,” in Information Systems and Neuroscience. Cham, Switzerland: Springer, 2017, pp. 157–163.
[42]W. Liu, D. Anguelov, D. Erhan, C. Szegedy, and S. Reed, “SSD: Single shot multibox detector,” arXiv:1512.02325, 2015.
[43]https://docs.opencv.org/4.5.1/d9/df8/tutorial_root.html
[42]J. K. Kim and J. M. Ahn, ‘‘Design of an optimal digital IIR filter for heart rate variability by photoplethysmogram,’’ Int. J. Eng. Res. Technol., vol. 11, no. 12, pp. 2009–2021, 2018.
[43]M. P. Tarvainen, P. O. Ranta-aho, and P. A. Karjalainen, ‘‘An advanced detrending method with application to HRV analysis,’’ IEEE Trans. Biomed. Eng., vol. 49, no. 2, pp. 172–175, Feb. 2002.
[44]Chiu H. Y., Shuai H. H., Chao P. C. P. (2020). Reconstructing QRS complex from PPG by transformed attentional neural networks. IEEE Sens. J. 20, 12374–12383.
[45] S. Pascual, A. Bonafonte, and J. Serrá, “SEGAN: Speech enhancement generative adversarial network,” in Proc. Interspeech, Aug. 2017, pp. 3642–3646.
[46]A. Radford, L. Metz, and S. Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434, 2015.
[47]K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification,” in Proc. IEEE Int. Conf. Comput. Vis., 2015, pp. 1026–1034.
[48]Y. Li, T. Bao, J. Gong, X. Shu, and K. Zhang, ‘‘The prediction of dam displacement time series using STL, extra-trees, and stacked LSTM neural network,’’ IEEE Access, vol. 8, pp. 94440–94452, 2020.
[49]Z. Cui, R. Ke, and Y. Wang. Deep bidirectional and unidirectional lstm recurrent neural network for network-wide traffic speed prediction. arXiv preprint arXiv:1801.02143, 2018.
[50]S. Siami-Namini, N. Tavakoli, and A. S. Namin, ‘‘The performance of LSTM and BiLSTM in forecasting time series,’’ in Proc. IEEE Int. Conf. Big Data (Big Data), Dec. 2019, pp. 3285–3292.
[51] GRAVES, A., MOHAMED, A.-R., AND HINTON, G. Speech recognition with deep recurrent neural networks. In 2013 IEEE international conference on acoustics, speech and signal processing (2013), IEEE, pp. 6645–6649.
[52]Goodfellow IJ, Pouget-Abadie J, Mirza M, et al (2014), “Generative adversarial networks,” Conf Neural Inf Process Syst. doi: 10.1001/jamainternmed.2016.8245
[53]M. Wiatrak, S. V. Albrecht, and A. Nystrom. Stabilizing generative adversarial networks: a survey. arXiv preprint arXiv:1910.00927, 2019.
[54]Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014.
[55]Santiago Pascual, Maruchan Park, Joan Serra, Antonio Bona- ` fonte, and Kang-Hun Ahn, “Language and noise transfer in speech enhancement generative adversarial network,” in ICASSP, 2018, pp. 5019–5023.
[56]A. Odena, C. Olah, and J. Shlens. Conditional image synthesis with auxiliary classifier gans. arXiv preprint arXiv:1610.09585, 2016.
[57]Z. Yang, W. Chen, F. Wang, and B. Xu. Improving neural machine translation with conditional sequence generative adversarial nets. arXiv preprint arXiv:1703.04887, 2017.
[58]Esteban, C., Hyland, S. L., and Ratsch, G. Real-valued ¨ (Medical) Time Series Generation with Recurrent Conditional GANs. ArXiv e-prints, June 2017.
[59] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Proc. Int. Conf. Medical Image Comput. Comput.-Assisted Intervention, 2015, pp. 234–241.
[60] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” arXiv:1512.03385, 2015.
[61]J. Gehring, M. Auli, D. Grangier, D. Yarats, and Y. N. Dauphin, “Convolutional sequence to sequence learning,” arXiv Preprint, arXiv:1705.03122, 2017.
[62]Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450, 2016.
[63]Dan Hendrycks and Kevin Gimpel. Gaussian Error Linear Units (GELUs). arXiv preprint arXiv:1606.08415, 2016.
[64]Elgendi, M.; Norton, I.; Brearley, M.; Abbott, D.; Schuurmans, D. Systolic peak detection in acceleration photoplethysmograms measured from emergency responders in tropical conditions. PLoS ONE 2013, 8, e76585.
[65]X. Mao, Q. Li, H. Xie, R. Y. Lau, and Z. Wang. Least squares generative adversarial networks. arXiv preprint arXiv:1611.04076, 2016.
[66]M. Arjovsky, S. Chintala, and L. Bottou. Wasserstein gan. arXiv preprint arXiv:1701.07875, 2017.
[67]Ian Goodfellow. Nips 2016 tutorial: Generative adversarial networks. arXiv preprint arXiv:1701.00160, 2016.
[68]M. Wiatrak, S. V. Albrecht, and A. Nystrom. Stabilizing generative adversarial networks: a survey. arXiv preprint arXiv:1910.00927, 2019.
[69]Karol Kurach, Mario Lucic, Xiaohua Zhai, Marcin Michalski, and Sylvain Gelly. A large-scale study on regularization and normalization in gans. In ICML, 2019.
[70]Lucic, Mario, Kurach, Karol, Michalski, Marcin, Gelly, Sylvain, and Bousquet, Olivier. Are GANs created equal? A large-scale study. arXiv preprint arXiv:1711.10337, 2017.
[71]Ishaan Gulrajani, Faruk Ahmed, Mart´ın Arjovsky, Vincent Dumoulin, and Aaron C. Courville. Improved training of Wasserstein GANs. CoRR, abs/1704.00028, 2017.
[72]Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. In ICLR, 2018.
[73]M. Xu, Y. Wang. An imbalanced fault diagnosis method for rolling bearing based on semi-supervised conditional generative adversarial network with spectral normalization. IEEE Access (2) (2021), pp. 27736-27747.
[74]M.-Z. Poh et al., “Advancements in noncontact, multiparameter physiological measurements using a webcam,” IEEE Trans. Biomed. Eng., vol. 58, no. 1, pp. 7–11, Jan. 2011
連結至畢業學校之論文網頁點我開啟連結
註: 此連結為研究生畢業學校所提供,不一定有電子全文可供下載,若連結有誤,請點選上方之〝勘誤回報〞功能,我們會盡快修正,謝謝!
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top
無相關期刊