臺灣博碩士論文加值系統

影像式呼吸量測可透過非接觸式方式在不影響受測者睡眠品質之情況下進行呼吸資訊的提取。然而，部分睡眠的場景，例如臉部被遮蔽造成人臉偵測不可用或因為棉被覆蓋使姿體偵測被影響，將會因為呼吸感興趣區域 (ROI) 的不確定性而對呼吸率量測造成挑戰。在本篇論文中，我們將從影像訊號處理與線性組合之觀點切入，研究呼吸 ROI 選擇之真正意涵與所要遵循之準則。接著我們提出一根據呼吸動作在影像中時間空間變化一致特性的呼吸 ROI 偵測算法 (TSC)，該算法主要訴求在於利用較少的計算資源提取呼吸的代表性特徵。此外，為了驗證 TSC 方法在長時間睡眠情境之可靠性，我們準備一睡眠資料庫，該資料庫包含超過 50 小時的可見光睡眠情境與約 20 小時的夜間睡眠情境。實驗結果顯示 TSC 方法以良好的運算效率 (100.2 FPS) 在可見光與夜間情境皆獲的最佳之訊噪比 (可見光情境平均訊噪比為 20.65dB，夜間情境平均訊噪比為 14.91dB)。而呼吸率預測準確率的部分，TSC 方法在可見光情境下將其他呼吸 ROI選取方法的平均絕對誤差從每分鐘 1.65 次降至 0.97 次，在夜間情境則從每分鐘 1.64次降至 1.37 次。

Vision based respiratory measurement can remotely measure respiratory information without affecting the sleeping quality of the user. However, several scenarios during sleeping, such as unavailable positions of the face or the body covered by blankets, can be a challenge to estimate respiratory rate due to the uncertainty of the region of interest(ROI). In this study, we firstly investigate the metrics and physical meanings of ROI selection from the perspective of array signal processing and the concept of linear combining. Then, we proposed a ROI detection algorithm based on both temporal and spatio consistency (TSC), which aims to extract the representative characteristics of respiratory with less computational resources. In addition, a sleeping database, containing data over 50 hours in daylight situation and about 20 hours in night situation, was built to investigate the performance of benchmarked ROI methods during long term sleeping. The experimental results show that TSC attains the best signal to noise ratio in both daylight situation and night situation (20.65dB in daylight situation and 14.91dB in night situation) with relatively low elapsed time (100.2 FPS). As for the accuracy of respiratory rate
estimation, TSC reduces the mean absolute error from 1.65 breath per minute (bpm) to 0.97 bpm in daylight situation of sleeping database and 1.64 breath per minute (bpm) to
1.37 bpm in night situation of sleeping database.

中文摘要 i
英文摘要 ii
致謝 iv
目錄 v
圖目錄 viii
表目錄 xiii
1 緒論 1
1.1 前言 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 研究目的 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 論文架構 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2 相關研究 4
2.1 熱成像呼吸 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2 雷達呼吸 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.3 遠距離光體積變化描記圖 (rPPG) 呼吸 . . . . . . . . . . . . . . . . . . . . 5
2.4 影像式呼吸動作偵測 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
v3 睡眠呼吸偵測演算法 10
3.1 感興趣區域選取 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.1.1 運算定義 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.1.2 問題描述 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.1.3 演算法說明 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.2 時域呼吸訊號提取 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.3 訊號處理 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.4 呼吸率計算 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.5 訊號品質指標 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.6 系統介面與流程 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.6.1 系統介面 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.6.2 系統流程 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4 實驗結果與分析 32
4.1 測試資料 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.1.1 可見光睡眠 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.1.2 IR 光睡眠 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.1.3 厚重覆蓋物與不同姿勢情境 . . . . . . . . . . . . . . . . . . . . . . 37
4.2 方法比較 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.3 選擇參數 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.4 比較指標 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.5 結果與討論 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.5.1 可見光睡眠 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.5.2 IR 光睡眠 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.5.3 厚重覆蓋物與不同姿勢情境 . . . . . . . . . . . . . . . . . . . . . . 55
vi4.6 公開資料庫 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5 貢獻與結論 61
6 未來展望 62
參考文獻 65

[1] Karl A Franklin and Eva Lindberg. “Obstructive sleep apnea is a common disorder
in the population—a review on the epidemiology of sleep apnea”. In: Journal of
thoracic disease 7.8 (2015), p. 1311.
[2] Won Lee, Swamy Nagubadi, Meir H Kryger, and Babak Mokhlesi. “Epidemiology
of obstructive sleep apnea: a population-based perspective”. In: Expert review of
respiratory medicine 2.3 (2008), pp. 349–364.
[3] Ching Wei Wang, Amr Ahmed, and Andrew Hunter. “Vision analysis in detecting
abnormal breathing activity in application to diagnosis of obstructive sleep apnoea”.
In: 2006 International Conference of the IEEE Engineering in Medicine and Biology
Society. IEEE. 2006, pp. 4469–4473.
[4] Ching-Wei Wang, Andrew Hunter, Neil Gravill, and Simon Matusiewicz. “Uncon-
strained video monitoring of breathing behavior and application to diagnosis of sleep
apnea”. In: IEEE transactions on biomedical engineering 61.2 (2013), pp. 396–404.
[5] Ta-Chi Chiang, Meng-Hsiung Tung, Hendrick Rick, Dedi Kurniadi, Zhi-Hao Wang,
and Gwo-Jia Jong. “The non-contact respiratory monitoring system using thermal
image processing”. In: 2016 3rd International Conference on Green Technology and
Sustainable Development (GTSD). IEEE. 2016, pp. 86–92.
[6] Carina Barbosa Pereira, Konrad Heimann, Boudewijn Venema, Vladimir Blazek,
Michael Czaplik, and Steff en Leonhardt. “Estimation of respiratory rate from ther-
mal videos of preterm infants”. In: 2017 39th Annual International Conference of the
IEEE Engineering in Medicine and Biology Society (EMBC). IEEE. 2017, pp. 3818–
3821.
[7] Aditya Singh and Victor M Lubecke. “Respiratory monitoring and clutter rejection
using a CW Doppler radar with passive RF tags”. In: IEEE Sensors Journal 12.3
(2011), pp. 558–565.
[8] Jong Deok Kim, Won Hyuk Lee, Yonggu Lee, Hyun Ju Lee, Teahyen Cha, Seung
Hyun Kim, Ki-Min Song, Young-Hyo Lim, Seok Hyun Cho, Sung Ho Cho, et al.
“Non-contact respiration monitoring using impulse radio ultrawideband radar in
neonates”. In: Royal Society open science 6.6 (2019), p. 190149.
[9] Walter Karlen, Srinivas Raman, J Mark Ansermino, and Guy A Dumont. “Mul-
tiparameter respiratory rate estimation from the photoplethysmogram”. In: IEEE
Transactions on Biomedical Engineering 60.7 (2013), pp. 1946–1953.
[10] Mark van Gastel, Sander Stuijk, and Gerard de Haan. “Robust respiration detection
from remote photoplethysmography”. In: Biomedical optics express 7.12 (2016),
pp. 4941–4957.
[11] Stefan Wiesner and Ziv Yaniv. “Monitoring patient respiration using a single optical
camera”. In: 2007 29th Annual International Conference of the IEEE Engineering
in Medicine and Biology Society. IEEE. 2007, pp. 2740–2743.
[12] Carlo Massaroni, Daniel Simões Lopes, Daniela Lo Presti, Emiliano Schena, and
Sergio Silvestri. “Contactless monitoring of breathing patterns and respiratory rate
at the pit of the neck: A single camera approach”. In: Journal of Sensors 2018
(2018).
[13] Prasara Jakkaew and Takao Onoye. “An Approach to Non-contact Monitoring of
Respiratory Rate and Breathing Pattern Based on Slow Motion Images”. In: 2019
IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia). IEEE.
2019, pp. 47–51.
[14] Fang Zhao, Meng Li, Yi Qian, and Joe Z Tsien. “Remote measurements of heart
and respiration rates for telemedicine”. In: PloS one 8.10 (2013).
[15] Edgar A Bernal, Lalit K Mestha, and Eribaweimon Shilla. “Non contact monitoring
of respiratory function via depth sensing”. In: IEEE-EMBS International Confer-
ence on Biomedical and Health Informatics (BHI). IEEE. 2014, pp. 101–104.
[16] Kuan-Yi Lin, Duan-Yu Chen, and Wen-Jiin Tsai. “Image-based motion-tolerant re-
mote respiratory rate evaluation”. In: IEEE Sensors Journal 16.9 (2016), pp. 3263–
3271.
[17] Mauricio Villarroel, João Jorge, Chris Pugh, and Lionel Tarassenko. “Non-contact
vital sign monitoring in the clinic”. In: 2017 12th IEEE International Conference
on Automatic Face & Gesture Recognition (FG 2017). IEEE. 2017, pp. 278–285.
[18] K Song Tan, Reza Saatchi, Heather Elphick, and Derek Burke. “Real-time vision
based respiration monitoring system”. In: 2010 7th International Symposium on
Communication Systems, Networks & Digital Signal Processing (CSNDSP 2010).
IEEE. 2010, pp. 770–774.
[19] Tomáš Lukáč, Jozef Púčik, and Lukáš Chrenko. “Contactless recognition of respira-
tion phases using web camera”. In: 2014 24th International Conference Radioelek-
tronika. IEEE. 2014, pp. 1–4.
[20] Gary R Bradski. “Computer vision face tracking for use in a perceptual user inter-
face”. In: (1998).
[21] David S Bolme, J Ross Beveridge, Bruce A Draper, and Yui Man Lui. “Visual
object tracking using adaptive correlation fi lters”. In: 2010 IEEE computer society
conference on computer vision and pattern recognition. IEEE. 2010, pp. 2544–2550.
[22] Paul Viola and Michael Jones. “Rapid object detection using a boosted cascade of
simple features”. In: Proceedings of the 2001 IEEE computer society conference on
computer vision and pattern recognition. CVPR 2001. Vol. 1. IEEE. 2001, pp. I–I.
[23] Rainer Lienhart and Jochen Maydt. “An extended set of haar-like features for rapid
object detection”. In: Proceedings. international conference on image processing.
Vol. 1. IEEE. 2002, pp. I–I.
[24] Joao Jorge, Mauricio Villarroel, Sitthichok Chaichulee, Alessandro Guazzi, Sara
Davis, Gabrielle Green, Kenny McCormick, and Lionel Tarassenko. “Non-contact
monitoring of respiration in the neonatal intensive care unit”. In: 2017 12th IEEE
International Conference on Automatic Face & Gesture Recognition (FG 2017).
IEEE. 2017, pp. 286–293.
[25] Mark Everingham, Josef Sivic, and Andrew Zisserman. “Hello! My name is... Buff y”–
Automatic Naming of Characters in TV Video.” In: BMVC. Vol. 2. 4. 2006, p. 6.
[26] Michael H Li, Azadeh Yadollahi, and Babak Taati. “A non-contact vision-based sys-
tem for respiratory rate estimation”. In: 2014 36th Annual International Conference
of the IEEE Engineering in Medicine and Biology Society. IEEE. 2014, pp. 2119–
2122.
[27] Arun Venkitaraman and Vishnu Vardhan Makkapati. “Motion-based segmentation
of chest and abdomen region of neonates from videos”. In: 2015 Eighth International
Conference on Advances in Pattern Recognition (ICAPR). IEEE. 2015, pp. 1–5.
[28] Rik Janssen, W. Wang, Andreia Moço, and Gerard Haan. “Video-based respiration
monitoring with automatic region of interest detection”. In: Physiological measure-
ment 37 (Dec. 2015), pp. 100–114. doi: 10.1088/0967-3334/37/1/100.
[29] D. Alinovi, G. Ferrari, F. Pisani, and R. Raheli. “Respiratory rate monitoring by
maximum likelihood video processing”. In: 2016 IEEE International Symposium on
Signal Processing and Information Technology (ISSPIT). 2016, pp. 172–177.
[30] AP Prathosh, Pragathi Praveena, Lalit K Mestha, and Sanjay Bharadwaj. “Esti-
mation of respiratory pattern from video using selective ensemble aggregation”. In:
IEEE Transactions on Signal Processing 65.11 (2017), pp. 2902–2916.
[31] Hao-Yu Wu, Michael Rubinstein, Eugene Shih, John Guttag, Frédo Durand, and
William Freeman. “Eulerian video magnifi cation for revealing subtle changes in the
world”. In: ACM transactions on graphics (TOG) 31.4 (2012), pp. 1–8.
[32] Neal Wadhwa, Michael Rubinstein, Frédo Durand, and William T Freeman. “Phase-
based video motion processing”. In: ACM Transactions on Graphics (TOG) 32.4
(2013), pp. 1–10.
[33] Davide Alinovi, Luca Cattani, Gianluigi Ferrari, Francesco Pisani, and Riccardo
Raheli. “Spatio-temporal video processing for respiratory rate estimation”. In: 2015
IEEE International Symposium on Medical Measurements and Applications (MeMeA)
Proceedings. IEEE. 2015, pp. 12–17.
[34] Ali Al-Naji and Javaan Chahl. “Remote respiratory monitoring system based on
developing motion magnifi cation technique”. In: Biomedical Signal Processing and
Control 29 (2016), pp. 1–10.
[35] Davide Alinovi, Gianluigi Ferrari, Francesco Pisani, and Riccardo Raheli. “Respira-
tory rate monitoring by video processing using local motion magnifi cation”. In: 2018
26th European Signal Processing Conference (EUSIPCO). IEEE. 2018, pp. 1780–
1784.
[36] Gaddisa Olani Ganfure. “Using video stream for continuous monitoring of breath-
ing rate for general setting”. In: Signal, Image and Video Processing 13.7 (2019),
pp. 1395–1403.
[37] Sitthichok Chaichulee, Mauricio Villarroel, Joao Jorge, Carlos Arteta, Gabrielle
Green, Kenny McCormick, Andrew Zisserman, and Lionel Tarassenko. “Multi-task
convolutional neural network for patient detection and skin segmentation in contin-
uous non-contact vital sign monitoring”. In: 2017 12th IEEE International Confer-
ence on Automatic Face & Gesture Recognition (FG 2017). IEEE. 2017, pp. 266–
272.
[38] João Jorge, Mauricio Villarroel, Sitthichok Chaichulee, Kenny McCormick, and Li-
onel Tarassenko. “Data fusion for improved camera-based detection of respiration in
neonates”. In: Optical Diagnostics and Sensing XVIII: Toward Point-of-Care Diag-
nostics. Vol. 10501. International Society for Optics and Photonics. 2018, p. 1050112.
[39] Sitthichok Chaichulee, Mauricio Villarroel, Joao Jorge, Carlos Arteta, Kenny Mc-
Cormick, Andrew Zisserman, and Lionel Tarassenko. “Cardio-respiratory signal ex-
traction from video camera data for continuous non-contact vital sign monitoring
using deep learning”. In: Physiological measurement 40.11 (2019), p. 115001.
[40] Jorge Brieva, Hiram Ponce, and Ernesto Moya-Albor. “A Contactless Respiratory
Rate Estimation Method Using a Hermite Magnifi cation Technique and Convolu-
tional Neural Networks”. In: Applied Sciences 10.2 (2020), p. 607.
[41] Weixuan Chen and Daniel McDuff . “Deepphys: Video-based physiological measure-
ment using convolutional attention networks”. In: Proceedings of the European Con-
ference on Computer Vision (ECCV). 2018, pp. 349–365.
[42] Mohammad Ghodratigohar, Hamideh Ghanadian, and Hussein Al Osman. “A Re-
mote Respiration Rate Measurement Method for Non-Stationary Subjects Using
CEEMDAN and Machine Learning”. In: IEEE Sensors Journal 20.3 (2019), pp. 1400–
1410.
[43] N. Patwari, J. Wilson, S. Ananthanarayanan, S. K. Kasera, and D. R. Westen-
skow. “Monitoring Breathing via Signal Strength in Wireless Networks”. In: IEEE
Transactions on Mobile Computing 13.8 (2014), pp. 1774–1786.
[44] Andrea Goldsmith. Wireless communications. Cambridge university press, 2005.
Chap. 7, pp. 190–200.
[45] Kim E Barrett, Susan M Barman, Scott Boitano, Heddwen L Brooks, et al. Ganong’s
review of medical physiology. 2016.
[46] Hao-Yu Wu, Michael Rubinstein, Eugene Shih, John Guttag, Frédo Durand, and
William Freeman. “Eulerian video magnifi cation for revealing subtle changes in the
world”. In: ACM transactions on graphics (TOG) 31.4 (2012), pp. 1–8.
[47] K Sreedhar and B Panlal. “Enhancement of images using morphological transfor-
mation”. In: arXiv preprint arXiv:1203.2514 (2012).
[48] Satoshi Suzuki et al. “Topological structural analysis of digitized binary images by
border following”. In: Computer vision, graphics, and image processing 30.1 (1985),
pp. 32–46.
[49] G. Bradski. “The OpenCV Library”. In: Dr. Dobb’s Journal of Software Tools
(2000).
[50] X. Li, J. Chen, G. Zhao, and M. Pietikäinen. “Remote Heart Rate Measurement
from Face Videos under Realistic Situations”. In: 2014 IEEE Conference on Com-
puter Vision and Pattern Recognition. 2014, pp. 4264–4271.
[51] Felix Scholkmann, Jens Boss, and Martin Wolf. “An Effi cient Algorithm for Auto-
matic Peak Detection in Noisy Periodic and Quasi-Periodic Signals”. In: Algorithms
5 (Nov. 2012), pp. 588–603. doi: 10.3390/a5040588.
[52] W. Wang et al. “Discriminative Signatures for Remote-PPG”. In: IEEE Trans.
Biomed. Eng. (2019), pp. 1–1. issn: 1558-2531. doi: 10.1109/TBME.2019.2938564.
[53] Guillaume Heusch, André Anjos, and Sébastien Marcel. “A Reproducible Study on
Remote Heart Rate Measurement”. In: ArXiv abs/1709.00962 (2017).