臺灣博碩士論文加值系統

本研究目的為利用深度學習透過一般的電腦斷層攝影(CT)影像建立一套適合臨床使用的肺栓塞預警系統。在現今對於肺栓塞(PE)病人的電腦檢測方法中，均需使用到電腦斷層肺血管攝影(CTPA)影像作為判斷及辨識病徵的資料，再利用深度學習的方法對於CTPA影像中的病灶區域做語意分割。因此本系統希望能透過CT影像提前發現並預警可能的PE病患。本系統將分為兩個階段，第一個階段為CT影像的分類，本階段將利用卷積神經網路對於胸腔的電腦斷層攝影影像進行分類，挑選出含有疾病的截面並再從中挑選出可能為PE的影像，希望透過我們設計的網路大幅的提升隱藏之PE病患的被發現機率，並減少PE病患等待檢查的時間，增加醫院影像科影像處理效率。透過這個分類模型，我們希望可以在醫院大量的檢查影像中，優先警示可能含有疾病的病患資料，使得這些影像被優先判讀及處理。第二個階段為透過深度學習模型模擬CTPA影像，用於PE病人病徵的識別，提供醫生除了第一階段之分類網路的結果外另一個判斷PE病患的參考依據。本研究分別於兩個階段使用兩種不同的深度學習網路架構。在CT影像分類上，我們使用空洞卷積及注意力機制，搭配多重醫療數位影像傳輸協定(DICOM)格式影像窗值輸入，加強深度學習網路對於肺栓塞的辨識能力，使得臨床上不須再進行CTPA攝影即可進行PE的辨識。而第二階段之模擬CTPA影像生成系統則使用生成對抗網路，在CT影像中強化肺部血管之特徵，加強影像的參考價值，提供醫院判斷PE病患的依據。我們使用來自成功大學醫院的53名病人的CT影像作為分類網路的訓練資料，在模擬CTPA影像的任務上我們使用同樣來自成功大學醫院的22名病人的CT影像及對應之CTPA影像作為訓練資料，並使用兩組生成對抗網路進行生成。
本研究期望能對於肺栓塞臨床診斷方式提出一個新的方案，在繁複的篩檢流程中，透過深度學習網路的輔助，並檢視生成的模擬CTPA影像，使得醫生能評估病患是否需要進行CTPA的詳細檢測，提升檢測肺栓塞的速度及大幅減少未檢出的病患。

The purpose of this study is to develop a clinically appropriate early warning system for pulmonary embolism using deep learning from computed tomography (CT) images. In today's computerized detection methods for patients with pulmonary embolism (PE), computerized tomography pulmonary angiography (CTPA) images are used to determine and identify the symptoms of the disease, and then deep learning is used to semantically segment the lesion areas in the CTPA images. Therefore, the system aims to detect and warn the possible PE patients through CT images. The system will be divided into two stages, the first stage is the classification of CT images, this stage will use the convolutional neural network to classify the computed tomography images of the thoracic cavity, select the cross-section containing diseases and then select the possible PE images from them, we hope that through our designed network to significantly improve the detection rate of hidden PE patients, and reduce the waiting time of PE patients, and increase the efficiency of hospital We hope that through the network we designed, we can significantly improve the detection rate of hidden PE patients, reduce the waiting time for PE patients, and increase the efficiency of image processing in hospitals. Through this classification model, we hope to prioritize patient data that may contain disease among a large number of examination images in hospitals, so that these images can be interpreted and processed first. In the second stage, we simulate CTPA images with deep learning models for the identification of PE patients' symptoms, providing physicians with another reference for determining PE patients in addition to the convolutional neural network in the first stage. In this study, two different deep learning network architectures are used in the two phases. In the CT image classification, we use the cavity convolution and attention mechanism with the DICOM format image window input to enhance the deep learning network for the recognition of pulmonary embolism, so that the clinical recognition of PE can be performed without the need for CTPA photography. In the second stage, the simulated CTPA image generation system uses a generative antagonistic network to enhance the features of pulmonary vessels in the CT images to strengthen the reference value of the images and provide a basis for hospitals to judge PE patients. We used the CT images of 53 patients from National Cheng Kung University Hospital as the training data for the classification network, and we used the CT images of 22 patients from National Cheng Kung University Hospital and the corresponding CTPA images as the training data for the task of simulating CTPA images and generated them using two sets of generative countermeasure networks.
This study is expected to propose a new approach to the clinical diagnosis of pulmonary embolism, in which a deep learning network is used to assist in the complex screening process and to review the generated simulated CTPA images, allowing physicians to assess whether a patient needs to undergo detailed testing for CTPA, improving the speed of detection of pulmonary embolism and significantly reducing the number of undetected patients.

中文摘要 i
Abstract iii
致謝 vi
目錄 vii
圖目錄 ix
表目錄 xi
第一章 緒論 1
1.1 研究背景 1
1.2 文獻回顧 3
1.3 研究動機與目標 7
1.4 論文架構 11
第二章 實驗原理 12
2.1 卷積神經網路 12
2.2 常見之分類模型網路架構 13
2.2.1 ResNet 13
2.2.2 Dilated Residual Net (DRN) 13
2.2.3 Efficient Net 14
2.3 注意力機制 15
2.4 生成對抗網路(GAN) 16
2.4.1 DC GAN 16
2.4.2 Pix2Pix 17
2.4.3 Cycle GAN 18
2.5 生成網路架構 19
2.5.1 U-Net 19
2.5.2 ResNet 20
2.6 判別器網路架構 20
第三章 實驗方法 21
3.1 肺栓塞檢測系統 - 實驗說明及進行步驟 23
3.1.1 實驗概述 23
3.1.2 影像來源 23
3.1.3 影像處理 24
3.2 肺栓塞檢測系統 - 實驗模型架構 24
3.2.1 影像處理 24
3.2.2 兩階段的分類網路架構設計 26
3.2.3 包含注意力機制的空洞卷積神經網路設計 27
3.2.4 多重窗值輸入模型的設計 28
3.3 生成模擬CTPA影像系統 - 實驗說明及進行步驟 30
3.3.1 實驗概述 30
3.3.2 影像來源 30
3.3.3 影像處理 31
3.4 生成模擬CTPA影像系統 - 模型架構 31
3.4.1 CTPA_Pix2Pix model 31
3.4.2 CTPA_Cycle GAN model 32
3.4.3 PE Classification CycleGAN 33
3.5 生成模擬CTPA影像系統 – 評估模型 35
3.5.1 峰值訊噪比 (PSNR) 35
3.5.2 結構相似性指標 (SSIM) 36
3.5.3 深度感知圖像相似度 (LPIPS) 36
3.5.4 Fréchet Inception Distance (FID) 37
3.6 實驗設備 37
第四章 實驗結果與討論 38
4.1 肺栓塞檢測系統 38
4.1.1 含有疾病的影像截面之檢測 38
4.1.2 肺栓塞影像之檢測 41
4.2 模擬之CTPA影像生成系統 45
4.2.1 生成對抗式模型架構比較 45
4.2.2 使用肺栓塞分類器監督之CycleGAN模型結果 48
第五章 結論與未來研究內容 55
5.1 肺栓塞檢測系統之結果 55
5.2 生成模擬CTPA影像之結果 55
5.3 深度學習肺栓塞預警系統 56
參考文獻 58

1. Goggs, Robert, et al., 2009, "Pulmonary thromboembolism." Journal of Veterinary Emergency and Critical Care 19.1: 30-52.
2. Stein, Paul D., et al., 2009, "Diagnosis of pulmonary embolism in the coronary care unit." The American journal of cardiology 103.6: 881-886.
3. Apfaltrer, Paul, et al., 2011, "CT Imaging of Pulmonary Embolism: Current Status." Current Cardiovascular Imaging Reports 4.6: 476-484.
4. Page, Patient, et al., 2006, "Effectiveness of managing suspected pulmonary embolism using an algorithm combining clinical probability, D-dimer testing, and computed tomography." Jama 295.2 (2006): 172-179.
5. Migneault, David, et al., 2015, "An unusual presentation of a massive pulmonary embolism with misleading investigation results treated with tenecteplase." Case reports in emergency medicine 2015.
6. Huang, Shih-Cheng, et al., 2020, "PENet—a scalable deep-learning model for automated diagnosis of pulmonary embolism using volumetric CT imaging." NPJ digital medicine 3.1: 1-9.
7. Pan, Ian, et al., 2021, "Deep Learning for Pulmonary Embolism Detection: Tackling the RSNA 2020 AI Challenge." Radiology: Artificial Intelligence: e210068.
8. Isola, Phillip, et al., 2017, "Image-to-image translation with conditional adversarial networks." Proceedings of the IEEE conference on computer vision and pattern recognition.
9. Goodfellow, Ian, et al., 2014, "Generative adversarial nets." Advances in neural information processing systems 27.
10. Armanious, Karim, et al., 2020, "MedGAN: Medical image translation using GANs." Computerized Medical Imaging and Graphics 79: 101684.
11. Tien, Hui-Ju, et al., 2021, "Cone-beam CT image quality improvement using Cycle-Deblur consistent adversarial networks (Cycle-Deblur GAN) for chest CT imaging in breast cancer patients." Scientific Reports 11.1: 1-12.
12. Albawi, Saad, et al, 2017, "Understanding of a convolutional neural network." 2017 International Conference on Engineering and Technology (ICET). Ieee.
13. He, Kaiming, et al., 2016, "Deep residual learning for image recognition." Proceedings of the IEEE conference on computer vision and pattern recognition.
14. Dumoulin, Vincent, et al., 2016, "A guide to convolution arithmetic for deep learning." arXiv preprint arXiv:1603.07285 .
15. Yu, Fisher, et al., 2017, "Dilated residual networks." Proceedings of the IEEE conference on computer vision and pattern recognition.
16. Tan, Mingxing, et al., 2019 "Efficientnet: Rethinking model scaling for convolutional neural networks." International Conference on Machine Learning. PMLR.
17. Woo, Sanghyun, et al., 2018, "Cbam: Convolutional block attention module." Proceedings of the European conference on computer vision (ECCV).
18. Hu, Jie, et al., 2018, "Squeeze-and-excitation networks." Proceedings of the IEEE conference on computer vision and pattern recognition.
19. Radford, Alec, et al., 2015, "Unsupervised representation learning with deep convolutional generative adversarial networks." arXiv preprint arXiv:1511.06434.
20. Zhu, Jun-Yan, et al., 2017, "Unpaired image-to-image translation using cycle-consistent adversarial networks." Proceedings of the IEEE international conference on computer vision.
21. Ronneberger, O., et al., 2015, "U-net: Convolutional networks for biomedical image segmentation." International Conference on Medical image computing and computer-assisted intervention. Springer, Cham.
22. Kang, Guixia, et al., 2017, "3D multi-view convolutional neural networks for lung nodule classification." PloS one 12.11: e0188290.
23. Wang, Fei, et al., 2017, "Residual attention network for image classification." Proceedings of the IEEE conference on computer vision and pattern recognition.
24. Safety, Patient, et al., 2012, "Radiation Dose in X-Ray and CT Exams." Am Coll Radiol Radiol Soc N Am April 2013.1: 2.
25. Hore, Alain, et al., 2010, "Image quality metrics: PSNR vs. SSIM." 2010 20th international conference on pattern recognition. IEEE.
26. Zhang, Richard, et al., 2018, "The unreasonable effectiveness of deep features as a perceptual metric." Proceedings of the IEEE conference on computer vision and pattern recognition.
27. Heusel, Martin, et al., 2017, "Gans trained by a two time-scale update rule converge to a local nash equilibrium." Advances in neural information processing systems 30.
28. Selvaraju, Ramprasaath R., et al., 2017, "Grad-cam: Visual explanations from deep networks via gradient-based localization." Proceedings of the IEEE international conference on computer vision.
29. Tasar, Onur, et al., 2020, "ColorMapGAN: Unsupervised domain adaptation for semantic segmentation using color mapping generative adversarial networks." IEEE Transactions on Geoscience and Remote Sensing.
30. Chen, Qifeng, et al., 2017, "Photographic image synthesis with cascaded refinement networks." Proceedings of the IEEE international conference on computer vision.

31. Mao, Xudong, et al., 2017, "Least squares generative adversarial networks." Proceedings of the IEEE international conference on computer vision.
32. Romano, Yaniv, et al., 2016, "RAISR: rapid and accurate image super resolution." IEEE Transactions on Computational Imaging 3.1: 110-125.
33. Wang, Ting-Chun, et al., 2018, "High-resolution image synthesis and semantic manipulation with conditional gans." Proceedings of the IEEE conference on computer vision and pattern recognition.