臺灣博碩士論文加值系統

大腸癌目前是全世界排名第三常見的癌症。大腸息肉為大腸癌之前身。所以大腸息肉的檢測為重要的臨床研究議題。如能及早發現大腸息肉，提前切除，就能有效降低死亡率。大腸鏡為偵測大腸息肉的首選方式。根據研究顯示，平均每人次完成大腸鏡檢仍有26% 的大腸息肉未被發現。因此利用深度學習來幫助醫師辨識出大腸息肉是目前重要的研究。大腸息肉又分為良性息肉與惡性息肉。惡性息肉中以扁平鋸齒狀腺瘤生長速度快，易惡變，但相對數量少且不易被發現。但訓練深度學習模型需要大量的訓練資料，因此本研究使用資料增強加上Conditional GAN生成出更多的扁平鋸齒狀腺瘤訓練資料集。再利用物件偵測模型YOLOv4訓練，以提供鏡檢醫師完善的電腦輔助偵測系統並提高腺瘤檢出率及準確的判斷息肉類別。此外大腸鏡即時操作時常會因晃動造成影像模糊而影響息肉辨識，所以本研究提出利用DeblurGAN-v2進行去模糊化。最後訓練Mask R-CNN，在偵測到息肉後，推算息肉的真實面積，提供鏡檢醫師一個可依照息肉大小判斷病情嚴重度及後續追蹤間隔時間的重要依據。經實驗結果後發現使用GAN資料比只使用資料擴增的模型的mAP提升了2.42%，對模糊圖片使用DeblurGAN-v2後再偵測的mAP提升5.10%，Mask R-CNN的mAP則為86.24%，可有效幫助醫師找出息肉並判斷病情嚴重度。

Colorectal cancer （CRC） is the third most common cancer in the world. Early detection and treatment can reduce mortality effectively. Colon polyp is the precursor of colon cancer. Colonoscopy is the “gold standard” for colon polyp detection. An average of 26% of miss rate after colonoscopy has been reported. Using deep learning algorithm to develop a computer-aided system for colon polyp detection is effective to reduce miss rate. Colon polyps are divided into neoplastic and non-neoplastic. Sessile serrated adenoma is one of the neoplastic polyp, which evolves to colon cancer in short duration, rare and not easy to detect. In order to train a reliable object detection model, a large database is needed. We used data augmentation technique and Conditional Generative Adversarial Networks to generate sessile serrated adenoma images with YOLOv4 to train the computer-aided detection and diagnosis model to improve polyp and adenoma detection rate. In addition, DeblurGAN-v2 was used to reduce the motion blurness during colonoscopy for detection accuracy enhancement. Lastly, a separate Mask R-CNN based model applies to the detected polyp for area estimation, which aids endoscopist to determine the risk of future CRC and surveillance interval. After the experimental results, it was found that the mAP of GAN data improved by 2.42% compared with the model using only data augmentation. The mAP of DeblurGAN-v2 for blurry images was improved by 5.10%. The mAP of Mask R-CNN is 86.24%, which can effectively help doctors to find polyps and determine the severity of the disease.

摘 要 i
目錄 iii
圖目錄 v
表目錄 vi
第一章 緒論 1
第一節 研究背景與動機 1
第二節 研究目的 2
第二章 文獻探討 4
第一節 CADe用於大腸息肉檢測 4
第二節 生成對抗網路GAN近期研究 10
第三節 去模糊化技術與DeblurGAN2 12
第四節 物件跟蹤技術 15
第三章 研究方法 17
第一節 GAN訓練流程 17
一、 調整圖片長寬 17
二、 標記資料 17
三、 使用Canny找出輪廓圖 18
四、 將輪廓圖覆蓋在ground truth圖上 19
五、 將覆蓋好的ground truth圖覆蓋在原圖上 19
六、 訓練Conditional GAN 20
七、 評估GAN訓練結果 24
(一) 峰值信噪比PSNR（Peak signal-to-noise ratio） 25
(二) 結構相似性SSIM（structural similarity） 25
第二節 YOLOv4訓練流程 26
一、 標記資料 26
二、 資料擴增 27
三、 訓練物件偵測模型 27
四、 模型指標比較 29
第三節 去模糊化測試資料 30
一、 對測試影片進行高斯模糊 30
二、 使用DeblurGAN-v2去模糊化 31
三、 評估去模糊化效果 31
第四節 bounding box大小比較和Deep SORT效果測試 31
第五節 息肉面積的估算 33
第四章 研究結果 36
第一節 GAN產生圖片比較 36
第二節 YOLOv4數據比較 38
第三節 YOLOv4數據不同資料 40
第四節 去模糊化比較 42
第五節 不同bounding box大小對準確度的影響 44
第六節 Deep SORT效果測試結果 46
第七節 Mask R-CNN數據與面積計算結果 47
第五章 結論 50
參考文獻 53

[1] 衛生福利部國民健康署大腸癌篩檢簡介。
Available：https://www.hpa.gov.tw/Pages/Detail.aspx?nodeid=621&pid=1136
（Accessed by 21 November 2021.）
[2] Zhao S, Wang S, Pan P, et al. Magnitude, Risk Factors, and Factors Associated With Adenoma Miss Rate of Tandem Colonoscopy: A Systematic Review and Meta-analysis. Gastroenterology, 156:1661-1674.e11, 2019.
[3] Brand, E. C., Dik, V. K., van Oijen, M. G., & Siersema, P. D. Missed adenomas with behind-folds visualizing colonoscopy technologies compared with standard colonoscopy: a pooled analysis of 3 randomized back-to-back tandem colonoscopy studies. Gastrointestinal endoscopy, 86（2）, 376-385, 2017.
[4] P .Domingos, "A few useful things to know about machine learning," Communications of the ACM, 55（10）, pp.78-87, 2012.
[5] H.C. Shin et al., "Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning," IEEE transactions on medical imaging, 35（5）, pp. 1285-1298, 2016.
[6] Fonollà, Roger, et al, "A CNN CADx System for Multimodal Classification of Colorectal Polyps Combining WL, BLI, and LCI Modalities," Applied Sciences, 10（15）, pp.5040, 2020.
[7] Li, T., Brown, J. R. G., Tsourides, K., Mahmud, N., Cohen, J. M., & Berzin, T. M. "Training a computer-aided polyp detection system to detect sessile serrated adenomas using public domain colonoscopy videos," Endoscopy International Open, 8（10）, 2020.
[8] Fan, DP., Ji, GP., Zhou, T. et al. "PraNet: Parallel Reverse Attention Network for Polyp Segmentation," International Conference on Medical Image Computing and Computer-Assisted Intervention, Lima, Peru, October 4-8, 2020.
[9] Brandao, P., Mazomenos, E., Ciuti, G., Caliò, R., Bianchi, F., Menciassi, A., Dario, P., Koulaouzidis, A., Arezzo, A. and Stoyanov, D., "Fully convolutional neural networks for polyp segmentation in colonoscopy," Medical Imaging 2017: Computer-Aided Diagnosis, vol. 10134, pp. 101340, 2017.
[10] Bochkovskiy, A., Wang, C. Y., & Liao, H. Y. M., "YOLOv4: Optimal Speed and Accuracy of Object Detection," arXiv preprint arXiv:2004.10934, 2020.
[11] Robin Zachariah, Jason Samarasena, Daniel Luba, Erica Duh, Tyler Dao, James Requa, Andrew Ninh, William Karnes, "Prediction of Polyp Pathology Using Convolutional Neural Networks Achieves ‘Resect and Discard’ Thresholds," The American journal of gastroenterology, vol. 115, pp. 138-144, 2020.
[12] Shin, Y., Qadir, HA., Aabakken, L., et al., "Automatic colon polyp detection using region based deep cnn and post learning approaches," IEEE Access, vol. 6 , pp. 40950-40962, 2018.
[13] Nicolai Wojke, Alex Bewley, Dietrich Paulus, "Simple Online and Realtime Tracking with a Deep Association Metric," 2017 IEEE International Conference on Image Processing （ICIP）, Beijing, China, September 17-20, 2017.
[14] He, K., Gkioxari, G., Dollár, P., & Girshick, R., "Mask R-CNN," In Proceedings of the IEEE international conference on computer vision, Venice, Italy, October 22-29, 2017.
[15] Wang, P., Liu, X., Berzin, T. M., Brown, J. R. G., Liu, P., Zhou, C., ... & Zhou, G., " Effect of a deep-learning computer-aided detection system on adenoma detection during colonoscopy （CADe-DB trial）: a double-blind randomised study," The Lancet Gastroenterology & Hepatology, 5（4）, pp. 343-351, 2020.
[16] Repici, A., Badalamenti, M., Maselli, R., Correale, L., Radaelli, F., Rondonotti, E., ... & Hassan, C., "Efficacy of Real-Time Computer-Aided Detection of Colorectal Neoplasia in a Randomized Trial," Gastroenterology, 159（2）, pp. 512-520, 2020.
[17] Zhang, X., Pan, W., & Xiao, P., "In-vivo Skin Capacitive Image Classification Using AlexNet Convolution Neural Network," In 2018 IEEE 3rd International Conference on Image, Vision and Computing （ICIVC）, Chongqing, China, June 27-29, pp. 439-443, 2018.
[18] Antioquia, A. M. C., Tan, D. S., Azcarraga, A., Cheng, W. H., & Hua, K. L. "ZipNet: ZFNet-level Accuracy with 48× Fewer Parameters," 2018 IEEE Visual Communications and Image Processing （VCIP）, Taichung, Taiwan, December 9-12 ,2018.
[19] Algorry, A. M., García, A. G., & Wofmann, A. G., "Real-time object detection and classification of small and similar figures in image processing," 2017 International Conference on Computational Science and Computational Intelligence （CSCI）, Las Vegas, Nevada, USA, December 14-16, 2017.
[20] Jing-Ran Su, Zhen Li, Xue-Jun Shao, Chao-Ran Ji, Rui Ji, Ru-Chen Zhou, Guang-Chao Li, Guan-Qun Liu, Yi-Shan He, Xiu-Li Zuo, Yan-Qing Li, "Impact of a real-time automatic quality control system on colorectal polyp and adenoma detection: a prospective randomized controlled study （with videos）," Gastrointestinal endoscopy, 91（2）, pp.415-424, 2020.
[21] W.-N. Liu et al., "Study on detection rate of polyps and adenomas in artificial-intelligence-aided colonoscopy," Saudi J Gastroenterol, 26（1）, pp.13-19, 2020.
[22] P. Wang et al., "Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study," Randomized Controlled Trial Gut, 68（10）, pp.1813-1819, 2019.
[23] Tan, Mingxing, and Quoc V. Le, "Efficientnet: Rethinking model scaling for convolutional neural networks," International Conference on Machine Learning （ICML 2019）, Long Beach Convention & Entertainment Center, Long Beach, CA, USA, June10-15, 2019.
[24] Gao, S., Cheng, M. M., Zhao, K., Zhang, X. Y., Yang, M. H., & Torr, P. H. "Res2net: A new multi-scale backbone architecture," IEEE transactions on pattern analysis and machine intelligence, 43（2）, 2019.
[25] MICCAI 2015比賽官網
Available：https://polyp.grand-challenge.org/databases/
（Accessed by 10 March 2021）.
[26] Qadir, H. A., Shin, Y., Solhusvik, J., Bergsland, J., Aabakken, L., & Balasingham, I., "Polyp detection and segmentation using mask R-CNN: Does a deeper feature extractor cnn always perform better?," 2019 13th International Symposium on Medical Information and Communication Technology （ISMICT） Oslo ,Norway, May 8-9, 2019.
[27] Zhe Guo, Ruiyao Zhang, Qin Li, Xinkai Liu, Daiki Nemoto, Kazutomo Togashi, S.M. Isuru Niroshana, Yuchen Shi, Xin Zhu, "Reduce False-Positive Rate by Active Learning for Automatic Polyp Detection in Colonoscopy Videos," 2020 IEEE 17th International Symposium on Biomedical Imaging （ISBI）, Iowa City, USA, April 3-7, 2020.
[28] Debesh Jha, Michael A. Riegler, Dag Johansen, Pal Halvorsen, Havard D. Johansen, "DoubleU-Net: A Deep Convolutional Neural Network for Medical Image Segmentation," 2020 IEEE 33rd International Symposium on Computer-Based Medical Systems （CBMS）, Minnesota, USA, July 28-30, 2020.
[29] Younghak Shin, Hemin Ali Qadir, Ilangko Balasingham, "Abnormal Colon Polyp Image Synthesis Using Conditional Adversarial Networks for Improved Detection Performance," IEEE Access, vol. 6, pp. 56007-56017, 2018.
[30] I. Goodfellow et al., "Generative adversarial nets." in Advances in neural information processing systems,” 27th International Conference on Neural Information Processing Systems （NPIS 2014）, Quebec, Canada, December 8-13, 2014.
[31] Radford, A., Metz, L., and Chintala, S., "Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks," International Conference on Learning Representations, San Juan, Puerto Rico, May 2-4, 2016.
[32] Arjovsky, M., Chintala, S., and Bottou, L., "Wasserstein GAN," Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia, August 6-11, 2017.
[33] Karras, T., Aila, T., Laine, S., et al. "Progressive growing of GANs for improved quality, stability,and variation," International Conference on Learning Representations, Vancouver, Canada, April 31 - May 3, 2018.
[34] Karras, T., Laine, S., and Aila, T. "A style-based generator architecture for generative adversarial networks," Proceedings of the IEEE conference on computer vision and pattern recognition, Long Beach, California, June 16-20, 2019.
[35] Karras, T., Laine, S., Aittala M., et al. "Analyzing and Improving the Image Quality of StyleGAN," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Washington, United States, June 14-19, 2020.
[36] Shin, Y., Qadir, H. A., & Balasingham, I. "Abnormal colon polyp image synthesis using conditional adversarial networks for improved detection performance," IEEE Access, vol. 6, pp. 56007-56017, 2018.
[37] Bass, C., Dai, T., Billot, B., "Image synthesis with a convolutional capsule generative adversarial network," Proceedings of the International Conference on Medical Imaging with Deep Learning, London, July 8-10, 2019.
[38] Gupta, A. Venkatesh,S. Chopra, S. et al., "Generative image translation for data augmentation of bone legion pathology," International Conference on Medical Imaging with Deep Learning, London, July 8-10, 2019.
[39] D. Gong et al, "From motion blur to motion flow: a deep learning solution for removing heterogeneous motion blur, " IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii, July 21-26, 2017.
[40] W.L. Zhang et al., "Multi-scale Network with the deeper and wider residual block for MRI motion artifact correction", IEEE 43rd Annual Computer Software and Applications Conference （COMPSAC）, Milwaukee, WI, USA, July 15-19, 2019.
[41] W. H. Richardson, "Bayesian-Based Iterative Method of Image Restoration," Journal of the Optical Society of America, 62（1）, pp.55-59, 1972.
[42] R. Fergus et al., "Removing camera shake from a single photograph", ACM transactions on graphics （TOG）, 25（3）, pp. 787-794, 2006.
[43] Kupyn, O., Martyniuk, T., Wu, J., & Wang, Z., "DeblurGAN-v2: Deblurring （orders-of-magnitude） faster and better," Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea, October 27 - November 2, 2019.
[44] Kupyn, O., Budzan, V., Mykhailych, M., Mishkin, D., & Matas, J. "DeblurGAN: Blind motion deblurring using conditional adversarial networks," Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, UT, USA, 2018.
[45] Alex Bewley, Zongyuan Ge, Lionel Ott, Fabio Ramos, Ben Upcroft, "Simple Online and Realtime Tracking," 2016 IEEE International Conference on Image Processing （ICIP）, Phoenix Arizona, USA, September 25-28, 2016.