臺灣博碩士論文加值系統

在台灣，慢性腎臟疾病是一種普遍的疾病(大約佔總人口數的12%)，而且大多數的患者並不知道自己罹患慢性腎病。對於慢性腎臟疾病的患者來說，腎臟的活檢組織檢查(renal biopsy)在腎臟疾病的診斷中是不可或缺的診斷程序。從腎臟活組織檢查中，臨床醫師會利用穿刺針或其他方式，穿過位於腎臟上方的皮膚病到腎臟中的位置後，以收集一小部分的腎臟組織，進而做成腎臟樣本影像。在病理學研究中，腎臟標本影像大致上分為三種不同的類型，其中光學顯微鏡影像為最重要且基本的腎臟病理圖像，醫師會根據影像中的微結構來進行診斷。在這些微結構之中，腎絲球是診斷慢性腎病最重要的指標；然而，腎臟病理圖像的分析和解釋過程非常耗費時間。為了提高效能並協助醫生診斷，目前有許多相關的研究將影像辨識技術應用在腎絲球的偵測和分類上。

隨著科技蓬勃發展，深度學習(Deep learning)已經成為最受注目的演算法之一，尤其相較於傳統的影像分析中，通常是要透過人工選取特徵，深度學習能夠自動學習特徵(無監督式學習)，讓辨識結果可以有顯著的改善。近年來，許多將深度學習應用於醫學影像分析的研究都陸陸續續被提出來，例如檢測及影像分類(癌症病變與非病變區域)、影像切割和目標檢測(神經元或細胞結構)等。

本次研究中，我們首先從台中榮民總醫院收集大量可用的腎臟活檢病理圖像(包含經由H&E (蘇木精-伊紅染色，Hematoxylin and Eosin stain) 和PAS (糖原染色Periodic Acid-Schiff stain) 染色的光學顯微影像，並依照不同的染色及放大倍率分類。接著，我們利用卷積神經網路(CNN，Convolutional Neural Networks) 和區域檢測基於卷積神經網路之特徵 (R-CNN，Region-based Convolutional Neural Networks features) 兩種深度學習的方式結合AlexNet模型應用於腎臟活檢病理圖像，以檢測影像中是否具有腎絲球，並進一步偵測腎絲球在影像中的位置。在CNN的部分，實驗結果表明AlexNet模型能夠有效分類出腎絲球和非腎絲球的影像，測試結果的F-measure為96.9%，其中precision為95.1%，recall為98.6%。在R-CNN和Faster R-CNN的部分，R-CNN的F-measure為43.4%，其中precision為33%，recall為63.6%；而Faster R-CNN的F-measure為83.5%，其中precision為75.8%，recall為92.8%。

透過實驗結果我們可以看到，Faster R-CNN的檢測能力比R-CNN好很多；然而，Faster R-CNN的部分還是有許多能夠改進的地方，像是有一些難以偵測出來的腎絲球(硬化、變形等狀況)而導致了false negative的結果，也有部分非腎絲球的影像(包括:腎小管、血管或腎間質等)被誤判成腎絲球，使得false positive的影像數量增加。為了改善目前的狀況，未來我們將採用其他的預訓練模型(如VGG16，GoogleNet或ResNet等)。另外，在染色影像上我們觀察到，H&E染色的影像會比PAS染色影像出現比較高的錯誤率，所以我們也會利用Cycle GAN色彩轉換的方式去調整H&E染色影像，以降低影像誤判的數量。為了讓影像資料更加完善，預期將會繼續收集更多不同類型的腎絲球(positive data)及非腎絲球(negative data)的影像，希望讓模型能學習到更詳盡的特徵，以提高準確度。未來我們也將進一步判別影像中之腎絲球有無硬化等其他病變。

In Taiwan, chronic kidney disease (CKD) is a common disease (about 12% of people, three million people), and most chronic kidney disease patients are unaware of their disease. For patients with chronic kidney disease, renal biopsy is a very important and irreplaceable diagnostic procedure. In renal biopsy procedure, the clinician will cut the area in the skin near the kidneys and the clinician will obtain a small proportion of kidney tissue which will become kidney specimens. There are three different kinds of kidney specimens image studies, and then light microscopy images of tissue taken from the renal biopsy are the most significant and basic renal pathological images. The doctor will diagnose with the micro-structure in light microscopy images. In these microstructures, glomerulus is one of the most important indicators of diagnosis of chronic kidney disease. The process of analyzing renal pathological images is so tedious that it takes a lot of time and manpower. In order to improve efficiency and assist doctors in diagnosis, many researchers have applied image processing techniques to the detection and classification of glomerulus.

In recent years, Deep Learning which focused on unsupervised feature learning is the most eye-catching and widely used algorithms in the field of image recognition. Many studies on deep learning in medical image analysis have also been proposed, including detection and classification (cancerous area vs. non-cancerous area), segmentation and object detection (neuronal or cell structure). In this study, we will collect and arrange renal biopsy pathological images in various stains (H&E, PAS, Masson, Sliver and other stains of the light microscopy images) from Taichung Veterans General Hospital at first And then, we take use of different deep learning methods to classify and detect glomerulus: CNN (Convolutional Neural Network) AlexNet model, R-CNN (Regions with CNN features) and Faster R-CNN with AlexNet on renal biopsy pathological images.

The experiment result shows that AlexNet CNN model can classify glomerulus and non- glomerulus and F-measure of the result is 96.9% with precision 95.1% and recall 98.6%. In the part of R-CNN and Faster R-CNN, F-measure of the result of R-CNN is 43.4% with precision 33% and recall 63.6%. F-measure of the result of Faster R-CNN is 83.5% with precision 75.8% and recall 92.8%. Through the test results of the image, we can know that the detection capability of the Faster R-CNN is better than that of the R-CNN. However, Faster R-CNN still has many false positives results that need to be improved and corrected. In order to improve the current situation, novel pre-training models (like VGG16, GoogleNet or ResNet) will be used to train our model in the future works. As a way to solve the problem of H&E stained images, Cycle GAN will be used as an image-to-image translation method to change the color of H&E stained images. After that, we will continue to collect images of glomerular (positive data) and non- glomerular (negative data), such as various types of glomerulus lesions and hard negative data, and then we may go further to detect glomerulosclerosis in the images.

Contents
Acknowledgments i
Chinese Abstract ii
English Abstract iv
Table of Contents vi
List of Figures viii
List of Tables x
Chapter 1. Background 1
1.1 Chronic kidney disease 1
1.2 Renal biopsy and Light microscopy image 2
1.3 Glomerulus and Other Microstructures from Kidney Images 4
1.4 The Variability of Glomerulus Appearance 5
1.5 Image recognition applications in medical imaging 9
1.5.1 Traditional image processing 10
1.5.2 Deep Learning 10
1.6 CNN and R-CNN 12
1.7 Transfer Learning vs From Scratch 13
1.8 The Aim of Our Study 14
Chapter 2. Literature Review 16
2.1 Traditional feature learning 16
2.2 Deep learning 20
Chapter 3. Material and Methods 22
3.1 Dataset 22
3.2 Research Process 24
3.2.1 Glomerulus Classification Using CNN 25
3.2.2 Glomerulus Detection Using R-CNN and Faster R-CNN 27
3.3 AlexNet 31
3.4 Validation and Measurement 33
Chapter 4. Experimental Results 35
4.1 CNN : Glomerulus and Non-Glomerulus 35
4.2 R-CNN and Faster R-CNN: Glomerulus Detection 39
Chapter 5. Conclusions and Discussions 49
Reference 51
Appendix 53


List of Figures
Fig.1. A: Glomerulus B: Renal tubule C: Interstitial tissue D: Blood vessel. 4
Fig.2. Diagram of the glomerulus and surrounding structures 4
Fig.3. Light microscopy images at different magnifications. 5
Fig.4. H&E staining images. 6
Fig.5. PAS staining images. 7
Fig.6. Different H&E staining images. 8
Fig.7. The images of lesions in the glomerulus. 8
Fig.8. Glomerular lesions in images. 9
Fig.9. Comparison Diagram of Transfer Learning and From Scratch. 13
Fig.10. The illustration of glomerulus detection in R-HOG. 17
Fig.11. A: R-HOG B: S-HOG 18
Fig.12. A flowchart of image preprocessing of Taras Kotyk's approach. 19
Fig.13. A: Original images B: Red channel image.. 19
Fig.13. C: Blue channel image D: Green channel image .. 19
Fig.14. A flowchart of glomerular calculation of Taras Kotyk's approach. 20
Fig.15. The glomerulus in the light microscopy images with different stain. 23
Fig.16. A: Normal glomerulus in the images B: Non-glomerulus. 24
Fig.16. C: Glomerulosclerosis D: Glomerular with other lesions. 24
Fig.17. The flow chart of glomerular classification using CNN. 25
Fig.18. The schematic diagram of image segmentation 26
Fig.19. A: Positive Data B: Negative Data. 27
Fig.20. The schematic diagram of image labeling. 27
Fig.21. The flow chart of glomerular detection using R-CNN and Faster R-CNN. 28
Fig.22. The architecture of R-CNN. 29
Fig.23. The architecture of R-CNN. 29
Fig.24. The schematic diagram of RoI-Pooling. 30
Fig.25. The architecture of AlexNet. 32
Fig.26. A: Glomerulus (Positive Data) B: Non- Glomerulus (Negative Data). 35
Fig.27. A: Glomerulus lesions (Positive Data) B: Non-Glomerulus (Negative Data). 35
Fig.28. The detection result using the R-CNN in H&E stained images. 41
Fig.29. The detection result using the faster R-CNN in H&E stained images. 42
Fig.30. The detection result using the R-CNN in PAS stained images. 43
Fig.31. The detection result using the Faster R-CNN in PAS stained images. 44
Fig.32. H&E stained images from Dataset(A) with detection results. 45
Fig.33. H&E stained images from Dataset(B) with detection results. 46
Fig.34. PAS stained images from Dataset(A) with detection results. 48
Fig.35. PAS stained images from Dataset(B) with detection results. 48
Fig.36. Negative Data which are hard to identify. 49
Fig.37. H&E stained images at 100x with detection results of R-CNN. 53
Fig.38. H&E stained images at 100x with detection results of Faster R-CNN. 54
Fig.39. H&E stained images at 200x with detection results of R-CNN 55
Fig.40. H&E stained images at 200x with detection results of Faster R-CNN.. 56
Fig.41. H&E stained images at 400x with detection results of R-CNN. 57
Fig.42. H&E stained images at 400x with detection results of Faster R-CNN. 58
Fig.43. PAS stained images at 100x with detection results of R-CNN 59
Fig.44. PAS stained images at 100x with detection results of Faster R-CNN. 60
Fig.45. PAS stained images at 200x with detection results of R-CNN. 61
Fig.46. PAS stained images at 200x with detection results of Faster R-CNN. 62
Fig.47. PAS stained images at 400x with detection results of R-CNN. 63
Fig.48. PAS stained images at 400x with detection results of Faster R-CNN. 64


List of Tables
Table.1. The statistics of renal biopsy pathological images. 22
Table.2. Total number of training data and test data 26
Table.3. Total number of Dataset (A). 31
Table.4. Total number of Dataset (B). 31
Table.5. The features of AlexNet. 33
Table.6. Table6. Definitions of TP, FP, FN, and TN. 36
Table.7. Results of CNN with AlexNet. 36
Table.8. The samples of false positive. 37
Table.9. The samples of false negative. 37
Table.10. The samples of true negative with low reliability. 38
Table.11. The samples of true positive with low reliability. 38
Table.12. Results with training different stained images. 39
Table.13. Total number of training data and test data of R-CNN and Faster R-CNN. 40
Table.14. Results of R-CNN and Faster R-CNN. 40
Table.15. Results with training both stained images and H&E stained images. 44
Table.16. Results with training both stained images and PAS stained images. 47

[1] Gregory D. Krol, “Chronic Kidney Disease (CKD)”, American Society of Nephrology, pp.4-5, 2011.
[2] Wen, C. P., et al., “All-cause mortality attributable to chronic kidney disease: a prospective cohort study based on 462293 adults in Taiwan”, Lancet, pp.2173-2182, June 2008.
[3] Bryan Chew. (2018, March 27). THE GLOMERULUS.[Online]. Available:http://teachmephysiology.com/urinary-system/glomerulus/glomerulus/
[4] Anna F. Lee, Henrike Rees, David A. Owen, David G. Huntsman, “Periodic Acid-Schiff Is Superior to Hematoxylin and Eosin for Screening Prophylactic Gastrectomies from CDH1 Mutation Carriers”, The American Journal of Surgical Pathology, pp.1007-1013, July 2010.
[5] Iliya Valchanov. (2018, January 25). Machine Learning: An Overview. [Online]. Available: https://www.datascience.com/blog/machine-learning-overview
[6] Hinton GE, Salakhutdinov RR, “Reducing the dimensionality of data with neural networks”, Science, pp.504-507, July 2006.
[7] Hirohashi, Y., Relator, R., Kakimoto, T., Saito, R., Horai, Y., Fukunari, A., Utsumi, H., Okada, K., Kato, T., “Automated quantitative image analysis of glomerular desmin immunostaining as a sensitive injury marker in Spontaneously Diabetic Torii rats”, Journal of Biomedical Image Processing, vol.1, Iss.1, pp. 20-28, May 2014.
[8] Kato, T., Relator, R., Ngouv, H., Hirohashi, Y., Takaki, O., Kakimoto, T., Okada, K., “Segmental HOG: new descriptor for glomerulus detection in kidney microscopy image”, BMC Bioinformatics, September 2015.
[9] Kotyk, T., Dey, N., Ashour, A. S., Dana, B. T., Chakraborty, S., Ashour, A. S. and Tavares , J. M. R. S., “Measurement of glomerulus diameter and Bowman's space width of renal albino rats”, Computer Methods and Programs in Biomedicine, pp. 143-153, April 2016.
[10] Pedraza. A., Gallego. J., Lopez. S., Gonzalez. L., Laurinavicius. A., Bueno. G., “Glomerulus classification with convolutional neural networks”, Medical Image Understanding and Analysis, vol. 723, pp. 839-849, June 2017.
[11] Gallego, J., Pedraza, A., Lopez, S., Steiner, G., Gonzalez, L., Laurinavicius, A., Bueno, G., “Glomerulus classification and detection based on convolutional neural networks”, Journal of Imaging, pp.1-19, January 2018.
[12] Zhao, X., Li, W., Zhang, Y., Gulliver, T. A., Chang, S., Feng, Z., “A faster RCNN-based pedestrian detection system”, in Proc. IEEE 84th Vehicular Technology Conference (VTC-Fall), September 2016, pp. 1-5.
[13] Ren, S., He, K., Girshick, R., Sun, J., “Faster R-CNN: towards real-time object detection with region proposal networks”, in Proc. Computer Vision and Pattern Recognition, January 2016.
[14] Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks”, in Proc. NIPS'12 Proceedings of the 25th International Conference on Neural Information Processing Systems, vol.1, December 2012,pp.1097-1105.
[15] Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation”, in Proc. Computer Vision and Pattern Recognition , October 2014.
[16] Fawcett, T., “An introduction to ROC analysis”, Pattern Recognition Letters, Lett. vol.27, Iss.8, pp. 861-874, June 2006.