臺灣博碩士論文加值系統

分枝桿菌感染仍然是世界衛生的一大重要疾病，根據世界衛生組織的報告，每年仍有六百萬新感染分枝桿菌的案例報告。隨著現在全球化的交通傳播，分枝桿菌是造成全球傳染病的一大來源。診斷分枝桿菌有許多方法，其中一項為顯微鏡檢驗病理組織以確診分枝桿菌感染，但此方法需要耗費人力在高倍率的顯微鏡下偵測分枝桿菌。隨著全掃描病理玻片影像的興起，數位影像可以儲存成整片病理玻片的高解析度資訊，這些完整的影像可供影像處理以及分析。近年來許多研究已指出使用全掃描病理玻片和傳統光學顯微鏡有相同的診斷能力；而深度學習與卷積神經網路已在電腦輔助診斷醫學影像有很大的進步。因此本研究探討利用基於深度學習的卷積神經網路，對全掃描病理玻片影像中的分枝桿菌判斷的準確率進行分析，並以轉移學習克服醫療影像樣本不足的限制。
在收集了分枝桿菌的全掃描病理玻片後，將影像尺寸切割為可辨識分枝桿菌的大小(20×20像素)，我們取出613個陽性的影像區塊做為轉移學習的陽性樣本數，為了測試此神經網路的強度，可能會被誤認為陽性的區塊影像我們再進一步取出1022個包含鮮豔紅色的陰性影像區塊與1363個紅色飽和度較低的陰生影像做為陰性樣本組。這些影像分為80%的訓練組及20%的驗證組，加入卷績神經網路的末三層進行測試。另一個實驗我們測試了彩色與灰階的影像組別，1202個陰性的影像區塊是本實驗的陰性樣本組，包含了各種可能染色與結構在高倍下可能會和桿菌難區分的狀況。
結果顯示，以AlexNet轉移學習在區分陽性與陰性的圖片區塊，在鮮豔的紅色組別準確率可以達96.6%；若是淡色的紅色組別準確率可以達95.3%。在比較彩色與灰階的影像去做分類，彩色的影像組別準確率可達95.3%，但若是將影像轉為灰階後，準確率則降為73.8%。
使用深度卷積神經網路在區分分枝桿菌陽性與陰性的病理影像有高度的準確率，在實驗中也顯現出色彩對於此神經網路學習的重要性。本研究所建立影像分類模型將可提升病理科醫師的診斷效率，減少臨床診斷時耗費過多的精力與時間。

Mycobacteria infection is an important disease in global health because six millions of new infectious cases reported worldwide according to World Health Organization 2016 report. As the globalized transportation advances and opportunistic disease emerges, mycobacteria infection is a threat for global health. Fast and corrective diagnosis of mycobacteria infection is a priority for disease control, and one of the important tool is histopathological examination. However, it takes pathologist much efforts and time to look for mycobacteria bacilli in high power fields under microscope. As the progress of whole slide images(WSI) scanning system and modern computer science, one can use digital images analysis technique for mycobacteria detection. In recent years, numerous large studies had concluded that whole slide scanning digital pathology images is equivalent to traditional glass slides for clinical diagnosis. With the recent success of deep learning methods, convolutional neural networks(CNN) have been applied for computer-aided diagnosis in medical images. Therefore, the aim of our study is to investigate the potential of deep learning method for classification of acid-fast stained mycobacteria images using transfer learning due to the limited sample size of medical images.
After acquiring WSI of cutaneous mycobacterial infection, the original WSI was cropped into the size for identifying bacilli(20×20 pixel). A total of 613 bacilli-positive image blocks were selected. To evaluate the robustness of our classifier, we included negative image blocks that may be mistaken as positive for bacilli, including 1022 splendid color image group and 1363 faint reddish image group. Image blocks were randomly partitioned into 80% training set and 20% validation set. The training images were input into the final three layer of DCNN. As another experiment we selected 1202 negative image blocks with all different features to compare performance between color and grayscale image dataset.
As a result, the model achieved 96.3% accuracy for the splendid color group, and 95.3% accuracy for the faint color group. The comparison between color and grayscale image dataset revealed color group achieved 95.3% accuracy while converting to greyscale we obtained only 73.8% accuracy.
Our proposed transfer learning DCNN model successfully distinguish bacilli images from other tissues in different cutaneous mycobacterial infectious diseases. The robust computer-aided diagnosis model achieved high accuracy. Furthermore, the importance of color information for bacilli detection was revealed. The model would benefit pathologists by locating the bacilli-positive image blocks within WSI and reduce the efforts and time needed for diagnosis.

Taipei Medical University Graduate Thesis Certification i
Thesis Publication Agreement ii
Confidentiality agreement & sign-in form iii
Acknowledgments iv
Table of Contents v
List of Tables vii
List of Figures viii
摘要 x
Abastract xii
Chapter I Introduction 1
1.1Background: Mycobacteria infectious disease 1
1.2 Digital pathology and whole slide images (WSI) 3
1.3 Computer-aided diagnosis and deep learning in digital pathology 4
Chapter II Literature Review 5
2.1 Previous works related to mycobacterium bacilli detection 5
2.2 Automated mycobacteria bacilli detection in histopathology 6
Chapter III Material and Methods 8
3.1 Histopathology whole slide image resources 9
3.2 Dataset and image labeling 10
3.3 Effect of color saturation and grayscale images on classifier performance 11
3.4 Transferred convolutional neural network for mycobacterium histopathological image classifications 15
Chapter IV Results 20
4.1 Prediction performance 20
4.2 Results 20
Chapter V Discussion 22
Chapter VI Conclusion 25
References 26

List of Tables

Table 1 the performance of our transfer learning classifier for different color saturation 21
Table 2 the performance of our transfer learning classifier for color and grayscale dataset 21

List of Figures

Figure 1 workflow of our study 8
Figure 2 The interface of whole slide image system 10
Figure 3 Cropped image blocks of positive acid-fast stained bacilli 11
Figure 4 Negative images with no acid-fast stained area and splendid reddish colored images. These pictures showed reddish to violaceous material and other features of negative acid-fast stained area. 12
Figure 5 Negative images showing features that ise difficult to distinguish from positive stained mycobacteria bacilli. The foamy structure that showed slit-like lesions. The amourphous background stain of collogen fibers showed rod-like reddish figures is also confusing. Aggregation of mast cells stained reddish and could be confused with postive results. The granules around mast cells also mimic positive stained bacilli. 13
Figure 6 examples of our training data for color and grayscale image blocks. In this experiment, negative images with different features were included for classification. The colored information was removed to test the classifier if it could still distinguish bacilli images from negative ones. 14
Figure 7 Convolutional neural networks are designed to process data in the form of multiple layers. The concepts include local connections, shared weights, pooling and the use of several layers. In our study, transfer learning method was conducted by removing of final layers with replacement of our target image datasets. 16
Figure 8 Diagram of workflow of histopathology image processing. (A) Scanning of digital whole slide pathology images. (B) tiling up the image to a size that fit mycobacteria bacilli (20X20 pixels) (C) Selection of representative images and classified into positive or negative mycobacteria bacilli. (D) CNN model AlexNet learn features from 15 million labeled natural images. The final three layers of AlexNet were replaced with our target image dataset. The performance of classifier was evaluated. 19

[1]W. H. Organization, "Global Tuberculosis Report," Geneva, 2017.
[2]K. L. Winthrop et al., "An Outbreak of Mycobacterial Furunculosis Associated with Footbaths at a Nail Salon," New England Journal of Medicine, vol. 346, no. 18, pp. 1366-1371, 2002/05/02 2002.
[3]P. M. LeBlanc, K. A. Hollinger, and K. C. Klontz, "Tattoo Ink–Related Infections — Awareness, Diagnosis, Reporting, and Prevention," New England Journal of Medicine, vol. 367, no. 11, pp. 985-987, 2012/09/13 2012.
[4]E. Henkle and K. L. Winthrop, "Nontuberculous mycobacteria infections in immunosuppressed hosts," Clinics in chest medicine, vol. 36, no. 1, pp. 91-99, 2015.
[5]D. E. Griffith et al., "An Official ATS/IDSA Statement: Diagnosis, Treatment, and Prevention of Nontuberculous Mycobacterial Diseases," American Journal of Respiratory and Critical Care Medicine, vol. 175, no. 4, pp. 367-416, 2007.
[6]L. B. Reller, M. P. Weinstein, Y. M. Hale, G. E. Pfyffer, and M. Salfinger, "Laboratory Diagnosis of Mycobacterial Infections: New Tools and Lessons Learned," Clinical Infectious Diseases, vol. 33, no. 6, pp. 834-846, 2001.
[7]S. Sharma and A. Mohan, "Extrapulmonary tuberculosis," Indian Journal of Medical Research, vol. 120, pp. 316-353, 2004.
[8]A. Hernandez Solis et al., "Skin biopsy: a pillar in the identification of cutaneous Mycobacterium tuberculosis infection," (in eng), J Infect Dev Ctries, vol. 6, no. 8, pp. 626-31, Aug 21 2012.
[9]A. C. Laga, D. A. Milner, Jr., and S. R. Granter, "Utility of acid-fast staining for detection of mycobacteria in cutaneous granulomatous tissue reactions," Am J Clin Pathol, vol. 141, no. 4, pp. 584-6, Apr 2014.
[10]A. A. Renshaw and E. W. Gould, "Thrombocytosis is associated with Mycobacterium tuberculosis infection and positive acid-fast stains in granulomas," Am J Clin Pathol, vol. 139, no. 5, pp. 584-6, May 2013.
[11]D. M. Lewinsohn et al., "Official American Thoracic Society/Infectious Diseases Society of America/Centers for Disease Control and Prevention Clinical Practice Guidelines: Diagnosis of Tuberculosis in Adults and Children," Clinical Infectious Diseases, vol. 64, no. 2, pp. e1-e33, 2017.
[12]J. Griffin and D. Treanor, "Digital pathology in clinical use: where are we now and what is holding us back?," Histopathology, vol. 70, no. 1, pp. 134-145, Jan 2017.
[13]M. Silver Spring. ( April 12, 2017). FDA allows marketing of first whole slide imaging system for digital pathology [news release]. Available: https://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/ucm552742.htm
[14]J. Ho et al., "Can digital pathology result in cost savings? A financial projection for digital pathology implementation at a large integrated health care organization," J Pathol Inform, vol. 5, no. 1, p. 33, 2014.
[15]C. Higgins, "Applications and challenges of digital pathology and whole slide imaging," Biotech Histochem, vol. 90, no. 5, pp. 341-7, Jul 2015.
[16]T. J. Fuchs and J. M. Buhmann, "Computational pathology: challenges and promises for tissue analysis," (in eng), Comput Med Imaging Graph, vol. 35, no. 7-8, pp. 515-30, 2011 Oct-Dec 2011.
[17]D. R. Snead et al., "Validation of digital pathology imaging for primary histopathological diagnosis," (in eng), Histopathology, vol. 68, no. 7, pp. 1063-72, Jun 2016.
[18]M. N. Kent et al., "Diagnostic Accuracy of Virtual Pathology vs Traditional Microscopy in a Large Dermatopathology Study," JAMA Dermatology, vol. 153, no. 12, p. 1285, 2017.
[19]S. Mukhopadhyay et al., "Whole Slide Imaging Versus Microscopy for Primary Diagnosis in Surgical Pathology: A Multicenter Blinded Randomized Noninferiority Study of 1992 Cases (Pivotal Study)," (in eng), Am J Surg Pathol, Sep 28 2017.
[20]S. Hoo-Chang et al., "Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning," IEEE transactions on medical imaging, vol. 35, no. 5, p. 1285, 2016.
[21]A. Madabhushi and G. Lee, "Image analysis and machine learning in digital pathology: Challenges and opportunities," Med Image Anal, vol. 33, pp. 170-5, Oct 2016.
[22]D. Shen, G. Wu, and H.-I. Suk, "Deep Learning in Medical Image Analysis," Annual review of biomedical engineering, vol. 19, pp. 221-248, 03/09 2017.
[23]G. Litjens et al., "A survey on deep learning in medical image analysis," Medical image analysis, vol. 42, pp. 60-88, 2017.
[24]G. Litjens et al., "Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis," Sci Rep, vol. 6, p. 26286, May 23 2016.
[25]K. H. Yu et al., "Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features," (in eng), Nat Commun, vol. 7, p. 12474, Aug 2016.
[26]B. Ehteshami Bejnordi, M. Veta, P. Johannes van Diest, and et al., "Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer," JAMA, vol. 318, no. 22, pp. 2199-2210, 2017.
[27]M. G. Forero, F. Sroubek, and G. Cristóbal, "Identification of tuberculosis bacteria based on shape and color," Real-Time Imaging, vol. 10, no. 4, pp. 251-262, 2004.
[28]R. Khutlang et al., "Classification of Mycobacterium tuberculosis in images of ZN-stained sputum smears," IEEE Trans Inf Technol Biomed, vol. 14, no. 4, pp. 949-57, Jul 2010.
[29]P. Sadaphal, J. Rao, G. W. Comstock, and M. F. Beg, "Image processing techniques for identifying Mycobacterium tuberculosis in Ziehl-Neelsen stains," (in eng), Int J Tuberc Lung Dis, vol. 12, no. 5, pp. 579-82, May 2008.
[30]Y. N. Law et al., "Low cost automated whole smear microscopy screening system for detection of acid fast bacilli," PLoS One, vol. 13, no. 1, p. e0190988, 2018.
[31]J. Chang et al., "Automated Tuberculosis Diagnosis Using Fluorescence Images from a Mobile Microscope," Medical image computing and computer-assisted intervention : MICCAI ... International Conference on Medical Image Computing and Computer-Assisted Intervention, vol. 15, no. Pt 3, pp. 345-352, 2012.
[32]M. G. Forero, G. Cristobal, and M. Desco, "Automatic identification of Mycobacterium tuberculosis by Gaussian mixture models," (in eng), J Microsc, vol. 223, no. Pt 2, pp. 120-32, Aug 2006.
[33]K. Veropoulos, G. Learmonth, C. Campbell, B. Knight, and J. Simpson, "Automated identification of tubercle bacilli in sputum: a preliminary investigation," Analytical and quantitative cytology and histology, vol. 21, no. 4, pp. 277-282, 1999.
[34]E. Priya and S. Srinivasan, "Automated Identification of Tuberculosis Objects in Digital Images Using Neural Network and Neuro Fuzzy Inference Systems," Journal of Medical Imaging and Health Informatics, vol. 5, no. 3, pp. 506-512, 2015.
[35]E. Priya and S. Srinivasan, "Automated object and image level classification of TB images using support vector neural network classifier," Biocybernetics and Biomedical Engineering, vol. 36, no. 4, pp. 670-678, 2016.
[36]P. J. Tadrous, "Computer-assisted screening of Ziehl-Neelsen-stained tissue for mycobacteria. Algorithm design and preliminary studies on 2,000 images," Am J Clin Pathol, vol. 133, no. 6, pp. 849-58, Jun 2010.
[37]M. K. Osman, M. Y. Mashor, and H. Jaafar, "Performance Comparison of Extreme Learning Machine Algorithms for Mycobacterium tuberculosis Detection in Tissue Sections," Journal of Medical Imaging and Health Informatics, vol. 2, no. 3, pp. 307-312, 2012.
[38]Y. Xiong, X. Ba, A. Hou, K. Zhang, L. Chen, and T. Li, "Automatic detection of mycobacterium tuberculosis using artificial intelligence," Journal of thoracic disease, vol. 10, no. 3, p. 1936, 2018.
[39]Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436-44, May 28 2015.
[40]J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, "Imagenet: A large-scale hierarchical image database," in Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, 2009, pp. 248-255: IEEE.
[41]J. Yosinski, J. Clune, Y. Bengio, and H. Lipson, "How transferable are features in deep neural networks?," in Advances in neural information processing systems, 2014, pp. 3320-3328.
[42]J. M. Brown et al., "Automated Diagnosis of Plus Disease in Retinopathy of Prematurity Using Deep Convolutional Neural Networks," JAMA ophthalmology, 2018.
[43]H. Haenssle et al., "Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists," Annals of Oncology, 2018.
[44]A. Esteva et al., "Dermatologist-level classification of skin cancer with deep neural networks," Nature, vol. 542, no. 7639, pp. 115-118, Feb 02 2017.
[45]L. D. Nguyen, D. Lin, Z. Lin, and J. Cao, "Deep CNNs for microscopic image classification by exploiting transfer learning and feature concatenation," in 2018 IEEE International Symposium on Circuits and Systems (ISCAS), 2018, pp. 1-5.
[46]A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in neural information processing systems, 2012, pp. 1097-1105.
[47]O. Russakovsky et al., "Imagenet large scale visual recognition challenge.," 2014.