臺灣博碩士論文加值系統

於現今醫療過程中，醫護人員經常會利用視診的方式，直接根據病患的外部表徵判斷病患是否患有某些疾病。例如眼白與皮膚的顏色可以判斷黃疸、面部血色與唇色可判斷貧血、面部對稱性可以判斷中風、耳朵位置與眼睛角度可以判斷唐氏症等等。視診於病患進入診間後即可開始，醫護人員視診的過程快速，且不須侵入式的檢測或是透過儀器拍攝體內影像。然而，醫護人員一般需要經過長時間的訓練與實務經驗，才有能力獨立進行視診或其他醫療行為。本論文以黃疸為範例，透過智慧型手機拍攝病患的影像，使用深度學習的方式根據影像中的眼白顏色來訓練黃疸預測模型，以此協助醫師於醫療行為中的視診或病患於家中的自我監測。

In the medical process, medical staff often directly judge whether the patient suffers from a certain disease according to the patient’s external characteristics through inspections. For example, the color of the sclera and skin can judge jaundice, the color of lips and skin can judge anemia, the symmetry of the face can judge stroke, and the position of the ears and the angle of the eyes can judge Down syndrome. The inspection can be started after the patient enters the consultation room. The inspection process is fast and does not require invasive testing or taking ultrasound or endoscopic images. However, medical staff generally need a long period of training and practical experience to be able to independently perform inspections or other medical actions. In this article, we take jaundice as an example, use deep learning and machine learning methods to train a jaundice prediction model based on patient images taken by smartphones.

口試委員會審定書 i
致謝 iii
摘要 iv
Abstract v
Contents vi
List of Figures ix
List of Tables xi
Chapter 1 Introduction 1
1.1 Inspection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Quantification of Patient’s External Characteristics . . . . . . . . . . 2
1.3 Total Bilirubin and Jaundice . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Measuring Method of Total Bilirubin . . . . . . . . . . . . . . . . . 4
Chapter 2 Related Work 7
2.1 Jaundice Eye Color Index (JECI) . . . . . . . . . . . . . . . . . . . 7
2.2 BiliCam . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.3 Scleral-Conjunctival Bilirubin (SCB) . . . . . . . . . . . . . . . . . 8
2.4 MobileCam-Estimated Bilirubin Concentration (MCB) . . . . . . . . 9
2.5 BiliScreen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.6 Summary of Related Work . . . . . . . . . . . . . . . . . . . . . . . 10
Chapter 3 Method 11
3.1 Architecture Overview . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.2 Standardized Light Source . . . . . . . . . . . . . . . . . . . . . . . 12
3.3 Use RAW Format Images . . . . . . . . . . . . . . . . . . . . . . . 14
3.4 Automatic ColorChecker Detection . . . . . . . . . . . . . . . . . . 15
3.5 Image Color Correction . . . . . . . . . . . . . . . . . . . . . . . . 17
3.5.1 Macbeth ColorChecker . . . . . . . . . . . . . . . . . . . . . . . . 17
3.5.2 Color Correction Methods . . . . . . . . . . . . . . . . . . . . . . 18
3.5.2.1 Linear Regression . . . . . . . . . . . . . . . . . . . . 18
3.5.2.2 Polynomial Regression . . . . . . . . . . . . . . . . . 19
3.5.2.3 Root Polynomial Regression . . . . . . . . . . . . . . 20
3.5.2.4 Support Vector Regression . . . . . . . . . . . . . . . 21
3.5.2.5 Multilayer Perceptron . . . . . . . . . . . . . . . . . . 21
3.5.2.6 Convolutional Neural Network . . . . . . . . . . . . . 23
3.5.3 Evaluation and Selection of Color Correction Methods . . . . . . . 24
3.5.4 JaundiceChecker . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.6 Facial Landmark Detection . . . . . . . . . . . . . . . . . . . . . . . 27
3.7 Sclera Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.8 Jaundice Classification and Regression . . . . . . . . . . . . . . . . 30
Chapter 4 Experiment Result 33
4.1 Color Correction Experiment . . . . . . . . . . . . . . . . . . . . . . 33
4.2 ColorChecker Detection and Sclera Segmentation . . . . . . . . . . . 34
4.3 Jaundice Classification and T-Bil Regression . . . . . . . . . . . . . 36
4.3.1 Jaundice Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.3.2 Using RAW Image and Color Correction . . . . . . . . . . . . . . . 38
4.3.3 Using Controlled Light Source . . . . . . . . . . . . . . . . . . . . 38
4.3.4 Compare SVM/SVR with ResNet . . . . . . . . . . . . . . . . . . 40
4.3.5 Results on Multiple Smartphones . . . . . . . . . . . . . . . . . . . 41
4.3.6 Compare SVM with Professional Medical Staff . . . . . . . . . . . 43
Chapter 5 Conclusion 45
References 47

[1] A. Abdalla, H. Cen, E. Abdel-Rahman, L. Wan, and Y. He. Color calibration of proximal sensing rgb images of oilseed rape canopy via deep learning combined with k-means algorithm. Remote Sensing, 11(24):3001, 2019.
[2] M. L. Barr, G. Guo, S. E. Colby, and M. D. Olfert. Detecting body mass index from a facial photograph in lifestyle intervention. Technologies, 6(3):83, 2018.
[3] J. C. Berry, N. Fahlgren, A. A. Pokorny, R. S. Bart, and K. M. Veley. An automated, high-throughput method for standardizing image color profiles to improve image-based plant phenotyping. PeerJ, 6:e5727, 2018.
[4] V. K. Bhutani, G. R. Gourley, S. Adler, K. Bill, C. Dalin, and L. H. Johnson. Nonin-vasive measurement of total serum bilirubin in a multiracial predischarge newborn population to assess the risk of severe hyperbilirubinemia. Pediatrics, 2000.
[5] F. Bousefsaf, A. Pruski, and C. Maaoui. 3d convolutional neural networks for remote pulse rate measurement and mapping from facial video. Applied Sciences, 9(20):4364, 2019.
[6] L. De Greef, M. Goel, M. J. Seo, E. C. Larson, J. W. Stout, J. A. Taylor, and S. N. Patel. Bilicam: using mobile phones to monitor newborn jaundice. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pages 331–342, 2014.
[7] H. H. Falk and O. D. Jensen. A machine learning approach for jaundice detection using color corrected smartphone images. Master’s thesis, NTNU, 2018.
[8] G. D. Finlayson, M. Mackiewicz, and A. Hurlbert. Color correction using root-polynomial regression. IEEE Transactions on Image Processing, 24(5):1460–1470, 2015.
[9] C. A. Hartanto and A. Wibowo. Development of mobile skin cancer detection using faster r-cnn and mobilenet v2 model. In 2020 7th International Conference on Information Technology, Computer, and Electrical Engineering (ICITACEE), pages 58–63. IEEE, 2020.
[10] S. Jiayun, X. Chunming, Y. Ke, Y. ZHANG, W. Yiqin, Y. Haixia, and Q. Peng. Tongue image color correction method based on root polynomial regression. In 2019 IEEE 8th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), pages 1337–1342. IEEE, 2019.
[11] T. S. Leung, F. Outlaw, L. W. MacDonald, and J. Meek. Jaundice eye color index (jeci): quantifying the yellowness of the sclera in jaundiced neonates with digital photography. Biomedical optics express, 10(3):1250–1256, 2019.
[12] Y. Lu, X. Li, L. Zhuo, J. Zhang, and H. Zhang. Dccn: A deep-color correction network for traditional chinese medicine tongue images. In 2018 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pages 1–6. IEEE, 2018.
[13] D. R. Lucio, R. Laroca, E. Severo, A. S. Britto, and D. Menotti. Fully convolutional networks and generative adversarial networks applied to sclera segmentation. In 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS), pages 1–7. IEEE, 2018.
[14] B. Luo, J. Shen, S. Cheng, Y. Wang, and M. Pantic. Shape constrained network for eye segmentation in the wild. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1952–1960, 2020.
[15] A. Mariakakis, M. A. Banks, L. Phillipi, L. Yu, J. Taylor, and S. N. Patel. Biliscreen: smartphone-based scleral jaundice monitoring for liver and pancreatic disorders. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 1(2):1–26, 2017.
[16] J. Niu, C. Zhao, and G.-Z. Li. A comprehensive study on color correction for medical facial images. International Journal of Machine Learning and Cybernetics, 10(5):935–947, 2019.
[17] J.-L. Niu, C.-B. Zhao, and G.-Z. Li. A novel color correction framework for facial images. In 2014 International Conference on Medical Biometrics, pages 47–54. IEEE, 2014.
[18] F. Outlaw, M. Nixon, O. Odeyemi, L. W. MacDonald, J. Meek, and T. S. Leung. Smartphone screening for neonatal jaundice via ambient-subtracted sclera chromaticity. PloS one, 15(3):e0216970, 2020.
[19] B. Qin, L. Liang, J. Wu, Q. Quan, Z. Wang, and D. Li. Automatic identification of down syndrome using facial images with deep convolutional neural network. Diagnostics, 10(7):487, 2020.
[20] F. Qingchun, C. Jian, and W. Xiu. Seedling image color correction method under natural illumination in greenhouse. IFAC-PapersOnLine, 51(17):81–84, 2018.
[21] A. Riskin, A. Tamir, A. Kugelman, M. Hemo, and D. Bader. Is visual assessment of jaundice reliable as a screening tool to detect significant neonatal hyperbilirubinemia? The Journal of pediatrics, 152(6):782–787, 2008.
[22] M. Vitek, P. Rot, V. Štruc, and P. Peer. A comprehensive investigation into sclera biometrics: a novel dataset and performance study. Neural Computing & Applications, 32(24), 2020.
[23] X. Wang and D. Zhang. A comparative study of color correction algorithms for tongue image inspection. In International Conference on Medical Biometrics, pages 392–402. Springer, 2010.
[24] K. Zhang, Z. Zhang, Z. Li, and Y. Qiao. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters, 23(10):1499–1503, 2016.