臺灣博碩士論文加值系統

現今許多不同產業或生活中皆導入一些有關人工智慧與影像辨識的相關技術，可以減少許多人力與時間提升決策的速度與可靠度。在此篇研究中我們，提出一套基於物件區塊訓練方式。這裡我們利用芒果作為我們主要的資料集，當中根據不同的外觀品質可以分為三個等級，以及不同程度的五種瑕疵類別。以辨別芒果為例，人類在辨識芒果等級或瑕疵時常常以某區塊表面特徵為辨識依據，因此本論文有別於其他研究成果直接將整張圖片作為卷積神經網路輸入來分類的方式，提出一種以區塊的基礎的分類方法來辨別水果的瑕疵與分級。本論文根據超像素分群技術，將芒果主體影像拆分成數個區塊，並根據這些區塊進一步處理取得一些統計特徵，像是計算出缺陷區塊面積比例與數量、代表色、平均顏色等。後續再針對瑕疵類別基於知識蒸餾的神經網路框架進行芒果瑕疵種類辨識來分辨各區塊之品質。最終運用這些基於區塊的簡易統計特徵向量來訓練芒果等級分類器。根據芒果資料集實驗結果顯示，利用本論文所提出的統計特徵向量來預測芒果等級可以達到一定的準確度。如果進一步將我們所得到的特徵向量與ResNet50網路提取的芒果影像特徵結合後訓練，即便該向量在整體特徵中佔比不高亦能明顯提升訓練模型的預測能力。預期這套方法不只能應用在芒果，也可以應用在像是麵包、其他水果、或醫療影像中物件形狀與結構較無特定規則的自然物件分類。

關鍵字：區塊化影像辨識、物件偵測與分類、特徵工程

Nowadays, there are many different industries and life situations where the introduction of technologies related to artificial intelligence and image recognition can reduce labor and time to improve the speed and reliability of decision making. In this study, we propose a patch-based training approach. We use mangoes as our primary dataset, which can be classified into three classes according to their appearance quality and five types of defects of different degrees. In this thesis, we propose a patch-based classification method to identify and classify the defects of mangoes, which is different from other research results that directly use the whole picture as input to a convolutional neural network. We first use clustering to divide the mango into several superpixels as patches of mango, and we can further process these patches to obtain some statistical features, such as the proportion of defective parts on the surface, representative color, average color, etc. We use a knowledge distillation-based neural network framework for five types of mango defects classification training to obtain the target defect proportion. Finally, we can use these simple statistical feature vectors for image classification. According to the experimental results on the mango dataset, the statistical feature vector proposed in this paper can be used to predict the mango grade with a certain degree of accuracy. If we further combine proposed statistical feature vectors with the traditional color features extracted from the ResNet50 network, the predictive power of the trained model can also be improved. It is expected that this method can be applied not only to mangoes but also to bread, other fruits, or medical images, where the shape and structure of objects are not regular.

Keywords: Patch-based image classification, Object detection and classification, Feature engineering

書名頁 i
論文口試委員審定書 ii
摘要 iii
ABSTRACT iv
誌謝 v
目錄 vi
圖目錄 viii
表目錄 x
第一章、緒論 1
1.1 研究背景與動機 1
1.2 章節概要 3
第二章、文獻研究 4
2.1 物件偵測 4
2.2 區塊化影像辨識 5
2.3 知識蒸餾 6
2.4 商品辨識 8
2.5 水果辨識 9
第三章、方法 11
3.1 資料集簡介 11
3.2 研究方法概覽 11
3.3 資料前處理 12
3.3.1 三分類資料集的建立過程 13
(1)取得顯著圖(Saliency Map) 13
(2)物件區塊化 13
(3)圖像修復 14
(4)好壞區塊重新分類 16
3.3.2 六分類資料集建立過程與分類器訓練 16
3.4 芒果等級預測 19
第四章、驗證 22
4.1 實驗規劃 22
4.2 實驗步驟 22
(1)單一簡易統計特徵測試 22
(2)代表色數量交叉測試 23
(3)特徵提取網路測試 24
(4)簡易統計特徵與ResNet50特徵結合測試 25
4.3 實驗結果與討論 27
第五章、結論 28
參考文獻 29

[1] Dalal, Navneet, and Bill Triggs. "Histograms of oriented gradients for human detection." 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05). Vol. 1. IEEE, 2005.
[2] Hearst, M. A., Dumais, S. T., Osuna, E., Platt, J., & Scholkopf, B. (1998). Support vector machines. IEEE Intelligent Systems and their applications, 13(4), 18-28.
[3] Girshick, Ross., Donahue, Jeff., Darrell, Trevor., & Malik, Jitendra. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 580-587)..
[4] Ren, Shaoqing., He, Kaiming., Girshick, Ross., & Sun, Jian. (2016). Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE transactions on pattern analysis and machine intelligence, 39(6), 1137-1149.
[5] Redmon, Joseph., Divvala, Santosh., Girshick, Ross., & Farhadi, Ali. (2016). You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 779-788).
[6] Lucey, S., & Chen, Tsuhan. (2004, June). A GMM parts based face representation for improved verification through relevance adaptation. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. (Vol. 2, pp. II-II). IEEE.
[7] Xu, Chunyan., Wang, Tianjiang., Gao, Junbin., Cao, Shougang., Tao, Wenbing., & Liu, Fang. (2013). An ordered-patch-based image classification approach on the image Grassmannian manifold. IEEE transactions on neural networks and learning systems, 25(4), 728-737.
[8] Yan, Shuicheng., Zhou, Xi., Liu, MingLiu., Hasegawa-Johnson, Mark., & Huang, Thomas. S. (2008, June). Regression from patch-kernel. In 2008 IEEE Conference on Computer Vision and Pattern Recognition (pp. 1-8). IEEE.
[9] Anavi, Yaron., Kogan, Ilya., Gelbart, Elad., Geva, Ofer., & Greenspan, Hayit. (2015, August). A comparative study for chest radiograph image retrieval using binary texture and deep learning classification. In 2015 37th annual international conference of the IEEE engineering in medicine and biology society (EMBC) (pp. 2940-2943). IEEE.
[10] Wang, Guotai., Li, Wenqi., Zuluaga, Maria. A., Pratt, Rosalind., Patel, Premal. A., Aertsen, Michael., Doel, Tom., David, Anna. L., Deprest, Jan., Ourselin, Sébastien. & Vercauteren, Tom. (2018). Interactive medical image segmentation using deep learning with image-specific fine tuning. IEEE transactions on medical imaging, 37(7), 1562-1573.
[11] Hinton, Geoffrey., Vinyals, Oriol., & Dean, Jeff. (2015). Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531.
[12] Furlanello, Tommaso., Lipton, Zachary. C., Tschannen, Michael., Itti, Laurent., & Anandkumar, Anima. (2018). Born again neural networks. arXiv preprint arXiv:1805.04770.
[13] Nie, Xuecheng., Li, Yuncheng., Luo, Linjie., Zhang, Ning., & Feng, Jiashi. (2019). Dynamic kernel distillation for efficient pose estimation in videos. In Proceedings of the IEEE International Conference on Computer Vision (pp. 6942-6950).
[14] Zhang, Linfeng., Song, Jiebo., Gao, Anni., Chen, Jingwei., Bao, Chenglong., & Ma, Kaisheng. (2019). Be your own teacher: Improve the performance of convolutional neural networks via self distillation. In Proceedings of the IEEE International Conference on Computer Vision (pp. 3713-3722).
[15] Hao, Yu, Yanwei Fu, and Yu-Gang Jiang. "Take Goods from Shelves: A Dataset for Class-Incremental Object Detection." Proceedings of the 2019 on International Conference on Multimedia Retrieval. 2019
[16] Ji, Zhong., Kong, Qiankun., Wang, Haoran., & Pang, Yanwei. (2019, October). Small and Dense Commodity Object Detection with Multi-Scale Receptive Field Attention. In Proceedings of the 27th ACM International Conference on Multimedia (pp. 1349-1357).
[17] Zhao, Linyu., Yao, Jian., Du, Hongyu., Zhao, Jinjie., & Zhang, Ruijie. (2019, September). A Unified Object Detection Framework for Intelligent Retail Container Commodities. In 2019 IEEE International Conference on Image Processing (ICIP) (pp. 3891-3895). IEEE.
[18] Unay, Devrim., & Gosselin, Bernard. (2005, September). Thresholding-based segmentation and apple grading by machine vision. In 2005 13th European Signal Processing Conference (pp. 1-4). IEEE.
[19] Haidar, Abdulhamid., Dong, Haiwei., & Mavridis, Nikolaos. (2012, October). Image-based date fruit classification. In 2012 IV International Congress on Ultra Modern Telecommunications and Control Systems (pp. 357-363). IEEE.
[20] Hossain, M. Shamim., Al-Hammadi, Muneer., & Muhammad, Ghulam. (2018). Automatic fruit classification using deep learning for industrial applications. IEEE transactions on industrial informatics, 15(2), 1027-1034.
[21] Qin, Xuebin., Zhang, Zichen., Huang, Chenyang., Dehghan, Masood., Zaiane, Osmar. R., & Jagersand, Martin. (2020). U2-Net: Going deeper with nested U-structure for salient object detection. Pattern Recognition, 106, 107404.
[22] Irving, Benjamin. (2016). maskSLIC: regional superpixel generation with application to local pathology characterisation in medical images. arXiv preprint arXiv:1606.09518.
[23] Achanta, Radhakrishna., Shaji, Appu., Smith, Kevin., Lucchi, Aurélien., Fua, Pascal., & Süsstrunk, Sabine. (2010). Slic superpixels (No. REP_WORK).
[24] Yu, Jiahui., Lin, Zhe., Yang, Jimei., Shen, Xin., Lu, Xin., & Huang, Thomas. S. (2018). Generative image inpainting with contextual attention. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 5505-5514).ISO 690
[25] He, Kaiming., Zhang, Xiangyu., Ren, Shaoqing., & Sun, Jian. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).
[26] Selvaraju, R. Ramprasaath., Cogswell, Michael., Das, Abhishek., Vedantam, Ramakrishna., Parikh, Devi., & Batra, Dhruv. (2017). Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision (pp. 618-626)
[27] Chen, Tianqi., & Guestrin, Carlos. (2016, August). Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining (pp. 785-794)