臺灣博碩士論文加值系統

多年來,瀝青路面的破損已成為全球性重大問題。這主要是由於人為錯誤、車輛數量增加
和氣候變遷所導致。
在各種道路損壞中,坑洞是最嚴重的㇐種,對車輛造成重大損害,導致車輛失控,引發致
命事故。以往的研究主要透過提供坑洞數據的傳統方法進行坑洞檢測(來自網路和智慧型
手機的數據)。然而傳統方法的坑洞數據種類有限,為解決此限制,並提供更廣泛的坑洞
圖像,本研究採用了深度學習方法,引入了穩定擴散和深度卷積生成對抗網絡(DCGAN)
兩種深度學習方法。這兩種模型具有生成特定物體之逼真的合成圖像的能力。
本研究旨在透過增加數據庫中坑洞圖像的多樣性,使模型能準確檢測不同類型的坑洞。我
們的目標是為了用來檢測被標記的坑洞之模型提供特定類別的多樣坑洞圖像。為此,我們
選擇了最合適的算法,即 You Only Look Once 第 8 版算法(YOLOv8),這是 YOLO 家
族的先進版本。
本研究使用 YOLOv8 方法進行了三個實驗。第㇐個實驗採用傳統方法,從網路和智慧型
手機的相機收集大量坑洞圖像,並根據特徵將其分類為五種類型。第二個實驗結合了傳統
方法和穩定擴散兩種方法,收集了十種不同坑洞類型的圖像,創建了更全面的數據集,以
提高檢測準確性。第三個實驗整合了來自傳統方法、穩定擴散和 DCGAN 模型的圖像,將
DCGAN 模型生成的五種坑洞圖像根據其特徵進行標記,並進㇐步囊括先前以傳統方式及
穩定擴散方式收集到的十種坑洞圖像來訓練模型,以期獲得更好的檢測效果。


VII


研究使用 YOLOv8 技術訓練了三個模型。第㇐個模型的準確性較低,但在預處理圖像並
結合穩定擴散生成的圖像後,準確性略有提高。此外,我們結合了三種模型的圖像所訓練
的第三個模型,在準確度方面獲得了顯著結果。第三個識別模型在坑洞類型 6、7、8、9、
11、12、13、14 和 15 的準確率分別達到 97%、97%、99%、97%、98%、99%、98%、
97%及 98%,而類型 10 的準確率為 94%。
本研究結果顯示,結合多樣化的圖像來源和先進的深度學習技術可以顯著提高坑洞檢測模
型的準確性。

Over the years asphalt road pavement has become the major issue around the world. This occurs due to human errors and increasing number of vehicles and climate change. There are many kinds of road damages occur on the road, one of the most severe damage is road pothole. It causes significant destruction to vehicles and lose control of vehicles that lead to fatal accidents. Previous studies have been shown the detection of pothole by providing the pothole dataset traditional method (dataset from internet and smartphones). Limited varieties of pothole dataset from traditional method, for providing a wide range of pothole images we have utilized the deep learning method. Introducing stable diffusion and deep convolutional generative adversarial network deep learning method. These two stable diffusions and DCGAN models have ability to generate realistic fake image of particular object. Our concern is to increasing the different variety of pothole images in our dataset so that we could train our model in order to detect the distinct pothole. Our aim is to feed diverse pothole images with specific class type to model in detection of labelled pothole. During this we have chosen You Only Look Once version 8 algorithm (YOLOv8 algorithm) for detection process. Our model is based on YOLOv8 technique, YOLOv8 is latest version 8 from the YOLO family. Three experiments have performed using YOLOv8 approach, in the first experiment we have collected pothole images from internet and smartphone’s camera as traditional method for detection of pothole with specified 5 class types based on pothole features. In the second experiment we have gathered pothole images including 10 class types pothole images from stable diffusion and traditional method both, preparing the combination of pothole image dataset and feeding model to expecting the best detection accuracy. The third experiment is based on combining three different group of pothole image from traditional method, stable diffusion model and DCGAN model. Similarly, we specified 5 types of pothole image generated from DCGAN model and labeling them according to features of DCGAN generated pothole image dataset further including previous 10 type of pothole images from traditional and stable diffusion model. Preparing dataset and training the model and looking for better detection outcomes. We have trained three model using YOLOv8 technique, first recognition model has lower accuracy then preprocessed the images and combining with stable diffusion generated images got slightly improved accuracy. Furthermore, we combining the three model images and trained the third mode and it has produced significant result in terms of accuracy. Third recognition model has given the critical accuracy for Type-6, type-7, type-8, type-9, type-11, type-12, type-13, type-14, type-15 which are 97%, 97%, 99%, 97%, 98%, 99%, 98%, 97%, 98%, 94% respectively.  

Abstract …. III
Abstract (Chinese) …. V
Table of Contents……………………………………………………………………………….VII
Lists of Figures and Tables ….. IX
Chapter 1 Introduction ….. 1
1.1 Research motivation ….. 2
1.2 Research objectives … 3
1.3 Related issues … 5
Chapter 2 Literature Review ….. 6
2.1 Convolution neural network ….. 6
2.2 Object detection...….. 9
2.3 Deep learning … 12
2.4 YOLO algorithm evaluation...….. 13
Chapter 3 Research Method /System Design and Development ….. 16
3.1 YOLOv8 … 16
3.2 YOLOv8 Architecture: ….. 17
3.3 Image Dataset: … 18
3.4 Image Classification : …. 18
3.5 System Equipment ….. 18
Chapter 4 Data analysis …. 38
4.1 First model (Traditional Method):...… 38
4.2 Second model (Traditional and Stable diffusion method) ….. 44
4.3 Third model (Traditional, Stable diffusion and GAN model) … 50
Chapter 5 Findings and Discussions … 57
5.1 First recognition model detection accuracy: ….. 57
5.2 Second recognition model detection accuracy ….. 57
5.3 Third recognition model detection accuracy ….. 58
5.4 Limitations: …. 59
Chapter 6 Conclusion ….. 60
Reference ….. 61


Lists of Figures and Tables


Figure 3.2- 1 YOLOv8 architecture visualization given by GitHub RangeKing....................18
Figure 3.3.1- 2 Road pothole image dataset taken from internet source...…20
Figure 3.3.2- 3 Pothole image dataset generated by Stable diffusion model...….21
Figure 3.3.3- 4 Image dataset generated by DCGAN model...…23
Figure 3.3.3- 5 Generated Single image output from DCGAN with 1000x1000 resolution...….25
Figure 3.4- 6 Type-1 Road pothole...….26
Figure 3.4- 7 Type-2 Road pothole...…26
Figure 3.4- 8 Type-3 Road pothole...…27
Figure 3.4- 9 Type-4 Road pothole...…27
Figure 3.4- 10 Type-5 Road pothole...….28
Figure 3.4- 11 Type-6 Road pothole...….28
Figure 3.4- 12 Type-7 Road pothole...….29
Figure 3.4- 13 Type-8 Road pothole...….30
Figure 3.4- 14 Type-9 Road pothole...….30
Figure 3.4- 15 Type-10 Road pothole...…31
Figure 3.4- 16 Type-11 Road pothole...…31
Figure 3.4- 17 Type-12 Road pothole...…32
Figure 3.4- 18 Type-13 Road pothole...…32
Figure 3.4- 19 Type-14 Road pothole...…33
Figure 3.4- 20 Type-15 Road pothole...…34
Figure 3.4- 21 Road pothole recognition system design...….39
Figure 4.1-21 Training image set by class labels of first model...…41
Figure 4.1-22 Validation image set of first model...….42
Figure 4.1-23 F1-Confidence curve between classes 1 to 5 of first model...…43
Figure 4.1-24 Precision Confidence curve between classes 1 to 5 of first model...….44
Figure 4.1-25 Recall Confidence between class 1 to 5 of first model...…45
Figure 4.1-26 Precision Recall between class 1 to 5 of first model...….46
Figure 4.1-27 Confusion matrix of 1 to 5 classes of first model...…47
Figure 4.2-28 Training image set by class labels of second model...…49
Figure 4.2-29 Valid image set with detection confidence of second model...….50
Figure 4.2-30 F1-Confidence curve between classes 1 to 10 of second model...…51
Figure 4.2-31 Precision Confidence curve between classes 1 to 10 of second model...…52
Figure 4.2-32 Recall-Confidence between class 1 to 10 of second model...…53
Figure 4.2-33 Precision Recall between class 1 to 10 of second model...…54
Figure 4.2-34 Confusion matrix of 1 to 10 classes of second model...….55
Figure 4.3-35 Training image set by class labels of third model...…57
Figure 4.3-36 Valid image set with detection confidence of third model...….58
Figure 4.3-37 F1-Confidence curve between classes 1 to 15 of third model...…59
Figure 4.3-38 Precision Confidence curve between classes 1 to 15 of third model...…60
Figure 4.3-39 Recall-Confidence curve between class 1 to 15 of third model...…..61
Figure 4.3-40 Precision-Recall curve between class 1 to 15 of third model...….62
Figure 4.3-41 Confusion matrix of 1 to 15 classes of third model...…63
Table 1 Comparative analysis of detection accuracy three models …..57

1. Chen, D., Chen, N., Zhang, X., Guan, Y. (2022). “Real-Time Road Pothole Mapping Based on Vibration Analysis in Smart City”, IEEE Xplore, Vol 15, pp. 6972- 6984.
2. Jordan, D, N., Paine, S., Mishra, A, K., Pidanic, J. (2023). “Road to Repair (R2R): An Afrocentric Sensor-Based Solution to Enhanced Road Maintenance”, IEEE Xplore, Vol 11, pp. 6010-6017.
3. Zhao, K., Liu, J., Wang, Q., Wu, X., Tu, J. (2022). “Road Damage Detection from Post-Disaster High-Resolution Remote Sensing Images Based on TLD Framework”, IEEE Xplore, Vol 10, pp. 43552-43561.
4. Yin, Y., Fu, W., Ma, X., Yu, J., Li, X., Dong, Z. (2024). “Road Surface Pits and Speed Bumps Recognition Based on Acceleration Sensor”, IEEE Xplore, Vol 24, No. 7, pp. 10670-10679.
5. Ayanoglu, E., Davaslioglu, K., Sagduyu, Y, E. (2022). “Machine Learning in NextG Networks via Generative Adversarial Networks”, IEEE Xplore, Vol 8, No. 2, pp. 480-501.
6. Chen, L, H., Bampis, C, G., Li, Zhi., Bovik, A, C. (2020). “Learning to Distort Images Using Generative Adversarial Networks”, IEEE Xplore, Vol 27, pp. 2144-2144.
7. Li, M., Lin, J., Meng, C., Ermon, S., Han, S., Zhu, J, Y. (2023). “Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Models”, IEEE Xplore, Vol. 45 No. 12, pp. 14465-14480.
8. Alhussan, A, A., Khafaga, D, S., El-kenawy, E, M., Ibrahim, A., Eid, M, M., Abdelhamid, A, A. (2022). “Pothole and Plain Road Classification Using Adaptive Mutation Dipper Throated Optimization and Transfer Learning for Self-Driving Cars”, IEEE Xplore, Vol.10, pp. 84188- 84211.
9. Li, Y., Liu, C., Gao, Q., Wu, D., Li, F., Du, Y. (2022). “ConTrack Distress Dataset: A Continuous Observation for Pavement Deterioration Spatio-Temporal Analysis”, IEEE Xplore, Vol. 23, No. 12, pp. 25004- 25017.
10. Raja, G., Anbalagan, S., Senthilkumar, S., Dev, K., Qureshi, N, M, F. (2022). “SPAS: Smart Pothole-Avoidance Strategy for Autonomous Vehicles”, IEEE Xplore, Vol. 23, No. 10, pp. 19827- 19836.
11. Sharma, N., Garg, R, D. (2023). “Real-Time IoT-Based Connected Vehicle Infrastructure for Intelligent Transportation Safety”, IEEE Xplore, Vol. 24, No. 8, pp. 8339- 8347.
12. Siddiqui, I., Mazhar, S., Hassan, N., Sultani, W. (2023). “Fine-Grained Road Quality Monitoring Using Deep Learning”, IEEE Xplore, Vol. 24, No. 10, pp. 10691- 10701.
13. Chen, L., He, X., Zhao, X., Li, H., Huang, Y., Zhou, B., Chen, W., Li, Y., Wen, C., Wang, C. (2024). “GoComfort: Comfortable Navigation for Autonomous Vehicles Leveraging High-Precision Road Damage Crowdsensing”, IEEE Xplore, Vol. 22, No. 11, pp. 6477- 6494.
14. Hijji, M., Iqbal, R., Pandey, A, K., Doctor, F., Karyotis, C., Rajeh, W., Alsheri, A., Aradah, F. (2023). “6G Connected Vehicle Framework to Support Intelligent Road Maintenance Using Deep Learning Data Fusion”, IEEE Xplore, Vol. 24, No. 7, pp. 7726- 7735.
15. Carlos, M, R., Gonzalez, L, C., Wahlstrome, J., Cornejo, R., Martinez, F. (2020). “Becoming Smarter at Characterizing Potholes and Speed Bumps from Smartphone Data — Introducing a Second-Generation Inference Problem”, IEEE Xplore, Vol. 20, No. 2, pp. 366- 376.
16. Yan, K., Zhang, Z. (2021). “Automated Asphalt Highway Pavement Crack Detection Based on Deformable Single Shot Multi-Box Detector Under a Complex Environment”, IEEE Xplore, Vol. 9, pp. 150925- 150938.
17. Dhiman, A., Klette, R. (2020). “Pothole Detection Using Computer Vision and Learning”, IEEE Xplore, Vol. 21, No. 8, pp. 3536- 3550.
18. Fan, R., Wang, H., Wang, Y., Liu, M., Pitas, I. (2021). “Graph Attention Layer Evolves Semantic Segmentation for Road Pothole Detection: A Benchmark and Algorithms”, IEEE Xplore, Vol. 30, pp. 8144- 8154.
19. Qu, Z., Chen, W., Wang, S, Y., Yi, T, M., Liu, L. (2022). “A Crack Detection Algorithm for Concrete Pavement Based on Attention Mechanism and MultiFeatures Fusion”, IEEE Xplore, Vol. 23, No. 8, pp. 11710- 11719.
20. Saisree, C., Kumaran, U. (2023). Science Direct “Pothole Detection Using Deep Learning Classification Method”, Science Direct, pp. 2144-2152.
21. Tang, P., Wang, C., Wang, X., Liu, W., Zeng, W., Wang, J. (2020). “Object Detection in High Quality Video Object Linking”, IEEE Xplore, Vol 42, No. 5, pp. 1272-1278.
22. Xu, H., Su, X., Wang, Y., Cai, H., Cui, K., Chen, X., (2019). “Automatic Bridge Crack Detection Using a Convolutional Neural Network”, Applied Sciences, Vol 23. Hosny, K, M., El- Hady, W, M., Samy, F, M., Vrochidou, E., Papakostas G, A. (2023). “Multi-Class Classification of Plant Leaf Diseases Using Feature Fusion of Deep Convolutional Neural Network and Local Binary Pattern”, IEEE Xplore, Vol 11, pp. 62307-62317.
24. Liu, Y., Chen, W., Qu, H., Wang, T., Ji, J., Miao, K. (2020). “WALLACE: Weakly Supervised Learning of Deep Convolutional Neural Networks with Multiscale Evidence”, IEEE Xplore, Vol. 8, pp. 20449-20457.
25. Yang, J., Wang, W., Lin, G., Li, Q., Sun, Y., Sun Y. (2019). “Infrared Thermal Imaging-based Crack Detection Using Deep Learning”, IEEE Xplore, Vol. 7, pp. 182060-182077.
26. Li, S., Zhao, X. (2019). “Image0Based Concrete Crack Detection Using Convolutional Neural Network and Exhaustive Search Technique”, Hindawi Advance in Civil Engineering, Vol., No., pp. 1-12.
27. Flah, M., Suleiman, A, R., Nehd, M, L. (2020). “Classification and Quantification of Cracks in Concrete Structures Using Deep Learning Image-based Techniques”, Science Direct, Vol.
28. Dung, C, V., Sekiyaa, H., Hiranob, S., Okatanic, T., Mikie, C. (2019). “A vision-based method for crack detection in gusset plate welded joints of steel Bridges using Deep Convolutional Neural Network”, Science Direct, Vol, pp. 217-229.
29. Zou, Z., Chen, K., Shi, Z., Guo, Y., Ye, J. (2023). “Object Detection in 20 Years: A Survey”, IEEE Xplore, Vol. 111, No. 3, pp. 257-276. 30. Cai, Y., Luan, T., Gao, H., Wang, H., Chen, L., Li, Y., Sotelo, M, A., Li, Z. (2021). “YOLOv4-5D: An Effective and Efficient Object Detector for Autonomous Driving,” IEEE Xplore, Vol 70, pp.1-14.
31. Batchuluun, G., Kang, G, K., Nguyen, D, T., Pham, T, D., Arsalan, M., Park, K, R. “Deep Learning-Based Thermal Image Reconstruction and Object Detection,” IEEE Xplore, Vol 9, pp. 5951-5971.
32. Feng, M., Lu, H., Yu, Y. (2020). “Residual Learning for Salient Object Detection,” IEEE Xplore, Vol. 29, pp. 4696 - 4708.
33. Dewi, C., Chen, R, C., Liu, Y, T., Jiang, X., Hartomo, K, D. (2021), “YOLOV4 for Advanced traffic Sign Recognition with Synthetic Training Data Generated by Various GAN,” IEEE Xplore, Vol. 4, pp. 97228- 97242
34. Zhang, D., Han, J., Guo, G., Zhao, L. (2019). “Learning Object Detectors with Semi-Annotated weak labels,” IEEE Xplore, Vol. 29, NO. 12, pp. 3622-3635.
35. Li, X., Song, Dong, Y. (2020). “Hierarchical Feature Fusion Network for Salient Object Detection,” IEEE Xplore, Vol. 29, pp. 9165-9175.
36. Jiang, P., Pan, Z., Tu, X., Vasconcelos, N., Chen, B., Peng, J. (2020). “Super Diffusion for Salient Object Detection,” IEEE Xplore, Vol. 29, pp. 2903-2917.
37. Carrasco, D, P., Rashwan, H, A., Garcia, M, A., Puig, D. (2023). “T-YOLO: Tiny Vehicle Detection Based on YOLO and Multi-Scale Convolutional Neural Networks,” IEEE Xplore, Vol. 11, pp. 22430-22440.
38. Cheng et al 2022
39. Guo, J, M., Markoni, H., Lee, J, D. (2022). “BARnet: Boundary Aware Refinement Network for Crack Detection,” IEEE Xplore, Vol. 23, No. 7, pp. 7343-7358.
40. Shen, Y., Zhang, D., Song, Z., Jiang, X., Ye, Q. (2023). “Learning to Reduce Information Bottleneck for Object Detection in Arial Images,” IEEE Xplore, Vol. 20.
41. Han, G., Zhou, W., Sun, N., Liu, J., Li, X., (2019). “Feature Fusion and Adversary Occlusion Networks for Object Detection,” IEEE Xplore, Vol. 7, pp. 124854-124865.
42. Sun, Z, W., Hua, Z, X., Li, H, C., Zhong, H, Y. (2024). “Flying Bird Object Detection Algorithm in Surveillance Video Based on Motion,” IEEE Xplore, Vol. 73.
43. Hou, B., Ren, Z., Zhao, W., Wu, Q., Jiao, L. (2020). “Object Detection in HighResolution Panchromatic Images Using Deep Models and Spatial Template Matching,” IEEE Xplore, Vol. 58, No. 2, pp. 956 -970.
44. Li, S., Li, Y., Li, M., Xu, X. (2021). “Improved YOLOv5 for Infrared Image Object Detection,” IEEE Xplore, Vol. 9, pp. 141861-141875.

45. Moon, S., Jeon J., Kim, D., Kim, Y. (2023). “Swarm Reconnaissance Drone System for Real-Time Object Detection Over a Large Area,” IEEE Xplore, Vol. 11, pp. 23505-23516
46. Wang S., (2022). “Learning Nonlinear Feature Mapping via Constrained NonConvex Optimization for Unsupervised Salient Object Detection,” IEEE Xplore, Vol. 10, pp. 40743-40752.
47. Kaur, R., Singh, S. (2021). “A Comprehensive Review of Object Detection with Deep Learning,” Science Direct. Pp. 1-17.
48. Hasan, A, S, M, M., Sohel, F., Diepeveen, D., Laga, H., Jones, M, G, K. (2021). “A survey of Deep Learning Techniques for Weed Detection from Images,” Science Direct, pp. 1-21.
49. Do, T, H., Nguyen, D, M., Bekoulis, G., Munteanu, A., Deligiannis, N. (2021). “Graph Convolutional Neural Network with Node Transition Probability-Based Message Passing and DropNode regularization,” Science Direct, pp. 1-12.
50. Ji, S, J., Ling, Q, H., Han F. (2022). “An Improved Algorithm for Small Object Detection Based on YOLOv4 and Multi-Scale Contextual Information,” Science Direct. pp. 1-13.