臺灣博碩士論文加值系統

隨著近些年晶片技術的不斷進步以及未來趨勢，邊緣計算的相關應用越來越多元，透過NVIDIA 等大廠所研發推出的邊緣計算硬體與軟體支援。如常見的影像辨識，在未來的邊緣計算的應用也越來越多。本研究是基於深度學習技術，比較各種偵測演算法，並實現最適合煙火偵測的演算法。考慮的因素包括串流、速度、準確性和可攜性且再邊緣設備上需要考慮其效率、功耗等問題。本論文選擇 NVIDIA Jetson Xavier NX 開發板作為我們的邊緣辨識設備 即時影像串流或是攝影鏡頭連結到NVIDIA Jetson Xavier NX上，使用影像偵測演算法，可以同時偵測煙霧和火的位置，並將結果透過Kafka協定進行遠端的資料傳輸到伺服器上，最終將成果於伺服器上進行即時顯示，實現遠端遙測，提供後續的火災控制或其他應用。開發實我們比較了許多種演算法，在比較後，我們使用YOLO作為我們的影像偵測演算法，並且部屬在NVIDIA Jetson Xavier NX以串流方式進行偵測結果，僅對捕捉場景進行實時偵測。該系統可以通過網絡通知需要通知的人員或團隊。

With the continuous advancement of chip technology in recent years and future trends, the applications of edge computing are becoming increasingly diverse, enabled by edge computing hardware and software support developed by major companies like NVIDIA. One common application is image recognition, which is also seeing growing utilization in edge computing. This study is based on deep learning techniques, comparing various detection algorithms and implementing the most suitable algorithm for fireworks detection. Factors considered include streaming, speed, accuracy, portability, as well as efficiency and power consumption on edge devices. In this work, we selected the NVIDIA Jetson Xavier NX development board as our edge recognition device, where real-time video streaming or camera input is connected to the NVIDIA Jetson Xavier NX. By utilizing image detection algorithms, the system can simultaneously detect the positions of smoke and fire. The results are transmitted remotely to a server using the Kafka protocol and displayed in real-time, enabling remote telemetry and facilitating subsequent fire control or other applications. During the development process, we compared multiple algorithms and ultimately chose YOLO as our image detection algorithm, deployed on the NVIDIA Jetson Xavier NX for real-time detection through streaming, specifically targeting scene capture. The system can notify the relevant personnel or teams through the network.

摘要i
Abstract ii
Table of Contents iii
List of Tables v
List of Figures vi
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Thesis Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2 Background Review and Related Works 4
2.1 Machine learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1.1 Machine learning . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1.2 Reinforcement Learning . . . . . . . . . . . . . . . . . . . . 6
2.2 Neural Network–MLP . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 Deep Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.3.1 Convolutional Neural Networks，CNN . . . . . . . . . . . . 7
2.3.2 Tools–Tensorflow, Keras, PyTorch, and TorchVision . . . . 7
2.3.3 Object Detection and Segmentation Using Deep Learning . 9
2.3.4 OpenCV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.3.5 You Only Look Once YOLO . . . . . . . . . . . . . . . . . . 11
2.3.6 Region-based Convolutional Neural Networks . . . . . . . . 11
2.3.7 Faster R-CNN . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3.8 Nvidia DeepStream . . . . . . . . . . . . . . . . . . . . . . . 13
2.4 Embedded Module . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.5 IoT application and Data processing . . . . . . . . . . . . . . . . . 14
2.5.1 Video streaming . . . . . . . . . . . . . . . . . . . . . . . . 14
2.5.2 Kafka application . . . . . . . . . . . . . . . . . . . . . . . . 14
2.6 Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3 3. Fire and Smoke Detection System Using YOLOv5 19
iii
TABLE OF CONTENTS iv
3.1 System Framework, Setup and Specification . . . . . . . . . . . . . 19
3.1.1 Function and Hardware Specifications . . . . . . . . . . . . 20
3.1.2 Flowchart . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.2 Client–Collecting object using camera, streaming or videos . . . . 28
3.3 Client–Fire and Smoke Detection Using YOLOv5 at Jetson NX . . 28
3.4 Server - Collect Result Using Kafka by FOG Computing . . . . . . 29
3.5 Client - Result Display and Warning at Remote User Site by Website
Base . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.6 Data Convert . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4 Data Collection, Processing and Experimental Results 32
4.1 Research Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.2 Data Collections . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.2.1 Model Training . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.2.2 Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.3 Benchmark . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.4 Comparison between Faster R-CNN, YOLOv4, YOLOv5, YOLOv7
and YOLOv8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.5 Kafka . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.6 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . 71
5 Conclusions and Future Work 73
5.1 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . 73
5.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
References 75

[1] https://github.com/ultralytics/yolov5.
[2] https://github.com/wongkinyiu/yolov7.
[3] https://github.com/ultralytics/ultralytics.
[4] Min Lu and Hemant Ishwaran. A machine learning alternative to p-values.
01 2017.
[5] Marius-Constantin Popescu, Valentina Balas, Liliana Perescu-Popescu, and
Nikos Mastorakis. Multilayer perceptron and neural networks. WSEAS Transactions
on Circuits and Systems, 8, 07 2009.
[6] Keiron O’Shea and Ryan Nash. An introduction to convolutional neural
networks, 2015.
[7] Sebastien Frizzi, Rabeb Kaabi, Moez Bouchouicha, Jean-Marc Ginoux, Eric
Moreau, and Farhat Fnaiech. Convolutional neural network for video fire and
smoke detection. In IECON 2016 - 42nd Annual Conference of the IEEE
Industrial Electronics Society, pages 877–882, 2016.
[8] https://www.tensorflow.org/overview.
[9] https://keras.io/.
[10] https://github.com/pytorch/pytorch.
[11] G. Bradski. The opencv library. Dr. Dobb’s Journal of Software Tools, 2000.
75
References 76
[12] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only
look once: Unified, real-time object detection, 2016.
[13] Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao. Yolov4:
Optimal speed and accuracy of object detection, 2020.
[14] Viswanatha V, Chandana R K, and Ramachandra A. C. Real time object
detection system with yolo and cnn models: A review, 2022.
[15] Chien-Yao Wang, Alexey Bochkovskiy, and Hong-Yuan Mark Liao. Yolov7:
Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors,
2022.
[16] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Regionbased
convolutional networks for accurate object detection and segmentation.
IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(1):142–
158, 2016.
[17] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards
real-time object detection with region proposal networks, 2016.
[18] Ross Girshick. Fast r-cnn. In Proceedings of the IEEE International Conference
on Computer Vision (ICCV), December 2015.
[19] DeepStream SDK — developer.nvidia.com. https://developer.nvidia.com/
deepstream-sdk. [Accessed 23-Jul-2023].
[20] Yoon-Ki Kim and Chang-Sung Jeong. Large scale image processing in realtime
environments with kafka. pages 207–215, 01 2017.
[21] Pedro Venâncio, Adriano Lisboa, and Adriano Barbosa. An automatic fire
detection system based on deep convolutional neural networks for low-power,
resource-constrained devices. Neural Computing and Applications, 09 2022.
[22] DongHyun Kim and WonSun Ruy. Cnn-based fire detection method on autonomous
ships using composite channels composed of rgb and ir data. International
Journal of Naval Architecture and Ocean Engineering, 14:100489,
2022.
References 77
[23] Wentao Mao, Wenpeng Wang, Zhi Dou, and Yuan Li. Fire recognition based
on multi-channel convolutional neural network. Fire Technology, 54, 01 2018.
[24] Rabeb Kaabi, Moez Bouchouicha, Aymen Mouelhi, Mounir Sayadi, and Eric
Moreau. An efficient smoke detection algorithm based on deep belief network
classifier using energy and intensity features. Electronics, 9(9), 2020.
[25] Soon-Young Kim and Azamjon Muminov. Forest fire smoke detection based
on deep learning approaches and unmanned aerial vehicle images. Sensors,
23(12), 2023.
[26] Mukhriddin Mukhiddinov, Akmalbek Bobomirzaevich Abdusalomov, and
Jinsoo Cho. A wildfire smoke detection system using unmanned aerial vehicle
images based on the optimized yolov5. Sensors, 22(23), 2022.
[27] Arnisha Khondaker, Arman Khandaker, and Jia Uddin. Computer visionbased
early fire detection using enhanced chromatic segmentation and optical
flow analysis technique. The International Arab Journal of Information Technology,
17:947–953, 11 2020.
[28] A. NAMOZOV and Y. CHO. An efficient deep learning algorithm for fire
and smoke detection with limited data. Advances in Electrical and Computer
Engineering, 18:121–128, 11 2018.