臺灣博碩士論文加值系統

近年來，深度學習(deep learning)廣泛被應用在自動車牌辨識(automatic license plate recognition :ALPR )系統，然而傳統 ALPR 大多截取正面的車牌，才能達到高準確的辨識率，這是因為傾斜車牌將使得傳統 ALPR 無法截取正確的車牌區域，導致後續車牌字元辨識發生錯誤或字元遺漏，大幅降低車牌辨識的準確率。因此，Silva 等學者最近提出一扭曲平面物件檢測(warped planar object detection: WPOD)網路來克服傾斜車牌的問題[12]，為了實現高準確率的 ALPR 系統，他們提出的 ALPR 架構主要分成三個部分，首先透過 YOLO(you only look once)來定位車輛[6]，接著利用 WPOD 網路來進行傾斜車牌定位和轉正，最後將轉正車牌輸入字元辨識網路來辨識出車牌上的字元。
雖然 WPOD 網路可以完成車牌定位和轉正，來大幅提高傾斜車牌辨識的準確率，但是 Silva 等學者在設計 WPOD 網路的損失函數(loss function)時，因為置信度(confidence)的計算相當複雜，所以在損失函數中並沒有採用置信度參數，這也導致網路在預測時，可能選擇到非最佳的車牌框，使後續的字元辨識網路容易發生辨識錯誤或出現字元遺漏的情形。為了提高 WPOD 網路車牌辨識的準確率，本論文提出一改良型 WPOD 網路，我們首先推導出簡易 IOU(intersection over union)的計算來快速獲得置信度，並將置信度加入損失函數，來完成更準確的車牌辨識網路。
本論文利用傾斜車牌預測框的四頂點座標來產生矩形框，再透過簡易 IOU運算，可以快速的獲得置信度參數，完成更精準的損失函數計算。因此，我們所提改良型 WPOD 網路，可以很容易將傾斜的 IOU 帶入損失函數中，完成更高的車牌辨識率。將所提方法與 WPOD 在相同公開數據集進行比較，從測試結果可以發現，我們在 IOU 的評分比高於 WPOD 平均約 3%，而在後續字元辨識的準確率，論文所提方法除了可以達到 95%的準確率外，也比 WPOD 系統的準確率平均高約 1%。

In recent years, automatic license plate recognition (ALPR) system is applied in some traffic-related applications based on deep learning. However, most ALPR systems capture a mostly frontal view of the vehicle and license plate (LP) to obtain high LP recognition rates. This is because the traditional ALPR cannot capture the correct area of oblique LP which results in an error in character recognition or missing characters. As a result, the traditional ALPR will largely reduce the accuracy of recognition for oblique LP. Recently, Silva et al. [12] proposed a warped planar object detection (WPOD) based on convolutional neural network (CNN) to overcome the oblique views of LP. In order to achieve an ALPR system of high accuracy, they divided ALPR into three stages. The first stage is to locate vehicles through YOLOv2 [9]. And then, the second stage locates the oblique LPs and allows a rectification of the LPs area to a rectangle which resembles a frontal view through the WPOD network. Finally, the rectified LPs are fed to an optical character recognition (OCR) in the third stage.
Although the WPOD network can achieve the location and rectification of LPs, the loss function of WPOD render the confidence parameter due to high computational complexity. This also leads to WPOD network cannot locate the optimal LP bounding box. In order to further improve the accuracy of ALPR system, we proposed a modified WPOD network using a complete loss function. The proposed method first develop a simple intersection over union (IOU) algorithm to speed up the calculating process of confidence. Therefore, the modified WPOD network can obtain higher LP recognition rate since it considers the confidence parameter in loss function.
In this thesis, the four-vertex coordinates of the label bounding box and prediction bounding box of oblique LP are used to generate two rectangular boxes, and then a simple IOU algorithm is used to fast calculate the approximate value of IOU. As a result, a more exact loss function can be finished. In order to compare the performance of LP recognition rate, we train and test the WPOD and the proposed modified WPOD in databases including OpenALPR EU, BR [19], and AOLP RP [17]. Simulation results show that the proposed ALPR system can obtain higher score ratios than those of Silva’s method. And the proposed system can arrive a high accuracy of LP recognition about 95% on an average. In addition, the proposed system also can achieve higher recognition rate about 1% when compared to the Silva’s ALPR system.

摘要 i
ABSTRACT iii
致謝 v
圖目錄 viii
表目錄 xii
第一章 緒論 1
1.1 研究背景 1
1.2 研究動機 2
1.3 論文架構 5
第二章 深度學習介紹及系統架構 6
2.1 類神經網路 6
2.2 深度學習 8
2.2.1 神經傳導原理 8
2.2.2 損失函數與優化流程 11
2.3 CNN網路架構 13
2.3.1 卷積層 14
2.3.2 池化層 16
2.3.3 全連接層 17
2.4 CNN網路應用 18
2.4.1 定位應用 18
第三章 基於深度學習之車牌辨識系統 24
3.1 WPOD車牌辨識系統架構 24
3.1.1 YOLOv2車輛定位 24
3.1.2 WPOD網路車牌定位 27
3.1.3 YOLOv2 字元辨識 34
3.2 文獻分析與討論 37
第四章 改良型WPOD網路 41
4.1 WPOD損失網路分析 41
4.2 IOU分析 42
4.3 簡易IOU損失函數 46
第五章 實驗結果分析與結論 57
5.1 實驗環境 57
5.2 實驗結果分析與比較 63
5.3 結論和未來工作 68
參考文獻 69

[1] Kaggle is an online community of data scientists and machine learners, owned by Google LLC, https://www.kaggle.com/
[2] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and F. F. Lin, “ImageNet Large Scale Visual Recognition Challenge,” International Journal of Computer Vision (IJCV), pp. 211-252, 2015
[3] A. Krizhevsky, I. Sutskever, and G. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems, pp. 1097-1105, 2012
[4] R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 580-587, 2014
[5] S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” in Advances in neural information processing systems, pp. 91-99, 2015
[6] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” Proceedings of the IEEE conference on computer vision and pattern recognition., pp. 779-788, 2016
[7] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, and S. Reed, “SSD: Single shot multibox detector,” in European conference on computer vision, pp. 21-37, 2016
[8] 交通部統計查詢網 https://stat.motc.gov.tw/mocdb/stmain.jsp?sys=100
[9] J. Redmon and A. Farhadi, “YOLO9000: Better, faster, stronger,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7263-7271, 2017
[10] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014
[11] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1-9, 2015
[12] S. Montazzolli and J. Rosito, “License plate detection and recognition in unconstrained scenarios,” European Conference on Computer Vision, pp. 593-609, 2018
[13] M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman. “The pascal visual object classes (voc) challenge,” International journal of computer vision, pp. 303-338, 2010
[14] T. Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in European conference on computer vision, pp. 740-755, 2014
[15] J. Krause, M. Stark, J. Deng, and L. Fei-Fei. “3D object representations for finegrained categorization,” Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 554-561, 2013
[16] G. Resende, et al, “Benchmark for license plate character segmentation,” Journal of Electronic Imaging, pp. 1-5, 2016
[17] G. S. Hsu, J. C. Chen and Y. Z. Chung, “Application-Oriented License Plate Recognition,” IEEE Transactions on Vehicular Technology, pp. 552-561, 2013
[18] D. P. Kingma and J. Ba, “A method for stochastic optimization,” CoRR abs/1412.6980, 2014
[19] OpenALPR is an automatic number-plate recognition library written in C++. https://www.openalpr.com/, 2014
[20] The CGAL Project. The Computational Geometry Algorithms Library. http://www.cgal.org, 2016
[21] L. Xie, T. Ahmad, L. Jin, Y. Liu and S. Zhang, “A New CNN-Based Method for Multi-Directional Car License Plate Detection,” IEEE Transactions on Intelligent Transportation Systems, pp.507-517, 2018