臺灣博碩士論文加值系統

車牌，做為每輛車獨一無二的標籤，在門禁管制、車輛查找、以及違規取締等應用都是直接與關鍵的依據。相較於傳統影像處理的方法，以深度學習技術為基礎的車牌偵測以及辨識技術，具壓倒性的效能優勢。深度學習技術的發展，不管是影像辨識、物件偵測、或語音識別等，同時仰賴新的神經網路架構與以及完備之代表性資料庫當作刺激。然而，目前文獻上，最常被引用的公開車牌資料庫—AOLP，其目前在各論文的精確率(Precision)大部分已達到95\%以上，實質上已不具挑戰性。建立一個更全面性、更挑戰性的資料庫，從學術研究或者是實際應用來看，對於車牌辨識相關研究者都有實際需求的迫切性。據此，本論文提出一套基於行車紀錄器的錄像所建置的新車牌資料庫，其具備更大的場景複雜度以及變化性。資料庫的取樣考量了包含：日間、夜間的時間變化；逆光、雨天等環境變化。相較於先前 AOLP資料庫的建置過程中有人為參與，預審排除極端的圖像，本新建置資料庫Taiwan License Plates(TLP)，除了避免同一序列視頻中前後幀類似影像的重覆性之外，我們儘量保持自行車記錄器的錄影的所有影像；如此，大幅加大本資料庫的挑戰性。同時，為了方便引用者測試時的定性分析，本資料庫針對所有標註的每幀影像中的每面車牌，提供包括：車牌大小、亮度、傾斜度以不同可辨識度之相關屬性數據的報表輸出，並依此標準將對資料進行如：日間、夜間、輬小、較暗、與較傾斜等特徵屬性的分群。最後，配合針對車牌的偵測與辨識的使用情境，本論文也補足相關績效準則的定義。在此基礎上，針對此新 TLP 資料庫，我們也提出訓練與測試的協定，並利用 YOLO v3 的架構，按此協定，提供相應偵測與辨識的基準績效，當作後續引用學者評比的參考。

License plates, as a unique ID for vehicles, play key roles in numerous applications in, e.g., access control, vehicle searching, and legal enforcement, which prompts increasing need for automatic license plate recognition technique. In contrast to traditional image processing based solutions, deep learning based solution has shown overwhelming improvement in addressing license plate recognition problems in just the past few years. The advancement of deep learning solutions come not only with better network architecture and GPU of high performance but also with a most representative and comprehensive datasets geared for the intended application. In the context of license plate recognition, the most cited dataset is the 2014 released AOLP which was been used for performance comparison in license related studied. However, key performance indexes like precision and F1-score have reached 99\%, rendering the AOLP no more challenging. As a result, a more challenging dataset is urgently in need. Accordingly, this work presents a new dataset for license plate, Taiwan License Plate (TLP) dataset, built on videos taken from dashcam (dashboard camera), where the recording conditions are comprehensive in that videos are taken not only in daylight and night, but also in adverse conditions like raining and backlighting. Different from human involvement in images selection, especially preclusion of too challenging images as in AOLP, the new TLP keeps mostly whatever was recorded despite of recording condition, with the only exception that frame rate is down-sampled to avoid too much similarity among images. In addition to license plate cropping and content tagging, TLP dataset also provides attribute quantifications on: size, brightness, tilt, and discernibility of each annotated plates. This way, special tests aiming for particular conditions, like, smaller, darker, or tilter plates can be easily grouped for training/testing as well as qualification analysis afterwards. Finally, performance criteria and training/testing protocols geared for detection and recognition of license plates are also defined. For baseline assessment, we have also conducted both detection and recognition by YOLO v3 on this newly constructed TLP dataset.

中文摘要. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I
英文摘要. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . III
誌謝. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . V
目錄. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VII
圖目錄. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . X
表目錄. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . XII
第1 章: 簡介. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 車牌資料庫的研究: 偵測與辨識. . . . . . . . . . . . . 1
1.2 舊的AOLP 資料庫之不具挑戰性. . . . . . . . . . . . . 1
1.3 理想車牌資料庫應具備的特色. . . . . . . . . . . . . . 4
1.4 本文研究目標與貢獻. . . . . . . . . . . . . . . . . . . 7
1.5 本文架構. . . . . . . . . . . . . . . . . . . . . . . . . . 8
第2 章: 更具辨識挑戰性的新車牌資料庫. . . . . . . . . . . . . 9
2.1 AOLP 數據集與相關的研究. . . . . . . . . . . . . . . . 9
2.1.1 AOLP baseline 作法. . . . . . . . . . . . . . . . 9
2.1.2 AOLP 上的最佳偵測方法:ALMD-YOLO . . . . 11
2.1.3 車牌辨識的其他作法:Reading Car License Plates
Using Deep Convolutional Neural Networks and
LSTMs . . . . . . . . . . . . . . . . . . . . . . . 12
2.1.4 其他國家的車牌資料庫: 巴西車牌資料庫UFPR 14
2.2 TLP 資料庫建置. . . . . . . . . . . . . . . . . . . . . . 16
2.2.1 資料集建置流程. . . . . . . . . . . . . . . . . . 16
2.2.2 反應難易度之屬性：尺寸、明亮、傾斜度、可
見度、拍攝時段. . . . . . . . . . . . . . . . . . 21
2.2.3 訓練與測試之績效準則. . . . . . . . . . . . . . 25
第3 章: TLP 資料庫與測試新基準. . . . . . . . . . . . . . . . . 30
3.1 訓練集與測試集的選取. . . . . . . . . . . . . . . . . . 30
3.2 車牌偵測. . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.2.1 車牌偵測正確率的定義. . . . . . . . . . . . . . 34
3.2.2 YOLO V3 改進概述. . . . . . . . . . . . . . . . 36
3.3 車牌辨識. . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.3.1 車牌辨識正確率的定義. . . . . . . . . . . . . . 38
3.3.2 Cycle GAN 產生車牌. . . . . . . . . . . . . . . 39
3.3.3 Recognition by detection . . . . . . . . . . . . . . 42
第4 章: 實驗. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.1 使用YOLO v3 之車牌偵測. . . . . . . . . . . . . . . . 44
4.1.1 交叉驗證. . . . . . . . . . . . . . . . . . . . . . 44
4.1.2 錯誤分析. . . . . . . . . . . . . . . . . . . . . . 46
4.1.3 與其他資料庫的效能比較. . . . . . . . . . . . 48
4.2 使用GAN 方法的車牌辨識. . . . . . . . . . . . . . . . 49
4.2.1 各子集上的效能. . . . . . . . . . . . . . . . . . 49
4.2.2 錯誤分析. . . . . . . . . . . . . . . . . . . . . . 51
4.2.3 與其他資料庫的效能比較. . . . . . . . . . . . 52
第5 章: 結論. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
5.1 成果. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
5.2 後續研究工作. . . . . . . . . . . . . . . . . . . . . . . 54
參考文獻. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

[1] G.-S. Hsu, J.-C. Chen, and Y.-Z. Chung, “Application-oriented license plate recognition,” IEEE transactions on vehicular technology, vol. 62, no. 2, pp. 552–561, 2013.
[2] L. Xie, T. Ahmad, L. Jin, Y. Liu, and S. Zhang, “A new cnn-based method for multi-directional car license plate detection,” IEEE Transactions on Intelligent Transportation Systems, 2018.
[3] H. Li and C. Shen, “Reading car license plates using deep convolutional neural networks and lstms,” arXiv preprint arXiv:1601.05610, 2016.
[4] R. Polishetty, M. Roopaei, and P. Rad, “A next-generation secure cloud-based deep learning license plate recognition for smart cities,” in Machine Learning and Applications (ICMLA), 2016 15th IEEE International Conference on, pp. 286–293, IEEE, 2016.
[5] Z. Selmi, M. B. Halima, and A. M. Alimi, “Deep learning system for automatic license plate detection and recognition,” in Document Analysis and Recognition (ICDAR), 2017 14th IAPR International Conference on, vol. 1, pp. 1132–1138, IEEE, 2017.
[6] J. Redmon and A. Farhadi, “Yolov3: An incremental improvement,” arXiv preprint arXiv:1804.02767, 2018.
[7] C. Zhang and Y. Tian, “Rgb-d camera-based daily living activity recognition,” Journal of Computer Vision and Image Processing, vol. 2, no. 4, p. 12, 2012.
[8] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycleconsistent adversarial networks,” arXiv preprint, 2017.
[9] M. R. Gupta, Y. Chen, et al., “Theory and use of the em algorithm,” Foundations and Trends® in Signal Processing, vol. 4, no. 3, pp. 223–296, 2011.
[10] J. Matas, O. Chum, M. Urban, and T. Pajdla, “Robust wide-baseline stereo from maximally stable extremal regions,” Image and vision computing, vol. 22, no. 10, pp. 761–767, 2004.
[11] L. Liu, H. Zhang, A. Feng, X. Wan, and J. Guo, “Simplified local binary pattern descriptor for character recognition of vehicle license plate,” in Computer Graphics, Imaging and Visualization (CGIV), 2010 Seventh International Conference on, pp. 157–161, IEEE, 2010.
[12] H. Zhao and P. C. Yuen, “Incremental linear discriminant analysis for face recognition,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 38, no. 1, pp. 210–221, 2008.
[13] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 779–788, 2016.
[14] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997.
[15] A. Graves, M. Liwicki, S. Fernández, R. Bertolami, H. Bunke, and J. Schmidhuber, “A novel connectionist system for unconstrained handwriting recognition,” IEEE transactions on pattern analysis and machine intelligence, vol. 31, no. 5, pp. 855–868, 2009.
[16] R. Laroca, E. Severo, L. A. Zanlorensi, L. S. Oliveira, G. R. Gonçalves, W. R. Schwartz, and D. Menotti, “A robust real-time automatic license plate recognition based on the yolo detector,” arXiv preprint arXiv:1802.09567, 2018.
[17] J. Redmon and A. Farhadi, “Yolo9000: better, faster, stronger,” arXiv preprint, 2017.
[18] M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman, “The pascal visual object classes (voc) challenge,” International journal of computer vision, vol. 88, no. 2, pp. 303–338, 2010.
[19] S. Geisser, Predictive inference. Routledge, 2017.
[20] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: a simple way to prevent neural networks from overfitting,” The Journal of Machine Learning Research, vol. 15, no. 1, pp. 1929–1958, 2014.
[21] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” arXiv preprint arXiv:1502.03167, 2015.
[22] S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” in Advances in neural information processing systems, pp. 91–99, 2015.
[23] C. Szegedy, S. Ioffe, V. Vanhoucke, and A. A. Alemi, “Inception-v4, inception-resnet and the impact of residual connections on learning.,” in AAAI, vol. 4, p.12, 2017.
[24] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in neural information processing systems, pp. 2672–2680, 2014.