臺灣博碩士論文加值系統

空拍設備之技術發展日益成熟，從高空俯視下，能夠透過單一空拍影像，一次觀測大環境之車流狀況，降低硬體設備所需成本，但是透過單一尺度之車輛樣本所訓練的檢測模型，所能檢測的影像尺度相當受限，而且在不同高度的影像觀測過程中，其車輛大小占影像之比例皆有所不同，對於越高的拍攝高度，車輛所能提取的資訊量也越少，導致車輛檢測難度提升。此外將車輛檢測模型延伸至車輛追蹤的過程中，由於車體面積甚小以及車速變異，容易導致相同車體的資料關聯匹配失敗，致使車輛追蹤不連續，影響追蹤效果。針對上述問題，本論文提出一套對於不同尺度下的日間空拍畫面，皆能保有高準確率的車流觀測系統。
本論文提出以下三點貢獻，第一點，本論文使用更深層的深度學習網路，並搭配測試資料擴增，使單一尺度的檢測準確率更加提升，同時藉由高度資訊的導入，計算地面取樣距離後，將單一尺度的檢測模型推廣至不同尺度下的影像皆可檢測；第二點，本論文提出卡爾曼濾波器結合光流法之追蹤技術，並且使用自適性速度閥值，解決車體在追蹤過程中遭受短暫遮蔽(約15公尺)而追蹤中斷，或連續追蹤之車體因匹配失敗而追蹤不連貫等問題；第三點，本論文整合車輛檢測模型與相關追蹤技術，建構出在多尺度下，定點車流觀測系統之雛形。

From taking pictures using drones up in the sky, it is capable of detecting the traffic situation on the streets, lowering the costs on hardwares. But through the single scale object detection model, the image scale detected was very limited. The higher the drone is, less information could be extracted from the data, causing the rise of difficulty on vehicle detection. Also, while the progress extends from vehicle detection to vehicle tracking, it might cause matching failure because of the small detecting scale on the vehicle and the moving speed. Through those problems pointed out, this paper comes up with a vehicle tracking system that has high accuracy during different scale of aero-drone shots in the day times.
This paper offers three contributions. First, deep learning neural network was applied in this paper with the test-data argument, making the accuracy on single scale detection rising higher. Also, through importing the altitude information, the ground sample distance was calculated. So, when the single-scale object detection model was changed into multi-scale, the image can still be tested. Second, this paper provides the tracking technology by combining Kalman filter with optical flow. Using the adaptive velocity threshold, the short occlusion during vehicle tracking process or continuous matching failure will be solved. Third, the paper combined vehicle detection model with related tracking technology, a prototype of multi-scale traffic monitoring system was built.

摘要.......................................i
ABSTRACT..................................ii
致謝......................................iii
目錄.......................................iv
圖目錄.....................................vi
表目錄.....................................x
第一章 緒論...............................1
1.1 研究動機...........................1
1.2 多尺度車況觀測系統之介紹.............2
1.3 相關研究之探討......................2
1.4 論文貢獻與架構......................4
第二章 相關技術與原理......................5
2.1 車輛檢測模型........................5
2.1.1 Faster R-CNN.......................5
2.1.2 ResNet 50.........................10
2.2 目標物追蹤.........................11
2.2.1 卡爾曼濾波器.......................12
2.2.2 光流法追蹤.........................15
2.3 K-means聚類演算法..................19
2.4 非極大值抑制.......................21
2.5 匈牙利演算法.......................23
第三章 系統流程與架構......................25
3.1 整體系統架構........................25
3.2 車輛偵測...........................27
3.2.1 訓練車輛檢測模型....................28
3.2.2 測試資料擴增.......................31
3.3 多尺度轉換.........................38
3.3.1 地面取樣距離.......................38
3.3.2 自動切割與縮放機制..................40
3.4 車輛追蹤...........................44
3.4.1 卡爾曼與光流法之整合................44
3.4.2 資料關聯...........................50
3.5 自適性速度閥值.....................53
第四章 實驗結果與分析.....................56
4.1 實驗機制..........................56
4.1.1 實驗設備與實驗場景.................56
4.1.2 效能評估標準......................62
4.2 實驗結果分析與討論.................65
4.2.1 檢測階段之實驗結果分析與討論........66
4.2.2 追蹤階段之實驗結果分析與討論........88
4.2.3 實驗結果比較......................103
第五章 結論與未來工作....................111
參考文獻.................................112

[1] W. Shao, W. Yang, G. Liu, and J. Liu, “Car Detection from High-resolution Aerial Imagery Using Multiple Features,” 2012 IEEE International Geoscience and Remote Sensing Symposium, Munich, pp. 4379-4382, 2012.
[2] S. Kluckner, G. Pacher, H. Grabner, H. Bischof, and J. Bauer, “A 3D Teacher for Car Detection in Aerial Images,” in 2007 IEEE 11th International Conference on Computer Vision, Rio de Janeiro, pp. 1-8, 2007.
[3] N. Dalal and B. Triggs, “Histograms of Oriented Gradients for Human Detection,” in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA, vol. 1, pp. 886-893, 2005.
[4] T. Ojala, M. Pietikainen, and D. Harwood, “Performance Evaluation of Texture Measures with Classification Based on Kullback Discrimination of Distributions,” in Proceedings of 12th International Conference on Pattern Recognition, Jerusalem, vol. 1, pp. 582-585, 1994.
[5] C. P. Papageorgiou, M. Oren, and T. Poggio, “A General Framework for Object Detection,” in Sixth International Conference on Computer Vision, Bombay, India, pp. 555-562, 1998.
[6] J. Leitloff, D. Rosenbaum, F. Kurz, O. Meynberg, and P. Reinartz, “An Operational System for Estimating Road Traffic Information from Aerial Images,” in Remote Sens., vol. 6, no. 11, pp. 11315–11341, 2014.
[7] S. Tuermer, F. Kurz, P. Reinartz, and U. Stilla, “Airborne Vehicle Detection in Dense Urban Areas Using HoG Features and Disparity Maps,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 6, no. 6, pp. 2327-2337, 2013.
[8] P. Viola and M. Jones, “Rapid Object Detection Using a Boosted Cascade of Simple Features,” in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. I-511-I-518, 2001.
[9] K. Liu and G. Mattyus, “Fast Multiclass Vehicle Detection on Aerial Images,” IEEE Geoscience and Remote Sensing Letters, vol. 12, no. 9, pp. 1938-1942, 2015.
[10] P. Dollar, Z. Tu, P. Perona, and S. Belongie, “Integral Channel Features,” Proceedings of the British Machine Conference, pp. 91.1-91.11, 2009.
[11] X. Chen, S. Xiang, C. L. Liu, and C. H. Pan, “Vehicle Detection in Satellite Images by Hybrid Deep Convolutional Neural Networks,” IEEE Geoscience and Remote Sensing Letters, vol. 11, no. 10, pp. 1797-1801, 2014.
[12] R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Region-Based Convolutional Networks for Accurate Object Detection and Segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 1, pp. 142-158, 2016.
[13] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” in Proceedings of the 25th International Conference on Neural Information Processing Systems, USA, vol. 1, pp. 1097-1105, 2012.
[14] M. D. Zeiler and R. Fergus, “Visualizing and Understanding Convolutional Networks,” in Proceedings Eur. Conference of Computer Vision, pp. 818-833, 2014.
[15] K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, pp. 770-778, 2016.
[16] Z. Zheng, G. Zhou, Y. Wang, and Y. Liu, “A Novel Vehicle Detection Method with High Resolution Highway Aerial Image,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 6, no. 6, pp. 2338-2343, 2013.
[17] C. Dicle, O. I. Camps, and M. Sznaier, “The Way They Move: Tracking Multiple Targets with Similar Appearance,” in 2013 IEEE International Conference on Computer Vision, Sydney, NSW, pp. 2304-2311, 2013.
[18] J. H. Yoon, M. H. Yang, J. Lim, and K. J. Yoon, “Bayesian Multi-Object Tracking Using Motion Context from Multiple Objects,” in 2015 IEEE Winter Conference on Applications of Computer Vision, Waikoloa, HI, pp. 33-40, 2015.
[19] A. Bewley, L. Ott, F. Ramos, and B. Upcroft, “Alextrac: Affinity Learning by Exploring Temporal Reinforcement within Association Chains,” in 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, pp. 2212-2218, 2016.
[20] C. Kim, F. Li, A. Ciptadi, and J. M. Rehg, “Multiple Hypothesis Tracking Revisited,” in 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, pp. 4696-4704, 2015.
[21] S. H. Rezatofighi, A. Milan, Z. Zhang, Q. Shi, A. Dick, and I. Reid, “Joint Probabilistic Data Association Revisited,” in 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, pp. 3047-3055, 2015.
[22] A. Geiger, M. Lauer, C. Wojek, C. Stiller, and R. Urtasun, “3D Traffic Scene Understanding from Movable Platforms,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 5, pp. 1012-1025, 2014.
[23] S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137-1149, 2017.
[24] K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” in International Conference on Live Coding (ICLR) on Computer Vision and Pattern Recognition, 2015.
[25] S. Ioffe and C. Szegedy, “Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift,” in International Conference on Machine Learning (ICML), 2015.
[26] K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, pp. 770-778, 2016.
[27] R. E. Kalman, “A New Approach to Linear Filtering and Prediction Problems,” Journal of Basic Engineering, pp. 35-45, 1960.
[28] B. D. Lucas and T. Kanade, “An Iterative Image Registration Technique with an Application to Stereo Vision,” in Proceedings of the 7th International Joint Conference on Artificial Intelligence, Canada, vol. 2, pp. 674-679, 1981.
[29] Jianbo Shi and C. Tomasi, “Good Features to Track,” in 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, pp. 593-600, 1994.
[30] J. MacQueen, “Some Methods for Classification and Analysis of Multivariate Observations,” in Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, Calif., vol. 1, pp. 281-297, 1967.
[31] A. Neubeck and L. Van Gool, “Efficient Non-Maximum Suppression,” in 18th International Conference on Pattern Recognition (ICPR'06), Hong Kong, pp. 850-855, 2006.
[32] T. Nathan Mundhenk, G. Konjevod, W. A. Sakla, and K. Boakye, “A Large Contextual Dataset for Classification, Detection and Counting of Cars with Deep Learning,” in European Conference on Computer Vision (ECCV) 2016 Pre-press revision, 2016.
[33] A. Bewley, Z. Ge, L. Ott, F. Ramos, and B. Upcroft, “Simple Online and Realtime Tracking,” in 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, pp. 3464-3468, 2016.
[34] Z. Deng, H. Sun, S. Zhou, J. Zhao, and H. Zou, “Toward Fast and Accurate Vehicle Detection in Aerial Images Using Coupled Region-Based Convolutional Neural Networks,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 10, no. 8, pp. 3652-3664, 2017.