臺灣博碩士論文加值系統

本研究中，我們提出一基於長短期記憶模型的停車場空位偵測方法，此方法包含狀態卷積網路與行為卷積網路。首先，我們先從影像連續序列中，分出個別的影像作為空間串流；再使用這些影像計算移動光流作為移動資訊的時間串流。在影像空間串流中，我們每次輸入一張影像到判別停車空位狀態的卷積神經網路，我們稱此架構為 “狀態卷積網路”，同時，將提取的高階特徵輸入至長短期記憶模型，加入長短期記憶模型能夠考慮前後影像的歷史狀態資訊，以避免單張影像而造成的狀態誤判，車位狀態分類為 “被占據” 與 “空位”。在時間串流中，我們先堆疊多張光流影像，再將堆疊後影像輸入至三維卷積神經網路，判斷駕駛者目前的停車狀態，架構類似於狀態卷積網路，我們稱此架構為 “行為卷積網路” ，行為卷積網路使用短時間的光流影像資訊判斷駕駛者的停車狀態，為了增加狀態判斷的準確性，此網路也結合長短期記憶模型以考慮前後光流影像的歷史資訊，增加長時間的移動資訊，進行停車狀態的辨識，停車狀態為 “停車行為” 、 “取車行為” 以及 “靜止行為”。最後，我們設計串流融合的架構，結合影像語意的判斷資訊與停車狀態的辨識結果，以整合影像分析與動作辨識的資訊，進行車位狀態的判斷。

In this research, we propose the LSTM-based parking lot detection method architecture. This framework divides to two parts, one is “Status ConvNet”, and another is “Action ConvNet”. Frist, we will separate individual frame from sequence of image to become the spatial stream. And then, we will calculate optical flow to be moving information to become the temporal stream. For spatial stream, we input an image to Convolutional Neural Network (CNN) to detect the status of parking space, called the network “Status ConvNet”. At the same time, input extracted high-level feature to LSTM that could consider the information of historical status to avoid wrong detection from single frame. The classes of space status are “Occupy” and “Vacant”. For temporal stream, we stack multiple images of optical flow, and input them to 3-dimension CNN to detection parking status of driver. The network is similar as Status ConvNet called “Action ConvNet”. Action ConvNet uses optical flow as short-term information to detect parking status of driver. In order to increase the accuracy, we also introduce LSTM in network to consider historical information of optical flow as long-term moving information. The classes of parking status are “Drop off”, “Pick up”, and “No action”. Finally, we design the two-stream architecture to fuse spatial and temporal information.

誌謝辭 i
摘要 ii
Abstract iii
目錄 iv
圖目錄 vi
表目錄 viii
第1章、 緒論 1
1.1 研究背景與動機 1
1.2 研究目標與困難 3
1.3 論文架構 4
第2章、 文獻探討與技術背景 5
2.1 機器學習方法 5
2.1.1 基於影像外貌之停車場空位偵測技術 5
2.1.1.1 特徵提取 6
2.1.1.2 特徵降維與特徵融合 7
2.1.1.3 車位狀態辨識 8
2.1.2 應用時間軸資訊之停車場空位偵測 10
2.2 深度學習方法 11
2.2 應用深度學習於資訊融合之技術 12
第3章、 基於長短期記憶模型之停車場空位偵測方法 15
3.1 系統架構 15
3.2 空間串流 15
3.2.1 空間轉換網路(Spatial Transformer Network) 17
3.2.2 卷積神經網路(Convolutional Nerual Network) 18
3.2.3 長短期記憶模型(Long Short-Term Memory) 20
3.3 時間串流 21
3.4 光流資訊 23
3.5 最佳化 25
第4章、 系統實現 27
4.1 系統平台 27
4.2 網路實現 27
4.2.1 網路設定 27
4.2.2 損失函數曲線 29
4.2.3 準確率曲線 31
第5章、 實驗結果 32
5.1 資料庫 32
5.2 停車場偵測的結果評估 32
第6章、 結論與未來展望 36
6.1 結論 36
6.2 未來展望 36
參考文獻 37
附錄 43

[1]M.Y.I. Idris, Y.Y. Leng, E.M. Tamil, N.M. Noor, and Z. Razak, “Car Park System: A Review of Smart Parking System and its Technology,” Information Technology Journal (ITJ), vol. 8, no. 2, pp. 101-113, 2009.
[2]C. C. Huang and S. J. Wang, “A Hierarchical Bayesian Generation Framework for Vacant Parking Space Detection,” IEEE TCSVT, Dec.2010, pp. 1770-1785.
[3]N. Dalal and B. Triggs, “Histograms of Oriented Gradients for Human Detection,” IEEE Conference on Computer Vision and Pattern Recognition, Jun.2005, pp. 886-893.
[4]C. C. Huang, H. T. Vu and Y. R. Chen, “A Multi-Layer Discriminative Framework for Parking Space Detection,” IEEE International Workshop on Machine Learning for Signal Processing, Sept. 17-20, 2015.
[5]C. C. Huang and H. T. Vu, “Vacant Parking Space Detection based on a Multi-layer Inference Framework,” IEEE TCSVT, May.2016.
[6]C. C. Huang, Y. S. Tai and S. J. Wang, “Vacant Parking Space Detection Based On Plane-Based Bayesian Hierarchical Framework,” IEEE Transactions on Circuits and Systems for Video Technology, Sept.2013.
[7]K. Yamada and M. Mizuno, “A Vehicle Parking Detection Method Using Image Segmentation,” Electronics and Communications, 2001.
[8]C. H. Lee, M. G. Wen, C. C. Han and D. C. Kuo, “An Automatic Monitoring Approach forUnsupervised Parking Lots in Outdoor,” IEEE International Conference on Security Technology, 2005.
[9]R. J. López-Sastre, P. G. Jimenez, F. J. Acevedo and S. M. Bascon, “Computer Algebra AlgorithmsApplied to Computer Vision in a Parking Management System,” IEEE International Symposium on Industrial Electronics, 2007.
[10]W. Lixia and J. Dalin, “A method of Parking space detection based on image segmentation and LBP,”International Conference on Multimedia Information Networking and Security, Nov.2012, pp. 229-232.
[11]P. Almeida, L. S. Oliveira, E. Silva Jr., A. Britto Jr. and A. Koerich, “Parking Space Detection usingTextural Descriptors,” IEEE International Conference on Systems, Man, and Cybetnetics, 2013.
[12]D. Delibaltov, W. Wu, R. P. Loce and E. A. Bernal, “Parking Lot Occupancy Determination fromLamp-post Camera Images,” Proceedings of the 16th International IEEE Annual Conference onIntelligent Transportation Systems, Oct. 6-9, 2013.
[13]P. Viola and M. Jones, “Rapid object detection using a boosted cascade of simple features,” IEEE Comp.Soc. Conf. on Computer Vision and Pattern Recognition (CVPR), 2001, pp. 1-511.
[14]M. Tschentscher and M. Neuhausen, “Video-based parking-space detection,” Proceedings of the Forum Bauinformatik, 2012, pp. 159-166.
[15]N. True, “Vacant parking space detection in static image,” Projects in Vision & Learning, University of California, 2007.
[16]D. B. L. Bong, K. C. Ting and N. Rajaee, “Car-park Occupancy Information System,” Third Real-Time Technology and Applications Symposium, Dec.2006.
[17]D. B. L. Bong, K. C. Ting and K. C. Lai, “Integrated Approach in the Design of Car Park Occupancy Information System (COINS),” International Journal of Computer Science, 2008, pp. 1-8.
[18]K. Blumer, H. Halaseh, M. Ahsan, H. Dong and N. Mavridis, “Cost-Effective Single-Camera Multi-Car Parking Monitoring and Vacancy Detection towards Real-world Parking Statistics and Real-time Reporting,” International Conference on Neural Information Processing, 2012.
[19]H. Bhaskar, N. Werghi and S. AL-Mansoori, “Rectangular Empty Parking Space Detection using SIFT based Classification,” Proceedings of the Sixth International Conference on Computer Vision Theory and Applications, Mar. 5-7, 2011.
[20]Q. Wu, C. C. Huang, S. Y. Wang, W. C. Chiu and T. H. Chen, “Robust Parking Space Detection Considering Inter-Space Correlation,” IEEE International Conference on Multimedia and Expo, 2007, pp. 659-662.
[21]H. Ichihashi, T. Katada, M. Fujiyoshi, A. Notsu and K. Honda, “Improvement in the performance of camera based vehicle detector for parking lot,” Proc. IEEE Int. Conf. Fuzzy Syst, 2010, pp. 1950-1956.
[22]N. Dan, “Parking Management System and Method,” US patent, Pub. No.: 20030144890A1, Jul.2003.
[23]C. C. Huang, S. J. Wang, Y. J. Chang and T. Chen, “A Bayesian Hierarchical Detection Framework for Parking Space Detection,” IEEE International Conference on Acoustics, Speech and Signal Processing, 2008.
[24]M. Tschentscher, C. Koch, M. König, J. Salmen and M. Schlipsing, “Scalable real-time parking lot classification: An evaluation of image features and supervised learning algorithms,” International Joint Conference on Neural Networks, Jul.2015.
[25]L. L. Ng and H. S. Chua, “Vision-Based Activities Recognition by Trajectory Analysis for Parking Lot Surveillance,” IEEE International Conference on Circuits and Systems, Oct.2012.
[26]P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio and P. A. Manzagol, “Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion,” The Journal of Machine Learning Research, vol. 11, 2010, pp. 3371-3408.
[27]Y. LeCun, K. Kavukcuoglu and C. Farabet, “Convolutional networks and applications in vision,” IEEE International Symposium on Circuits and Systems,2010, pp. 253-256.
[28]G. E. Hinton, “Deep belief networks,” Scholarpedia, vol. 4, no. 5,2009, pp. 5947.
[29]R. Salakhutdinov and G. E. Hinton, “Deep boltzmann machines,” International Conference on Artificial Intelligence and Statistics,2009, pp. 448-455.
[30]Y. LeCun, L. Bottou, Y. Bengio and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11,1998, pp. 2278-2324.
[31]S. Valipour, M. Siam, E. Stroulia and M. Jagersand, “Parking Stall Vacancy Indicator System Based on Deep Convolutional Neural Networks,” arXiv:1606.09367 [cs.CV],Jun.2016.
[32]S. Ji, W. Xu, M. Yang and K. Yu, “3D Convolutional Neural Networks For Human Action Recognition,” Proc. Int. Conference on Machine Learning (ICML), 2010.
[33]A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar and F.F. Li, “Large-scale Video Classification with Convolutional Neural Networks,” Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR),2014, pp. 1725-1732.
[34] A. J. Robinson and F. Fallside, “Static and dynamic error propagation networks with application to speech coding,” Proc. Advances in Neural Information Processing Systems (NIPS), 1988, pp. 632-641.
[35]S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Comput., 1997, pp. 1735-1780.
[36]M. Baccouche, F. Mamalet, C. Wolf, C. Garcia and A. Baskurt, “Sequential deep learning for human action recognition,” Human Behavior Understanding, 2011, pp. 29-39.
[37]M. Jaderberg, K. Simonyan, A. Zisserman, and K. Kavukcuoglu, “Spatial Transformer Networks,” in Advances in Neural Information Processing Systems, pp. 2017–2025, 2015.
[38]H.T. Vu and C.C. Hung, “Parking Space Detection upon a Deep CNN and Multi-task Contrastive Network with Spatial Transform,” in IEEE Transactions on Circuits and Systems for Video Technology, 2018.
[39]J. Bromley, I. Guyon, Y. Lecun, E. Sackinger and R. Shah, “Signature Verification Using a "Siamese" Time Delay Neural Network,” in Neural Information Processing Systems (NIPS), 1994.
[40]S. Chopra, R. Hadsell, and Y. LeCun, “Learning a Similarity Metric Discriminatively, with Application to Face Verification,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Diego CA, June, 2005.
[41]J. Hu, J. Lu and Y.-P. Tan, “Discriminative Deep Metric Learning for Face Verification in the Wild,” in IEEE Conference on Computer Vision and Pattern Recognition, pp. 1875-1882, 2014.
[42]E. Simoserra, E. Trulls, L. Ferraz, I. Kokkinos and F. Moreno-Noguer, “Fracking Deep Convolutional Image Descriptors,” in arXiv:1412.6537v2 [cs.CV], Feb. 2015.
[43]R. Hadsell, S. Chopra and Y. LeCun, "Dimensionality Reduction by Learning an Invariant Mapping," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[44]B. D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” in International Joint Conference on Artificial Intelligence (IJCAI), Vancouver, BC, Canada, 24-28 Aug., 1981.
[45]G. Farnebäck, “Two-Frame Motion Estimation Based on Polynomial Expansion,” in Scandinavian Conference on Image Analysis (SCIA), Halmstad, Sweden, 29 June-2 July, 2003.
[46]S. Baker, D. Scharstein, J. P. Lewis, S. Roth, M. J. Black, and R. Szeliski, “A Database and Evaluation Methodology for Optical Flow,” in International Journal of Computer Vision (IJCV), vol. 92, no. 1, pp. 1-31, 2011.
[47]H.T. Vu, and C.C. Huang, "A Multi-Task Convolutional Neural Network With Spatial Transform For Parking Space Detection", in 2017 IEEE International Conference on Image Processing (ICIP), 2017.