臺灣博碩士論文加值系統

端對端方向盤自動控制方法中，可分為輸入單張影像以及多張連續影像。二維卷積網路通常使用單張影像作為輸入，使用當前影像去做方向盤預測，其缺點為缺少時間上的連續特徵，預測也相對不穩定。在三維卷積網路通常使用多張影像作為輸入，使用最近幾幀去做方向盤預測，解決二維卷積網路預測不穩定的缺點，但其代價為網路架構比較複雜。本論文利用三維卷積擷取空間與時間特徵的能力，建構出一個端對端方向盤自動控制模型。我們透過多項實驗改進架構，並且在公開數據集Udacity取得很好的成績，我們提出的模型可以有效率的保留空間以及時間資訊。我們與現今先進的模型比較可以看出我們在預測方向盤角度能有更好的準度，可以達到3.4度均方根誤差。
本模型除了在電腦上做開發，我們更將此模型放至嵌入式平台上做驗證。在車用嵌入式平台NVidia Drive PX2 上在112*112輸入下可以達到10fps。本研究利用車輛模擬分析軟體 CarSim 建構出硬體迴路模擬，透過硬體迴路模擬驗證其演算法可行性，以保證未來實車測試的安全性

End-to-end steering control methods can divided in two kinds of methods: single image base and sequence-images base. 2D convolution networks usually use single image as input and predict steering wheel angle by current frame. The disadvantages are the lack of continuous features in temporal domain, and the prediction is relatively unstable. 3D convolutional networks usually use multiple images as input, and predict steering wheel angle with series frames, which solves the disadvantage of 2D convolutional network prediction instability. The cost is 3D convolutional networks is more complicated than 2D convolutional network is. In this thesis, a 3D convolution method is used to construct an end-to-end steering wheel control model. We improved the architecture through experiments and achieved good results in the public dataset Udacity. Our purposed model can effectively preserve temporal and spatial information. The comparison with other state-of-the-art models shows our superior performance in predicting steering angle. We can achieve 3.4 degree in root mean square error.
The purposed model is not only developed on PCs but implemented on embedded systems. We can achieve 112*112 input resolution with 10 fps on automotive embedded system, NVidia Drive PX2. The vehicle simulation software, CarSim, was used to construct a hardware-in-the-loop simulation. The feasibility of the algorithm was verified by hardware loop simulation to ensure the safety of real vehicle test in the future.

摘要 I
Abstract II
誌謝 IV
List of Figures VII
List of Tables IX
Chapter 1 Introduction 1
1.1 Background 1
1.2 Organization of Thesis 3
Chapter 2 Related Works 4
2.1 Overview 4
2.2 Mediated perception approach 5
2.3 Behavior reflex approach 6
Chapter 3 Proposed Algorithm and Optimization 14
3.1 3D Convolutional Network 14
3.1.1 3D Convolutional Layer 14
3.1.2 3D Convolutional Network 15
3.1.3 Fully connected layer extension 17
3.2 Optimization 18
3.2.1 Input length 19
3.2.2 Input resolution 19
3.2.3 Number of Kernel 20
3.3 Final Purposed Architecture 21
Chapter 4 Dataset 23
4.1 Dataset collection 23
4.2 Data augmentation 26
4.2.1 Brightness augmentation 26
4.2.2 Flip augmentation 27
4.2.3 Shift augmentation 28
4.2.4 Rotation augmentation 28
4.3 Data Preprocessing 29
Chapter 5 Implementation and Performance Evaluation 30
5.1 Implementation 30
5.1.1 Device Specification 30
5.1.2 CarSim Simulator 31
5.2 Performance Evaluation 32
5.2.1 Loss Function and Optimization 32
5.2.2 Comparison with Competing Algorithms 33
5.2.3 Comparison with Different Input 35
5.3 Hardware-in-the-loop (HIL) Simulation Flow 36
5.3.1 Control Area Network (CAN) 36
5.3.2 Simulation Flow 37
5.4 HIL Simulation Result 38
5.5 Visualize Feature Maps 39
Chapter 6 Conclusion and Future Work 42
Reference 43

[1] S. Singh, “Critical Reasons for Crashes Investigated in the National Motor Vehicle Crash Causation Survey,” NHTSAs National Center for Statistics and Analysis, Tech. Rep., 2015.
[2] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” in Advances in neural information processing systems, 2012, pp. 1097–1105.
[3] C. Chen, A. Seff, A. L. Kornhauser, and J. Xiao, “Deepdriving: Learning affordance for direct perception in autonomous driving,” in Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 2722–2730.
[4] M. Bojarski, D. Del Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel, M. Monfort, U. Muller, J. Zhang, et al.,“End to End Learning for Self-Driving Cars,” arXiv preprint arXiv:1604.07316, 2016.
[5] M. Bojarski, P. Yeres, A. Choromanska, K. Choromanski, B. Firner, L. Jackel, and U. Muller, “Explaining how a deep neural network trained with end-to-end learning steers a car,” arXiv preprint arXiv:1704.07911,2017.
[6] L. Chi and Y. Mu, “Deep steering: Learning end-to-end driving model from spatial and temporal visual cues,” arXiv preprint arXiv:1708.03798, 2017.
[7] U. Muller, J. Ben, E. Cosatto, B. Flepp, and Y. L. Cun, “Off-road obstacle avoidance through end-to-end learning,” in Advances in neural information processing systems, 2006, pp. 739–746
[8] H. Xu, Y. Gao, F. Yu, and T. Darrell, “End-to-end learning of driving models from large-scale video datasets,” in Computer Vision and Pattern Recognition, 2017
[9] Z. Yang, Y. Zhang, J. Yu, J. Cai, and J. Luo, “End-to-end multi-modal multi-task vehicle control for self-driving cars with visual perception,” arXiv preprint arXiv:1801.06734, 2018.
[10] M. D. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks,” in European conference on computer vision. Springer, 2014, pp. 818–833.
[11] M. Bojarski, A. Choromanska, K. Choromanski, B. Firner, L. Jackel, U. Muller, and K. Zieba, “Visualbackprop: visualizing cnns for autonomous driving,” arXiv preprint arXiv:1611.05418, 2016
[12] X. Shi, Z. Chen, H. Wang, D. Yeung, W. Wong, and W. Woo, “Convolutional LSTM network: A machine learning approach for precipitation nowcasting,” in Neural Information Processing Systems, 2015
[13] B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, “Learning deep features for discriminative localization,” in Computer Vision and Pattern Recognition, 2016.
[14] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in International Conference on Learning Representations, 2015
[15] V. Rausch, A. Hansen, E. Solowjow, C. Liu, E. Kreuzer, and J. K. Hedrick, “Learning a deep neural net policy for end-to-end control of autonomous vehicles, ” in IEEE American Control Conference, 2017.
[16] B. Huval, T. Wang, S. Tandon, J. Kiske, W. Song, J. Pazhayampallil, M. Andriluka, P. Rajpurkar, T. Migimatsu, R. Cheng-Yue, F. Mujica, A. Coates, and A. Y. Ng, “An empirical evaluation of deep learning on highway driving, ” CoRR, abs/1504.01716, 2015.
[17] D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, “Learning spatiotemporal features with 3d convolutional networks, ” in International Conference on Computer Vision ,2015.
[18] Ana I. Maqueda, A. Loquercio, G. Gallego, N. Garc´ıa, and D.Scaramuzza, “Event-based vision meets deep learning on steering prediction for self-driving cars, ” in IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[19] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation, 9(8):1735–1780,1997.
[20] D. A. Pomerleau, “Alvinn: An autonomous land vehicle in a neural network,” in Advances in neural information processing systems, 1989, pp. 305–313.
[21] D. Kingma and J. Ba, “Adam: A method for stochastic optimization, ” In International Conference on Learning Representations, 2015
[22] E. Santana and G. Hotz, “Learning a driving simulator,” arXiv preprint arXiv:1608.01230, 2016.
[23] D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114, 2013.
[24] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in neural information processing systems, pages 2672–2680, 2014.
[25] Z. Chen and X. Huang, “End-to-End Learning for Lane Keeping of Self-Driving Cars,” in IEEE Intelligent Vehicles Symposium, 2017
[26] S. Du. H. Guo and A. Simpson, “Self-Driving Car Steering Angle Prediction Based on Image Recognition,” Based on Image Recognition. http://cs231n.stanford.
edu/reports/2017/pdfs/626.pdf
[27] https://medium.com/udacity