臺灣博碩士論文加值系統

近幾年來，電腦軟、硬體技術的發展，各種影像處理、辨識等的複雜問題，均可利用電腦的快速運算來解決，傳統的影像辨識系統，缺乏彈性及低準確率，早期的多層感知器及深度學習被應用最廣，但其本身隱藏層類神經元的架構過於簡單，而有收斂速度過慢等問題，使得網路訓練時間過長。
目前多數的研究者均採用R-CNN(Region-Convolution Neural Network)架構於即時影像辨識系統，其雖然準確率極高，但架構十分複雜，且需要大量硬體支援，本論文乃提出使用輕量級的捲積類神經網路(Convolution Neural Network)並將其導入樹莓派微電腦硬體中，用於即時快速偵測車輛。本文訓練數據集經過數據增強及正規化才輸入捲積類神經網路，以加速網路的學習，利用捲積層提取特徵及池化層壓縮特徵，再由隱藏層神經元辨識，輸出種類的概率，以逆向傳播方式調整訓練網路參數。本文所提之方法，網路訓練收斂時間短，效率佳，架構簡單，可以導入許多可攜式的微晶片模組，更具一般化的實用。
經實驗調校與優化網路參數群後，以最佳參數做實際推論測試，本文使用11層神經網路為最佳模型，完成車輛辨識的訓練後，其訓練資料集準確率為96%。使用測試數據進行驗證，其準確率為94%。

With rapid development of computer hardware and software technology, complex image processing and recognition can be performed by computer in recent years. The traditional image recognition has some issues of lack of flexibility and poor accuracy. These disadvantages have improved by the neural network. The earlier multi-layer perceptron is widely applied in various areas, however, its hidden layers are simple and slower convergence issues to cause longer training time.
Most researchers apply R-CNN to the real time recognition system. These methods have higher accuracy but their layers are more complicated and need a lot of hardware resources to support. In this thesis, it proposes a CNN with lighter neural layers implemented to the Raspberry Pi of small single-board computer, and use this method to real time recognize vehicles. For speed up network training, this thesis uses data augmentation and normalization to enhance training dataset before input. Then, applying convolution layers extract features and pooling layers compress, and the backward propagation method is used to update network parameters. This proposed method have short time for training convergence and better efficient. Due to simple structure, it can be easy implemented other single board modules to improve generalization.
Based on the experiments, the best model is 11 layers used for future inference after optimized network parameters. The accurate rate of training dataset is 96% after training stage processing. The test dataset is used for validation and the accurate rate is 94%. The proposed method has simple structures and higher accuracy to the real time recognition system.

目錄
摘要 I
ABSTRACT II
目錄 III
圖目錄 VII
表目錄 X
第一章 緒論 1
1.1研究背景和動機 1
1.2研究方法 2
1.3論文架構 4
第二章 文獻探討 5
2.1起源 5
第一次AI浪潮 6
第二次AI浪潮 6
第三次AI浪潮 7
2.2演算法 8
遺傳演算法 8
專家系統 8
類神經網路 9
2.3應用領域 10
語音辨識 10
影像辨識 10
自然語言處理 10
2.4機器學習 11
2.5機器學習方法 12
監督式學習 12
非監督式學習 12
半監督式學習 12
增強學習 12
2.6深度學習 13
2.7類神經網路 14
2.8類神經網路種類 15
RNN 遞迴神經網路(Recurrent Neural Network) 15
LSTM 長短時記憶網路(Long and Short Term Memory) 16
捲積網路(Convolutional Network) 17
生成對抗網路GAN（Generative Adversarial Network） 18

第三章 實驗架構與流程 19
3.1 硬體規格 19
3.2 軟體架構與流程 21
數據集 22
資料前處理 22
資料擴增 23
資料正規化 24
捲積類神經網路模型 24
捲積層: 24
池化層: 26
全連接層: 26
隱藏層: 27
淘汰層(Dropout) 29
輸出 29
7. 活化函數 29
Sigmoid函數 30
tanh函數 31
Softmax函數 32
ReLU函數(Rectified Linear Unit) 33
8. 學習優化器 34
隨機梯度下降法(Stochastic Gradient Decent) 34
Momentum(動量接近法) 35
AdaGrad 35
RMSprop 36
Adam 37
3.3模型結構 38
第四章 實驗結果 43
4.1活化函數 43
4.2學習優化器 46
4.3捲積層數量 48
第五章 結論與未來展望 56
5.1結論 56
5.2未來展望 56
第六章 參考文獻 57

參考文獻
[1] Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton, "ImageNet classification with deep convolutional neural networks," NIPS'12 Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1, Pages 1097-1105
[2] Karen Simonyan, Andrew Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition," LCLR 2015, arXiv：1409.1556v6
[3] S. Chopra, R. Hadsell, Y. LeCun, Learning a similarity metric discriminatively, with application to face verification, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), 20-25 June 2005, ISBN: 0-7695-2372-2
[4] Mohammad Javad Shafiee, Francis Li, Brendan Chwyl, Alexander Wong, "SquishedNets: Squishing SqueezeNet further for edge device scenarios via deep evolutionary synthesis," 20 Nov, 2017, arXiv：1711.07459v1
[5] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich, "Going deeper with convolutions," 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015, pages1-9
[6] Yansheng Li, Chao Tao, Yihua Tan, Ke Shang, and Jinwen Tian, "Unsupervised Multilayer Feature Learning for Satellite Image Scene Classification," IEEE GEOSCIENCE and REMOTE SENSING LETTERS, VOL. 13, NO. 2, FEBRUARY 2016, pages157-161
[7] Yakoub Bazi, Farid Melgani, "Convolutional SVM Networks for Object Detection in UAV Imagery," IEEE Transactions on Geoscience and Remote Sensing, Volume 56, Issue 6, June 2018, Page 3107 - 3118
[8] Clay Sheppard, Maryam Rahnemoonfar, "Real-time scene understanding for UAV imagery based on deep convolutional neural networks," 2017, IEEE International Geoscience and Remote Sensing Symposium (IGARSS), July 2017, page 2153-7003
[9] Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun, "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 39, Issue 6, June 2017, Page 1137 - 1149
[10] Heehoon Kim, Hyoungwook Nam, Wookeun Jung, Jaejin Lee, "Performance analysis of CNN frameworks for GPUs, " 2017 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) DOI: 10.1109/ISPASS.2017.7975270
[11] Bin Liu, Wencang Zhao, Qiaoqiao Sun, "Study of object detection based on Faster R-CNN," 2017 Chinese Automation Congress (CAC) DOI: 10.1109/CAC. 2017. 8243900
[12] Shih-Chung Hsu, Chung-Lin Huang, Cheng-Hung Chuang, "Vehicle detection using simplified fast R-CNN," 2018 International Workshop on Advanced Image Technology (IWAIT) DOI: 10.1109/IWAIT. 2018. 8369767
[13] Rui Xu, Sheng Ma, Yang Guo"Performance Analysis of Different Convolution Algorithms in GPU Environment," 2018 IEEE International Conference on Networking, Architecture and Storage (NAS) DOI: 10.1109/NAS. 2018. 8515695
[14] Shoji Kido, Yasusi Hirano, Noriaki Hashimoto, "Detection and classification of lung abnormalities by use of convolutional neural network (CNN) and regions with CNN features (R-CNN)," 2018 International Workshop on Advanced Image Technology (IWAIT) DOI: 10.1109/IWAIT. 2018. 8369798
[15] Hideaki Yanagisawa, Takuro Yamashita, Hiroshi Watanabe," A study on object detection method from manga images using CNN," 2018 International Workshop on Advanced Image Technology (IWAIT) DOI: 10.1109/IWAIT. 2018. 8369633
[16] Liqiong Lu, Yaohua Yi, Faliang Huang, Kaili Wang, Qi Wang, "Integrating Local CNN and Global CNN for Script Identification in Natural Scene Images," IEEE Access ( Volume: 7 ) DOI: 10.1109/ACCESS. 2019. 2911964
[17] Suresh Prasad Kannojia, Gaurav Jaiswal, "Ensemble of Hybrid CNN-ELM Model for Image Classification," 2018 5th International Conference on Signal Processing and Integrated Networks (SPIN) DOI: 10.1109/SPIN. 2018. 8474196
[18] Bin Liu, Wencang Zhao, Qiaoqiao Sun, "Study of object detection based on Faster R-CNN," 2017 Chinese Automation Congress (CAC) DOI: 10.1109/CAC. 2017. 8243900
[19] Wei Zhang, Shihao Wang, Sophanyouly Thachan, Jingzhou Chen, Yuntao Qian, "Deconv R-CNN for Small Object Detection on Remote Sensing Images," IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium DOI: 10.1109/IGARSS. 2018. 8517436
[20] Shih-Chung Hsu, Chung-Lin Huang, Cheng-Hung Chuang, "Vehicle detection using simplified fast R-CNN," 2018 International Workshop on Advanced Image Technology (IWAIT) DOI: 10.1109/IWAIT. 2018. 8369767