臺灣博碩士論文加值系統

在生活中人們常常經由語言以外的方式表達情緒，其中又以人的臉部表情作為強烈表達情緒的溝通管道，透過觀察對方的情緒，人們得以採取相對應的行動以達到自己的目的，故情緒辨識成為人工智慧產業相當重要的技術，相關應用包括安全駕駛、商業廣告評估等等。近幾年雖然已有許多成人情緒辨識模型出現，但因幼兒的臉部特徵有別於成人，導致成人的情緒辨識模型在辨識幼兒情緒上的成效不彰。本研究針對此問題，使用幼兒園四個班級的學生作為資料集，建立一基於卷積神經網路建構幼兒情緒預測模型，一開始先對資料集圖片進行灰階、裁切等處理，並為每張照片標記一種情緒作為訓練時應得出的正確結果，共分為生氣、厭惡、害怕、高興、悲傷、驚訝、無情緒、質疑七種情緒，接著將資料輸入進卷積神經網路架構中，因卷積神經網路有別於傳統特徵工程方式建立之網路，不需手動設計即可自行學習出該類別的情緒特徵，且同時透過在模型中加入殘差模組(residual modules)及深度可分離卷積運算(depth-wise separable convolutions)減少網路的複雜度與參數數量，最後使用支援向量機進行結果分類，完成訓練後經由測試資料對模型評估準確率，以證明本研究模型可有效辨識幼兒情緒。

In daily life, people often express emotions through ways other than language. Among them, people’s facial expression is the most common way to express their emotion, people can take corresponding actions to achieve their goals via facial expression. Thus the emotion recognition has become an important technology in the artificial intelligence industry, the related work including Safe Driving, Commercial Advertising Evaluation, etc. Although there are several emotion recognition models for adult appeared in recent years, they didn’t have better performance on children facial expression through children has different facial features from adult’s. To solve this problem, this research takes four different classes in the kindergarten as the dataset, building children emotion recognition model based on Convolutional Neural Network. First, this research cropped and converted the dataset picture to grayscale, and labeled an emotion in seven classes as the right result for the picture, including angry, disgust, fear, happiness, sadness, surprise, neutral, contempt. Then input these data into the Convolutional Neural Network. Since Convolutional Neural Network is different from the neural network based on traditional feature engineering, it can learn the emotional features for the classes itself without manual setting, At the same time, this research added residual modules and depth-wise separable convolutions to reduce the depth and parameter in the Convolutional Neural Network. At last, this research used a support vector machine to classify the result. After finishing the training, this research used the test data to evaluate the accuracy of the model, to prove the model can effectively recognize children's expression emotions.

摘　要 i
ABSTRACT iii
誌　謝 v
目　錄 vi
表目錄 ix
圖目錄 x
第一章 緒論 1
1.1 研究動機與目的 1
1.2 研究貢獻 1
1.3 章節編排 2
第二章 文獻探討 3
2.1 深度學習(Deep Learning) 3
2.1.1 深度神經網路 4
2.1.2 激勵函式(Activation Function) 5
2.1.3 最佳化方法(Optimization Method) 6
2.1.4 應用深度學習於情緒辨識 7
2.2 卷積神經網絡 7
2.2.1 Convolutional Neural Network核心架構 7
2.2.2 CNN衍生之架構 11
2.3 Residual Network架構 12
2.4 Xception架構 14
2.5 支援向量機 16
2.5.1 多類SVM應用於圖像分類 18
2.6 局部響應標準化(Local Response Normalization) 19
2.7 避免過擬合的方法 19
2.7.1 L2正規化 20
2.7.2 丟棄法 20
第三章 情緒辨識模型設計 23
3.1 模型設計流程 23
3.2 資料準備與資料集 24
3.2.1 fer2013資料集與欄位組成 25
3.2.2 幼兒資料集與欄位組成 26
3.2.3 資料的前置處理 28
3.2.4 資料切割 29
3.3 建構深度神經網路 29
3.3.1 神經網路架構 29
3.3.2 神經網路的損失函數 31
3.3.3 深度可分離卷積運算 32
第四章　模型實作 36
4.1 系統資訊 36
4.2 深度神經網路模型 36
4.2.1 不平衡資料處理 36
4.2.2 基於平方鉸鏈損失之支援向量機目標函數 37
4.2.3 L2正規化 37
4.2.4 深度神經網路訓練 38
第五章　實驗 40
5.1 不同懲罰參數比較 40
5.2 不同最佳化方法及學習率 44
5.3 不同網路模型比較 47
第六章　結論與未來研究方向 53
6.1 結論 53
6.2 未來研究方向 53
參考文獻 54

[1] 周育如、黃迺毓，「文化下的建構：幼兒對情緒內涵的理解」，人類發展與家庭學報，第十三期，2011，第1-26頁。
[2] 人工神經網路, Available: https://zh.wikipedia.org/wiki/%E4%BA%BA%E5%B7%A5%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C#cite_note-21
[3] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Nature, vol. 323, no. 6088, pp. 533–536, Oct. 1986.
[4] J. Han and C. Moraga, “The influence of the sigmoid function parameters on the speed of backpropagation learning,” in From Natural to Artificial Neural Computation, Springer, Berlin, Heidelberg, pp. 195–201, 1995.
[5] V. Nair and G. E. Hinton, “Rectified linear units improve restricted boltzmann machines,” Proceedings of 27th International Conference on Machine Learning, pp. 807-814, 2010.
[6] J. Duchi, E. Hazan, and Y. Singer, “Adaptive Subgradient Methods for Online Learning and Stochastic Optimization,” Journal of Machine Learning Research, vol. 12, no. Jul, pp. 2121–2159, 2011.
[7] D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” arXiv:1412.6980 [cs], Dec. 2014.
[8] Deepjoy Das and Alok Chakrabarty, “Emotion Recognition from Face Dataset Using Deep Neural Nets,” International Symposium on Innovations in Intelligent Systems and Applications, 2016.
[9] Y. Lecun, L. Bottou, Y. Bengio and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the Institute of Electrical and Electronics Engineers, vol. 86, issue 11, pp. 2278-2324, 1998.
[10] Karen Simonyan and Andrew Zisserman, “Very deep convolutional networks for large-scale image recognition,” Proceedings of the International Conference on Learning Representations, 2015.
[11] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich, “Going deeper with convolutions,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9, 2015.
[12] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, “Deep Residual Learning for Image Recognition,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-788, 2016.
[13] Francois Chollet, “Xception: Deep learning with depthwise separable convolutions,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[14] Octavio Arriaga, Matias Valdenegro-Toro, Paul Plöger, “Real-time Convolutional Neural Networks for Emotion and Gender Classification,” Proceedings of the International Conference on Robotics and Automation, 2017.
[15] C. Cortes and V. Vapnik, “Support-vector networks,” Machine Learning, vol. 20, pp. 273-297, 1995.
[16] S. K. Shrivastava and S. S. Gharde, “Support Vector Machine for Handwritten Devanagari Numeral Recognition,” International Journal of Computer Applications, vol. 7, no. 11, pp. 9-14, 2010.
[17] Ganapathiraju, J. Hamaker, and J. Picone, “Applications of support vector machines to speech recognition,” IEEE Transactions on Signal Processing, vol.52, no. 8, pp. 2348-2355, 2004.
[18] R. Kumar, A. Srivastava, B. Kumari, and M. Kumar, “Prediction of beta-lactamase and its class by Chou’s pseudo-amino acid composition and support vector machine,” Journal of Theoretical Biology, vol. 365, pp. 96-103, 2015.
[19] Abien Fred M. Agarap, “An Architecture Combining Convolutional Neural Network (CNN) and Support Vector Machine (SVM) for Image Classification,” arXiv:1712.03541 [cs], Feb. 2019.
[20] Chandrashekhar S. Janadri, B. G. Sheeparamatti, and Vishwanath Kagawade, “Multiclass classification of kirlian images using svm technique,” Proceedings of International Conference on Advances in Computing, Communications and Informatics, 2017.
[21] Monzurul Islam, Anh Dinh, Khan Wahid, and Pankaj Bhowmik, “Detection of potato diseases using image segmentation and multiclass support vector machine,” Proceedings of IEEE 30th Canadian Conference on Electrical and Computer Engineering, 2017.
[22] Qilong Li and Xiaohong Wang, “Image Classification Based on SIFT and SVM,” Proceedings of IEEE/ACIS 17th International Conference on Computer and Information Science, 2018.
[23] A. Krizhevsky, I. Sutskever, and G. Hinton, “Imagenet classification with deep convolutional neural networks,” In Advances in neural information processing systems, pp. 1097-1105, 2012.
[24] Y. Bengio, “Practical recommendations for gradient-based training of deep architectures,” In Neural Networks: Tricks of the Trade, Springer, pp. 437-478, 2013.
[25] N. Srivastava, G. E. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: a simple way to prevent neural networks from overfitting,” The Journal of Machine Learning Research, vol. 15, pp. 1929-1958, 2014.
[26] fer2013 dataset, Available: https://www.kaggle.com/deadskull7/fer2013
[27] P. Ekman, “Facial Expressions of Emotion: An Old Controversy and New Findings,” Philosophical Transactions of the Royal Society of London, Series B: Biological Science, no. 335, pp. 63-69, 1992.
[28] Encoding skill, Available: http://scikit-learn.org/stable/modules/classes.html#module- sklearn.preprocessing
[29] Ching-Pei Lee and Chih-Jen Lin, “A Study on L2-Loss (Squared Hinge-Loss) Multiclass SVM,” Neural Computation, vol. 25, issue 5, pp. 1302-1323, 2013.
[30] Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam, “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications,” arXiv:1704.04861 [cs], Apr. 2017.
[31] M. Abadi et al., “TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems,” Mar. 2016.
[32] Compute class weight, Available: https://scikit-learn.org/stable/modules/generated/sklearn.utils.class_weight.compute_class_weight.html