(3.232.129.123) 您好!臺灣時間:2021/03/04 17:53
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果

詳目顯示:::

我願授權國圖
: 
twitterline
研究生:蘇崇維
研究生(外文):SU,CHUNG-WEI
論文名稱:使用卷積神經網路實現人臉情緒辨識之互動系統
論文名稱(外文):An Interactive System of Face-Emotion Recognition Using Convolutional Neural Networks
指導教授:陸清達陸清達引用關係廖岳祥廖岳祥引用關係
指導教授(外文):Lu,Ching-TaAnthony Y. H. Liao
口試委員:陸清達廖岳祥陳松雄王玲玲
口試委員(外文):Lu,Ching-TaAnthony Y. H. LiaoChen,Song-ShyongWang,Ling-Ling
口試日期:2020-06-04
學位類別:碩士
校院名稱:亞洲大學
系所名稱:行動商務與多媒體應用學系
學門:電算機學門
學類:電算機應用學類
論文種類:學術論文
論文出版年:2020
畢業學年度:108
語文別:中文
論文頁數:52
中文關鍵詞:人臉辨識深度學習卷積神經網路人工智慧情緒辨識人機互動
外文關鍵詞:face recognitiondeep-learningconvolutional neural networkartificial intelligenceemotion recognitionhuman-computer interaction
相關次數:
  • 被引用被引用:0
  • 點閱點閱:55
  • 評分評分:系統版面圖檔系統版面圖檔系統版面圖檔系統版面圖檔系統版面圖檔
  • 下載下載:5
  • 收藏至我的研究室書目清單書目收藏:0
在現在這個科技發展快速的年代,人們越來越講求各種便利生活的方式,所以
人工智慧的發展可謂是蒸蒸日上,使得人工智慧更加貼近我們的生活,逐漸的取
代許多人類可以做到的事情,在深度學習中影像的辨識更是被廣泛的運用,最直
接的應用就是用於人臉的擷取、辨識與身分辨識。近年來,深度學習神經網路也
使用在人類的情緒辨識上,透過多種方法如何有效並正確的辨識出即時的情緒是
一項極為重要的研究方向。本文提出一個結合身分辨識與即時情緒辨識的人機互
動系統,蒐集與分析人臉特徵,並且分別切割成眼睛部位和嘴巴部位,使用卷積
神經網路學習特徵,辨識使用者當下的情緒。首先以攝像鏡頭拍攝人的影像,然
後以 Viola-Jones 演算法分別切割人臉、雙眼與嘴巴區域,再將切割的區域影像
進行影像特徵強化,達到提高卷積神經網路辨識使用者身分與情緒的正確率,最
後輔以 3D 動畫進行人機互動。實驗結果證明本文提出的情緒辨識系統可以快速
且正確地進行身分與情緒辨識,也可以讓使用者體驗人機互動的樂趣。
The applications of deep-learning neural networks (DLNN) intelligence are
progressively developed in various fields. The DLNN can be employed for the face
recognition, face identification, and emotion recognition. This thesis proposes a humancomputer interaction system that combines face recognition and emotion recognition,
and interesting 3D animation. First, the Viola-Jones algorithm is used to segment the
areas around the face, eyes, and mouth. A convolutional neural network (CNN) is
employed to recognize the identity and emotion status according to the features in eye
and mouth areas. Finally, a 3D animation is played according to the identified emotion.
Experimental results show that the proposed emotion interaction system can accurately
recognize the identity and emotion, and also allow users to experience the fun of
human-computer interaction.
摘要 ................................................................................................................................i
Abstract.........................................................................................................................ii
致謝 ..............................................................................................................................iii
目錄 ..............................................................................................................................iii
圖目錄 ...........................................................................................................................v
第一章 緒論 .................................................................................................................1
1.1 研究動機與目的.............................................................................................1
1.2 文獻探討.........................................................................................................1
第二章 人臉範圍切割與深度學習神經網路 .............................................................6
2.1 Viola-Jones 演算法..........................................................................................6
2.1.1 人臉範圍切割......................................................................................6
2.1.2 眼睛特徵切割......................................................................................7
2.1.3 嘴巴特徵切割......................................................................................7
2.2 卷積神經網路.................................................................................................8
2.2.1 卷積層..................................................................................................8
2.2.2 池化層..................................................................................................9
2.2.3 全連接層............................................................................................10
2.3 情緒辨識之卷積神經網路訓練...................................................................11
第三章 人臉情緒辨識與互動內容 ...........................................................................12
3.1 人臉情緒辨識與互動系統流程...................................................................12
3.2 人臉影像擷取...............................................................................................12
3.3 資料預處理...................................................................................................14
3.4 嘴巴區域偵測...............................................................................................15
3.5 眼睛區域偵測...............................................................................................16
iiii
3.6 人臉卷積神經網路訓練...............................................................................17
3.6.1 嘴巴特徵部位卷積神經網路訓練....................................................21
3.6.2 眼睛特徵部位卷積神經網路訓練....................................................22
3.7 3D 動畫應用..................................................................................................26
第四章 實驗結果 .......................................................................................................29
4.1 人臉辨識精確率評估...................................................................................30
4.1.1 嘴巴情緒辨識精確率評估................................................................32
4.1.2 眼睛情緒辨識精確率評估................................................................34
4.2 人臉辨識召回率評估...................................................................................38
4.2.1 嘴巴情緒辨識召回率評估................................................................39
4.2.2 眼睛情緒辨識召回率評估................................................................40
4.3 人臉辨識 F 測度評估 ..................................................................................42
4.3.1 嘴巴情緒辨識 F 測度評估 ...............................................................42
4.3.2 眼睛情緒辨識 F 測度評估 ...............................................................43
4.4 系統整體評估...............................................................................................45
4.5 情緒辨識互動系統.......................................................................................47
第五章 結論 ...............................................................................................................49
參考文獻 .....................................................................................................................50
圖目錄
圖 2.1 人臉切割示意圖,(a)人臉辨識影像;(b)人臉切割影像。.....................7
圖 2.2 眼睛特徵示意圖,(a)演算法自定義切割框示意圖;(b)加大自訂義切
割框示意圖。.........................................................................................................7
圖 2.3 嘴巴特徵示意圖,(a)演算法自定義切割框示意圖;(b)加大自訂義切
割框示意圖。.........................................................................................................8
圖 2.4 CNN 架構圖 ................................................................................................8
圖 2.5 卷積層示意圖.............................................................................................9
圖 2.6 池化層示意圖...........................................................................................10
圖 2.7 全連接層示意圖.......................................................................................10
圖 2.8 CNN 訓練流程圖 ......................................................................................11
圖 3.1 系統流程圖...............................................................................................13
圖 3.2 表情特徵影像,(a)「開心」之表情特徵影像;(b)「不開心」之表情
特徵影像。...........................................................................................................14
圖 3.3 圖像預處理比較圖;分別使用(a)二值化;(b)增加對比度;(c)統一平
均灰度的方法.......................................................................................................15
圖 3.4 嘴巴區域偵測示意圖...............................................................................16
圖 3.5 眼睛區域偵測示意圖...............................................................................16
圖 3.6 使用各 600 張影像訓練的人臉 CNN 軌跡圖與結果圖 (a)辨認精確度
軌跡圖 (b)人臉 CNN 訓練結果圖......................................................................18
圖 3.7 使用各 1200 張影像訓練的人臉 CNN 軌跡圖與結果圖 (a)辨認精確度
軌跡圖 (b)人臉 CNN 訓練結果圖......................................................................19
圖 3.8 使用 4 層卷積層數量的人臉 CNN 訓練軌跡圖與結果圖 (a)訓練軌跡
圖 (b)訓練結果圖 ................................................................................................20
圖 3.9 嘴巴 CNN 訓練軌跡圖與結果圖 (a)訓練軌跡圖 (b)訓練結果圖 .......22
v
圖 3.10 使用 3 層卷積層訓練眼睛 CNN 訓練軌跡圖與結果圖 (a)訓練軌跡圖
(b)訓練結果圖 ......................................................................................................23
圖 3.11 使用 4 層卷積層訓練眼睛 CNN 訓練軌跡圖與結果圖 (a)訓練軌跡圖
(b)訓練結果圖 ......................................................................................................24
圖 3.12 使用 5 層卷積層訓練眼睛 CNN 訓練軌跡圖與結果圖 (a)訓練軌跡圖
(b)訓練結果圖 ......................................................................................................25
圖 3.13 角色建模模型圖.....................................................................................26
圖 3.14 模型套入 UV 材質.................................................................................26
圖 3.15 角色、骨架、控制器完成圖.................................................................27
圖 3.16 打招呼的動畫基本動作.........................................................................27
圖 3.17 對應「開心」之動畫截圖.....................................................................27
圖 3.18 對應「不開心」之動畫截圖.................................................................28
圖 3.19 對應「心情不錯」之動畫截圖.............................................................28
圖 4.1 測示範例圖...............................................................................................30
圖 4.2 人臉辨認測試結果範例圖.......................................................................30
圖 4.3 其他辨識結果範例圖................................................................................31
圖 4.4 人臉辨識結果圖.......................................................................................31
圖 4.5 嘴巴情緒辨識測試結果範例圖...............................................................32
圖 4.6 其他辨識結果範例圖...............................................................................33
圖 4.7 嘴巴情緒辨識結果為「開心」之辨認結果圖.......................................33
圖 4.8 嘴巴情緒辨識結果為「不開心」之辨認結果圖...................................34
圖 4.9 眼睛情緒辨識測試結果範例圖...............................................................35
圖 4.10 其他辨識結果範例圖.............................................................................36
圖 4.11 眼睛情緒辨識結果為「開心」之辨認結果圖.....................................37
圖 4.12 眼睛情緒辨識結果為「不開心」之辨認結果圖.................................38
圖 4.13 人臉辨識結果圖.....................................................................................39
v
圖 4.14 嘴巴情緒辨識召回率比較圖.................................................................40
圖 4.15 眼睛情緒辨識召回率比較圖.................................................................41
圖 4.16 嘴巴情緒辨識 F 測度比較圖.................................................................43
圖 4.17 眼睛情緒辨識 F 測度比較圖.................................................................45
圖 4.18 特徵不佳之較遠鏡頭影像示意圖.........................................................46
圖 4.19 嘴巴特徵擷取不完全示意圖.................................................................46
圖 4.20 眼睛特徵擷取不完全示意圖.................................................................47
圖 4.21 使用者介面示意圖.................................................................................47
圖 4.22 系統成果範例.........................................................................................48
[1] Zhang, K., Zhang, Z., Li, Z., & Qiao, Y. (2016). Joint face detection and alignment
using multitask cascaded convolutional networks. IEEE Signal Processing
Letters, 23(10), pp.1499-1503.
[2] Li, H., Lin, Z., Shen, X., Brandt, J., & Hua, G. (2015). A convolutional neural
network cascade for face detection. In Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition, pp. 5325-5334.
[3] Mathias, M., Benenson, R., Pedersoli, M., & Van Gool, L. (2014). Face detection
without bells and whistles. In Proceedings of the European Conference on
Computer Vision, pp. 720-735.
[4] Chen, D., Ren, S., Wei, Y., Cao, X., & Sun, J. (2014). Joint cascade face detection
and alignment. In Proceedings of the European Conference on Computer Vision,
pp. 109-122.
[5] Farfade, S. S., Saberian, M. J., & Li, L. J. (2015). Multi-view face detection using
deep convolutional neural networks. In Proceedings of the 5th ACM on
International Conference on Multimedia Retrieval, pp. 643-650
[6] Jiang, H., & Learned-Miller, E. (2017). Face detection with the faster R-CNN.
In Proceedings of the IEEE International Conference on Automatic Face &
Gesture Recognition, pp. 650-657.
[7] Ranjan, R., Patel, V. M., & Chellappa, R. (2017). Hyperface: A deep multi-task
learning framework for face detection, landmark localization, pose estimation, and
gender recognition. IEEE Transactions on Pattern Analysis and Machine
Intelligence, 41(1), pp. 121-135.
[8] Qin, H., Yan, J., Li, X., & Hu, X. (2016). Joint training of cascaded CNN for face
detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition, pp. 3456-3465.
[9] Sun, X., Wu, P., & Hoi, S. C. (2018). Face detection using deep learning: An
improved faster RCNN approach. Neurocomputing, 299, pp.42-50.
[10] Chen, D., Hua, G., Wen, F., & Sun, J. (2016). Supervised transformer network
for efficient face detection. In Proceedings of the European Conference on
Computer Vision, pp. 122-138.
[11] Ganong, R., Waugh, D. C., Ro, Y. M., Plataniotis, K., & Studholme, C.
(2017). U.S. Patent No. 9,639,740. Washington, DC: U.S. Patent and Trademark
Office.
51
[12] Li, Y., Sun, B., Wu, T., & Wang, Y. (2016). Face detection with end-to-end
integration of a convnet and a 3d model. In Proceedings of the European
Conference on Computer Vision, pp. 420-436.
[13] Zhu, C., Zheng, Y., Luu, K., & Savvides, M. (2017). Cms-rcnn: contextual multiscale region-based CNN for unconstrained face detection. In Proceedings of the
Deep Learning for Biometrics, pp. 57-79.
[14] Zhang, J., Wu, X., Hoi, S. C., & Zhu, J. (2020). Feature agglomeration networks
for single stage face detection. Neurocomputing, 380, pp.180-189.
[15] Shi, X., Shan, S., Kan, M., Wu, S., & Chen, X. (2018). Real-time rotationinvariant face detection with progressive calibration networks. In Proceedings of
the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2295-2303.
[16] Yang, S., Luo, P., Loy, C. C., & Tang, X. (2016). Wider face: A face detection
benchmark. In Proceedings of the IEEE Conference on Computer Vision and
Pattern Recognition, pp. 5525-5533.
[17] Samira Ebrahimi Kahou, Xavier Bouthillier, Pascal Lamblin, Caglar Gulcehre,
Vincent Michalski, Kishore Konda, Sébastien Jean, Pierre Froumenty, Yann
Dauphin, Nicolas Boulanger-Lewandowski, Raul Chandias Ferrari, Mehdi Mirza,
David Warde-Farley, Aaron Courville, Pascal Vincent, Roland Memisevic,
Christopher Pal & Yoshua Bengio (2016). Emonets: Multimodal deep learning
approaches for emotion recognition in video. Journal on Multimodal User
Interfaces, 10(2), pp.99-111.
[18] Poria, S., Chaturvedi, I., Cambria, E., & Hussain, A. (2016). Convolutional MKL
based multimodal emotion recognition and sentiment analysis. In Proceedings of
the IEEE International Conference on Data Mining, pp. 439-448.
[19] Fan, Y., Lu, X., Li, D., & Liu, Y. (2016). Video-based emotion recognition using
CNN-RNN and C3D hybrid networks. In Proceedings of the 18th ACM
International Conference on Multimodal Interaction, pp. 445-450.
[20] Tzirakis, P., Trigeorgis, G., Nicolaou, M. A., Schuller, B. W., & Zafeiriou, S.
(2017). End-to-end multimodal emotion recognition using deep neural
networks. IEEE Journal of Selected Topics in Signal Processing, 11(8), pp.1301-
1309.
[21] Tarnowski, P., Kołodziej, M., Majkowski, A., & Rak, R. J. (2017). Emotion
recognition using facial expressions. Procedia Computer Science, 108, pp.1175-
1184.
[22] Ranganathan, H., Chakraborty, S., & Panchanathan, S. (2016). Multimodal
emotion recognition using deep learning architectures. In Proceedings of the
IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1-9.
52
[23] Liu, W., Zheng, W. L., & Lu, B. L. (2016). Emotion recognition using
multimodal deep learning. In Proceedings of the International Conference on
Neural Information Processing, pp. 521-529.
[24] Zhang, Y., Ji, X., & Zhang, S. (2016). An approach to EEG-based emotion
recognition using combined feature extraction method. Neuroscience Letters, 633,
pp.152-157.
[25] Jain, N., Kumar, S., Kumar, A., Shamsolmoali, P., & Zareapoor, M. (2018).
Hybrid deep neural networks for face emotion recognition. Pattern Recognition
Letters, 115, pp.101-106.
[26] Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of
simple features. In Proceedings of the IEEE Computer Society Conference on
Computer Vision and Pattern Recognition. , pp. 511-518.
[27] J. Han, Z., Zhang, F., Ringeval, B., Schuller. (2017). Prediction-based learning
for continuous emotion recognition in speech. In Proceedings of the 2017 IEEE
international conference on acoustics, speech and signal processing, pp. 5005-
5009
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top
系統版面圖檔 系統版面圖檔