臺灣博碩士論文加值系統

在現今互動藝術中，除了利用特殊機械結構設計的互動裝置外，隨著電腦的高速發展、數位運算的興起，也開始在互動藝術裝置中導入電腦視覺與機器學習，以數位化方式結合真實影像來進行參數化的藝術創作，利用相機鏡頭機獲取真實空間的人體姿態，並分析姿態變化擷取特徵進而識別人體動作與手勢。

透過識別人體的動作與手勢，可將不同角度、不同背景環境的相同動作識別歸納為同一類別，在動作觸發後依據人體位置設計相應的動畫特效。本研究將設計多種動作與姿態類別並建立其影像資料庫，將每種類別設計出相對應的視覺特效，使觀者能實時的觸發出相對應的動畫特效，實現由觀者透過自身的肢體語言來控制介面並以第一人稱視角融入其場景特效。

In today's interactive art, in addition to interactive devices designed with special mechanical structures, with the rapid development of computers and the rise of digital computing, computer vision and machine learning have also been introduced into interactive art devices to combine real images in a digital way for parametric art creation, using cameras to acquire real human postures in space and analyze the changes in posture to extract features for recognizing human The camera lens machine is used to acquire the human pose in real space, and analyze the pose change to capture the features to recognize human movement and gesture.

　　By recognizing human movements and gestures, the same movements from different angles and in different backgrounds can be grouped into the same category, and the corresponding animation effects can be designed according to the position of the human body after the movements are triggered. In this study, we will design various motion and gesture categories and establish their image database, and design corresponding visual effects for each category, so that the viewer can trigger the corresponding animation effects in real time, and realize the viewer to control the interface through his or her own body language and integrate the first-person perspective into the scene effects.

第一章 緒論 1
1.1 研究動機及背景 1
1.2 研究問題 2
1.3 研究目的 4
1.4 研究限制 5
1.5 研究架構 6
第二章 文獻回顧 8
2.1 人機互動 8
2.1.1 圖形使用者介面(GUI) 8
2.1.2 自然互動使用者介面(NUI) 9
2.2 人體姿態偵測與動作識別 10
2.2.1 人體姿態偵測 11
2.2.2 人體動作識別 14
2.3 降維與分類演算法 16
2.3.1 分類演算法 16
2.3.2 降維演算法 19
2.4 LSTM神經網路模型 20
第三章 建立資料集與資料預處理 25
3.1 建立資料集與資料前處理 27
3.2 動作識別網路架構 29
3.2.1 關鍵點回歸模型 31
3.2.2 特徵回歸模型 35
3.2.3 基於ATTENTION機制LSTM模型 37
第四章 訓練與測試結果 40
4.1 分類模型比較分析 42
4.2 降維算法比較分析 43
4.3 特徵回歸模型比較分析 44
4.4 動作識別網路綜合分析 45
第五章 建構互動裝置 48
5.1 互動裝置開發平台 48
5.2 環境建置與模型導入 49
5.3 互動機制與效果 50
5.4 使用者測試與反饋 53
第六章 結論與未來展望 55
6.1 結論 55
6.2 未來展望 55
參考文獻 56

1. 曾鈺涓(2010)。當代數位互動藝術之特質。國立交通大學，應用藝術研究所。
2. 楊景亮, et al. (2020)。基於 Attention-LSTM 的有載調容變壓器運行方式優化研究。 電網技術， 44(7)，2449-2456。
3. 21c Museum Hotel(2000). Text Rain. Camille Utterback website. Retrieved December,10,2021, from http://camilleutterback.com/projects/text-rain/
4. Bonnie Eisenman(2020). Is Processing Java? Bonnie Eisenman website. Retrieved December,10,2021, from
https://blog.bonnieeisenman.com/blog/what-is-processing/
5. Barron, J. L., Fleet, D. J., & Beauchemin, S. S. (1994). Performance of optical flow techniques. International journal of computer vision, 12(1), 43-77.
6. Baker, S., Scharstein, D., Lewis, J. P., Roth, S., Black, M. J., & Szeliski, R. (2011). A database and evaluation methodology for optical flow. International journal of computer vision, 92(1), 1-31.
7. Baradel, F., Wolf, C., & Mille, J. (2017). Human action recognition: Pose-based attention draws focus to hands. In Proceedings of the IEEE International Conference on Computer Vision Workshops (pp. 604-613).
8. Cao, Z., Simon, T., Wei, S. E., & Sheikh, Y. (2017). Realtime multi-person 2d pose estimation using part affinity fields. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7291-7299).
9. Cao, Z., Hidalgo, G., Simon, T., Wei, S. E., & Sheikh, Y. (2019). OpenPose: realtime multi-person 2D pose estimation using Part Affinity Fields. IEEE transactions on pattern analysis and machine intelligence, 43(1), 172-186.
10. Deng, X., Yang, S., Zhang, Y., Tan, P., Chang, L., & Wang, H. (2017). Hand3d: Hand pose estimation using 3d neural network. arXiv preprint arXiv:1704.02224.
11. Fan, Z., Zhao, X., Lin, T., & Su, H. (2018). Attention-based multiview re-observation fusion network for skeletal action recognition. IEEE Transactions on Multimedia, 21(2), 363-374.
12. Horn, B. K., & Schunck, B. G. (1981). Determining optical flow. Artificial intelligence, 17(1-3), 185-203.

13. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).
14. Kim, H. I., & Park, R. H. (2018). Residual LSTM attention network for object tracking. IEEE Signal Processing Letters, 25(7), 1029-1033.
15. Li, J., Zhang, S., & Huang, T. (2019, July). Multi-scale 3d convolution network for video based person re-identification. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 33, No. 01, pp. 8618-8625).
16. Liu, J., Wang, G., Hu, P., Duan, L. Y., & Kot, A. C. (2017). Global context-aware attention lstm networks for 3d action recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1647-1656).
17. Luvizon, D. C., Picard, D., & Tabia, H. (2018). 2d/3d pose estimation and action recognition using multitask deep learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 5137-5146).
18. Park, S., Hwang, J., & Kwak, N. (2016, October). 3d human pose estimation using convolutional neural networks with 2d pose information. In European Conference on Computer Vision (pp. 156-169). Springer, Cham.
19. Pavllo, D., Feichtenhofer, C., Grangier, D., & Auli, M. (2019). 3d human pose estimation in video with temporal convolutions and semi-supervised training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 7753-7762).
20. Piper, Andrew(2012).Book Was There: Reading in Electronic Times. Chicago, IL and London: The University of Chicago Press
21. Song, S., Lan, C., Xing, J., Zeng, W., & Liu, J. (2018). Spatio-temporal attention-based LSTM networks for 3D action recognition and detection. IEEE Transactions on image processing, 27(7), 3459-3471.
22. Wang, Y., Huang, M., Zhu, X., & Zhao, L. (2016, November). Attention-based LSTM for aspect-level sentiment classification. In Proceedings of the 2016 conference on empirical methods in natural language processing (pp. 606-615).
23. Xiu, Y., Li, J., Wang, H., Fang, Y., & Lu, C. (2018). Pose Flow: Efficient online pose tracking. arXiv preprint arXiv:1802.00977.
24. Xiao, B., Wu, H., & Wei, Y. (2018). Simple baselines for human pose estimation and tracking. In Proceedings of the European conference on computer vision (ECCV) (pp. 466-481).
25. Xie, Y., Liang, R., Liang, Z., Huang, C., Zou, C., & Schuller, B. (2019). Speech emotion classification using attention-based LSTM. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 27(11), 1675-1685.