臺灣博碩士論文加值系統

穿戴式裝置中的動作辨識主要透過萃取三軸加速度器和陀螺儀中的資料特徵進行動作分類。然而在資料蒐集過程中，傳統標籤方法多半透過影像資訊再經由人工進行比對標記，其所耗費的時間和成本皆非常地高。同時，感測器蒐集而來的訊息容易存在大量的雜訊，進而影響特徵工程的效能與效率。因此，本研究利用半監督式機率生成模型結合深度神經網路將資料進行動作分類；該方法可以使分類模型在有限的標籤資料下，仍然達到良好的分類水準。在特徵提取階段，深度神經網路中的變分自動編碼器(Variational Autoencoder, VAE)產生原始資料降維後之隱性特徵，並利用該隱性特徵來輔助深度神經網路模型的分類。對於未標籤資料，將隱性特徵與未知動作標籤視為兩組機率圖模型中的隱含變數；對於已標籤資料，將隱性特徵與動作標籤之先驗機率分配視為兩組機率圖模型中的隱含變數。接者利用變分推估(Variational Inference)來近似貝氏統計中的後驗機率，透過迭代逐步調升目標式的變分下界(Evidence of Lower Bound, ELBO)來達到期望值最大化(Expectation-Maximization)。本研究實驗三組公開資料集，實驗結果顯示20%以上的標籤樣本下，該方法即達到相當程度競爭性的分類水準。

Activity recognition in wearable devices is mainly classified by extracting data features from triaxial accelerometers and gyroscopes. However, during the process of data collection, the traditional labeling method is typically based on an observation of video-based information and manually corrected, which is very time-consuming and costly. At the same time, human-labeled results are prone to a large amount of noise, which might affect the performance and efficiency of the feature engineering. Therefore, this thesis uses a semi-supervised probabilistic generative model combined with a deep neural network to classify the sensor data into actions. This method can still make the classification model a reliable classifier under conditions with limited labeled data. In the feature extraction stage, the Variational Autoencoder (VAE) in the deep neural network generates the latent features of the original data after dimension reduction and uses the latent features to assist the classification of the deep neural network model. For unlabeled data, the latent features and the unknown action label are regarded as implicit variables in the two probabilistic models; for the labeled data, the prior probability distribution of the latent feature and the action label are regarded as two sets of probabilistic models. Then uses the Variational Inference to approximate the posterior probability in Bayesian statistics, and gradually increases the target's Evidence of Lower Bound (ELBO) to reach Expectation-Maximization. In this study, three groups of public datasets were assessed. The experimental results show that the accuracy of this study is competitive to that of other models using datasets with 20% of the label samples or more.

摘要i
Abstract ii
誌謝iv
Content v
List of Tables viii
List of Figures ix
1 Introduction 1
1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.4 Thesis Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2 Related Works 7
2.1 Activity Recognition with Wearable Device . . . . . . . . . . . . . . . 7
2.1.1 Conventional Feature Extraction . . . . . . . . . . . . . . . . . 7
2.1.2 Feature Extraction by Deep Neural Network . . . . . . . . . . . 9
2.2 Semi-supervised Learning Algorithm . . . . . . . . . . . . . . . . . . . 10
2.2.1 Variational Inference of Latent Feature . . . . . . . . . . . . . 12
2.2.2 SSL with Variational Autoencoder . . . . . . . . . . . . . . . . 13
2.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3 Methodology 15
3.1 Frameworks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.2 Data Pre-processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.3 Semi-supervised VAE in Activity Classification . . . . . . . . . . . . . 19
3.3.1 Latent Feature Discriminative Model . . . . . . . . . . . . . . 19
3.3.2 Generative Semi-supervised Model . . . . . . . . . . . . . . . 21
3.3.3 Training Model . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4 Experiment 27
4.1 Experimental Architecture . . . . . . . . . . . . . . . . . . . . . . . . 27
4.1.1 Dataset Description . . . . . . . . . . . . . . . . . . . . . . . . 27
4.1.2 Experimental Procedure . . . . . . . . . . . . . . . . . . . . . 29
4.1.3 Experimental Environment . . . . . . . . . . . . . . . . . . . . 30
4.2 Data Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.2.1 Kalman Filter Denoise . . . . . . . . . . . . . . . . . . . . . . 30
4.3 Deep Learning with SSLVAE . . . . . . . . . . . . . . . . . . . . . . . 31
4.3.1 Evaluation Tool . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.3.2 Parameter Tuning . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.3.3 Robustness Assessment . . . . . . . . . . . . . . . . . . . . . 47
4.4 The Comparison with Other Methods . . . . . . . . . . . . . . . . . . 49
4.4.1 Comparison Result . . . . . . . . . . . . . . . . . . . . . . . . 49
4.4.2 Robustness Comparison . . . . . . . . . . . . . . . . . . . . . 53
4.4.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
5 Conclusion and Future Work 56
5.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
5.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
References 58

Alsheikh, M. A., Selim, A., Niyato, D., Doyle, L., Lin, S., and Tan, H.-P. (2016). Deep activity recognition models with triaxial accelerometers. In AAAI Workshop: Artificial Intelligence Applied to Assistive Technologies and Smart Environments.

Anguita, D., Ghio, A., Oneto, L., Parra, X., and Reyes-Ortiz, J. L. (2013). A public domain dataset for human activity recognition using smartphones. In Esann.

Arcidiacono, C., Porto, S., Mancino, M., and Cascone, G. (2017). Development of a threshold-based classifier for real-time recognition of cow feeding and standing behavioural activities from accelerometer data. Computers and Electronics in Agriculture, 134:124–134.

Belkin, M., Niyogi, P., and Sindhwani, V. (2006). Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research, 7(Nov):2399–2434.

Blum, A. and Mitchell, T. (1998). Combining labeled and unlabeled data with cotraining. In Proceedings of the Eleventh Annual Conference on Computational Learning Theory, pages 92–100. ACM.

Butepage, J., Kjellstrom, H., and Kragic, D. (2019). A probabilistic semi-supervised approach to multi-task human activity modeling.

Chernbumroong, S., Cang, S., and Yu, H. (2015). Genetic algorithm-based classifiers fusion for multisensor activity recognition of elderly people. IEEE Journal of Biomedical and Health Informatics, 19(1):282–289.

Diosdado, J. A. V., Barker, Z. E., Hodges, H. R., Amory, J. R., Croft, D. P., Bell, N. J., and Codling, E. A. (2015). Classification of behaviour in housed dairy cows using an accelerometer-based activity monitoring system. Animal Biotelemetry, 3(1):15.

Figo, D., Diniz, P. C., Ferreira, D. R., and Cardoso, J. M. (2010). Preprocessing techniques for context recognition from accelerometer data. Personal and Ubiquitous Computing, 14(7):645–662.

Gani, M. O., Fayezeen, T., Povinelli, R. J., Smith, R. O., Arif, M., Kattan, A. J., and Ahamed, S. I. (2019). A light weight smartphone based human activity recognition system with high accuracy. Journal of Network and Computer Applications, 141:59– 72.

Ghazvininejad, M., Rabiee, H. R., Pourdamghani, N., and Khanipour, P. (2011). Hmm based semi-supervised learning for activity recognition. In Proceedings of the 2011 International Workshop on Situation Activity ＆ Goal Awareness, pages 95–100. ACM.

Guo, H., Chen, L., Chen, G., and Lv, M. (2016). Smartphone-based activity recognition independent of device orientation and placement. International Journal of Communication Systems, 29(16):2403–2415.

He, J., Zhang, Z., Wang, X., and Yang, S. (2019). A low power fall sensing technology based on fd-cnn. IEEE Sensors Journal, 19(13):5110–5118.

Kalman, R. E. (1960). A new approach to linear filtering and prediction problems. Journal of Basic Engineering, 82(1):35–45.

Kamminga, J. W., Bisby, H. C., Le, D. V., Meratnia, N., and Havinga, P. J. (2017). Generic online animal activity recognition on collar tags. In Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers, pages 597–606. ACM.

Kautz, T., Groh, B. H., Hannink, J., Jensen, U., Strubberg, H., and Eskofier, B. M. (2017). Activity recognition in beach volleyball using a deep convolutional neural network. Data Mining and Knowledge Discovery, 31(6):1678–1705.

Kingma, D. P. and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.

Kingma, D. P., Mohamed, S., Rezende, D. J., andWelling, M. (2014). Semi-supervised learning with deep generative models. In Advances in Neural Information Processing Systems, pages 3581–3589.

Kulikov, G. Y. and Kulikova, M. V. (2016). The accurate continuous-discrete extended kalman filter for radar tracking. IEEE Transaction on Signal Processing, 64(4):948– 958.

LeCun, Y., Bengio, Y., and Hinton, G. (2015). Deep learning. Nature, 521(7553):436.

Li, T., Chen, S., Yao, Z., Chen, X., and Yang, J. (2018). Semi-supervised network traffic classification using deep generative models. In 2018 14th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD), pages 1282–1288. IEEE.

Lv, M., Chen, L., Chen, T., and Chen, G. (2018). Bi-view semi-supervised learning based semantic human activity recognition using accelerometers. IEEE Transactions on Mobile Computing.

Martiskainen, P., J¨arvinen, M., Sk¨on, J.-P., Tiirikainen, J., Kolehmainen, M., and Mononen, J. (2009). Cow behaviour pattern recognition using a three-dimensional accelerometer and support vector machines. Applied Animal Behaviour Science, 119(1-2):32–38.

Mohammadi, M., Al-Fuqaha, A., Guizani, M., and Oh, J.-S. (2017). Semisupervised deep reinforcement learning in support of iot and smart city services. IEEE Internet of Things Journal, 5(2):624–635.

Nweke, H. F., Teh, Y. W., Al-Garadi, M. A., and Alo, U. R. (2018). Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: State of the art and research challenges. Expert Systems with Applications.

Nweke, H. F., Teh, Y. W., Mujtaba, G., and Al-garadi, M. A. (2019). Data fusion and multiple classifier systems for human activity detection and health monitoring: Review and open research directions. Information Fusion, 46:147–170.

Ord´o˜nez, F. J. and Roggen, D. (2016). Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors, 16(1):115.

Rashid, K. M. and Louis, J. (2019). Times-series data augmentation and deep learning for construction equipment activity recognition. Advanced Engineering Informatics, 42:100944.

Rasmus, A., Berglund, M., Honkala, M., Valpola, H., and Raiko, T. (2015). Semisupervised learning with ladder networks. In Advances in Neural Information Processing Systems, pages 3546–3554.

Ravi, D., Wong, C., Lo, B., and Yang, G.-Z. (2017). A deep learning approach to on node sensor data analytics for mobile or wearable devices. IEEE Journal of Biomedical and Health Informatics, 21(1):56–64.

Robnik-ˇ Sikonja, M. and Kononenko, I. (2003). Theoretical and empirical analysis of relieff and rrelieff. Machine Learning, 53(1-2):23–69.

Ronao, C. A. and Cho, S.-B. (2016). Human activity recognition with smartphone sensors using deep learning neural networks. Expert Systems with Applications, 59:235– 244.

Shen, C., Chen, Y., Yang, G., and Guan, X. (2018). Toward hand-dominated activity recognition systems with wristband-interaction behavior analysis. IEEE Transactions on Systems, Man, and Cybernetics: Systems.

Stikic, M., Van Laerhoven, K., and Schiele, B. (2008). Exploring semi-supervised and active learning for activity recognition. In 2008 12th IEEE International Symposium on Wearable Computers, pages 81–88. IEEE.

Stisen, A., Blunck, H., Bhattacharya, S., Prentow, T. S., Kjærgaard, M. B., Dey, A., Sonne, T., and Jensen, M. M. (2015). Smart devices are different: Assessing and mitigating mobile sensing heterogeneities for activity recognition. In Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems, pages 127–140. ACM.

Sucerquia, A., L´opez, J., and Vargas-Bonilla, J. (2017). Sisfall: A fall and movement dataset. Sensors, 17(1):198.

Tanaka, D., Ikami, D., Yamasaki, T., and Aizawa, K. (2018). Joint optimization framework for learning with noisy labels. arXiv preprint arXiv:1803.11364.

Vahdat, A. (2017). Toward robustness against label noise in training deep discriminative neural networks. In Advances in Neural Information Processing Systems, pages 5596–5605.

Valpola, H. (2015). From neural pca to deep unsupervised learning. In Advances in Independent Component Analysis and Learning Machines, pages 143–171. Elsevier.

Vilarinho, T., Farshchian, B., Bajer, D. G., Dahl, O. H., Egge, I., Hegdal, S. S., Lønes, A., Slettevold, J. N., and Weggersen, S. M. (2015). A combined smartphone and smartwatch fall detection system. In 2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing, pages 1443–1448. IEEE.

Wang, A., Chen, G., Yang, J., Zhao, S., and Chang, C.-Y. (2016). A comparative study on human activity recognition using inertial sensors in a smartphone. IEEE Sensors Journal, 16(11):4566–4578.

Wang, B. and Tsotsos, J. (2016). Dynamic label propagation for semi-supervised multiclass multi-label classification. Pattern Recognition, 52:75–84.

Wang, J., He, Z., Zheng, G., Gao, S., and Zhao, K. (2018). Development and validation of an ensemble classifier for real-time recognition of cow behavior patterns from accelerometer data and location data. PloS One, 13(9):e0203546.

Xiao, T., Xia, T., Yang, Y., Huang, C., and Wang, X. (2015). Learning from massive noisy labeled data for image classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2691–2699.

Xu, W., Sun, H., Deng, C., and Tan, Y. (2017). Variational autoencoder for semisupervised text classification. In Thirty-First AAAI Conference on Artificial Intelligence.

Zeng, M., Nguyen, L. T., Yu, B., Mengshoel, O. J., Zhu, J., Wu, P., and Zhang, J. (2014). Convolutional neural networks for human activity recognition using mobile sensors. In IEEE Mobile Computing, Applications and Services (MobiCASE), 2014 6th International Conference on, pages 197–205. IEEE.

Zhang, C., Bengio, S., Hardt, M., Recht, B., and Vinyals, O. (2016). Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530.

Zhang, Y., Wen, J., Wang, X., and Jiang, Z. (2014). Semi-supervised learning combining co-training with active learning. Expert Systems with Applications, 41(5):2372–2378.

Zhou, Z.-H. and Li, M. (2010). Semi-supervised learning by disagreement. Knowledge and Information Systems, 24(3):415–439.

Zhu, C. and Sheng, W. (2011). Wearable sensor-based hand gesture and daily activity recognition for robot-assisted living. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 41(3):569–573.