臺灣博碩士論文加值系統

行走輔助機器人是幫助老年人或殘疾人安全、穩定、高效地移動的輔助設備。與傳統助行器提供的步行和復健功能相比，主動式輔助行走機器人爲特殊的使用者提供了不同程度的人工智慧以提升使用者的操控性和舒適度。
在本文中我們提出了一種多模態界面的基於學習的共享控制系統。該多模態介面包括利用步態分析所得特徵的認知人機互動介面和傳統的測量使用者施力狀態的物理互動介面。該介面通過一種新穎的傳感器資訊融合方法來預測使用者意圖，即一種結合模糊決策與具有長短期記憶的神經元網絡的架構：(一)深度相機用來估計使用者下肢運動和推斷使用者偏離機器人的速度方向的傾向；(二)力傳感器用來測量使用者手部作用于助行器扶手上的力。在此基礎上，考慮到機器人需具有自主適應不同使用者的操作習慣和運動能力的能力，我們提出了基於強化學習的共享控制算法，使機器人可以根據觀察到的具有個體差異性的控制效率及步行環境以動態地調整使用者控制的權重，從而提供適當的輔助以提高使用者在使用設備時的舒適度，以達到自動適應使用者行爲的效果。最後，通過仿真實驗和真實環境下的真人實驗，我們驗證了該算法在特定環境下的有效性。

Walking-aid robot is developed as an assistance device for enabling safe, stable and efficient locomotion for elderly or disabled individuals, which, furthermore, provides different levels of intelligence as additional aids to suit certain applications, such as navigation and rehabilitation function.
In this thesis, a learning-based shared control system with multimodal interface is proposed, containing both cognitive human-robot interaction (HRI) for gait analysis and traditional physical HRI for measuring user''s exerted force. The interface extracts navigation intentions from a novel neural network based method, which combines features from: (i) a depth camera to estimate the user legs'' kinematics and to infer user orientation deviating from robot''s velocity direction, and (ii) force sensors to measure the physical interaction between the human''s hands and the robotic walker. Then, considering the robot''s ability to autonomously adapt to different user''s operation preference and motor abilities, a reinforcement learning-based shared control algorithm is proposed. By dynamically adjusting the user control weight according to different user''s control efficiencies and walking environments, the robot not only can improve the user''s degree of comfort while using the device but also can automatically adapt to user''s behavior. Finally, the effectiveness of our algorithm is verified by simulation and experiments in a specified environment.

口試委員會審定書 i
誌謝 ii
摘要 iii
Abstract iv
Contents v
List of Figures vii
List of Tables x
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2.1 Mobility Assistance Robot . . . . . . . . . . . . . . . . . . . . . 3
1.2.2 Shared Control Strategies for Walking Assistance Robot . . . . . 5
1.3 Objective and Contribution . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2 Preliminaries 11
2.1 Previous Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.1.1 Hardware Structure . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.1.2 Depth Image based Gait Analysis . . . . . . . . . . . . . . . . . 18
2.2 Learning Schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.2.1 LSTM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21
2.2.2 Fuzzy Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.3 Reinforcement Learning . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.3.1 Standard Modeling . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.3.2 Markov Decision Processes . . . . . . . . . . . . . . . . . . . . 30
2.3.3 Value Functions and Action-Value Functions . . . . . . . . . . . 32
2.3.4 Q-Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.3.5 Actor-Critic Algorithms . . . . . . . . . . . . . . . . . . . . . . 35
3 Learning-based Shared Control 37
3.1 Human-Walker Interaction . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.1.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . 38
3.1.2 Multimodal Interface . . . . . . . . . . . . . . . . . . . . . . . . 43
3.2 Shared Control System with Hybrid Learning Schemes . . . . . . . . . . 46
3.2.1 System Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.2.2 Fused fuzzy LSTM for Intent Inference . . . . . . . . . . . . . . 48
3.2.3 Reinforcement Learning for Provision of Assistance . . . . . . . 51
4 Experimental Verification and Evaluation 55
4.1 Control Strategy Verification . . . . . . . . . . . . . . . . . . . . . . . . 55
4.1.1 Human Intent Detection from Gait Cadence . . . . . . . . . . . . 56
4.1.2 Human Maneuverability Evaluation . . . . . . . . . . . . . . . . 59
4.1.3 Controller Verification . . . . . . . . . . . . . . . . . . . . . . . 59
4.1.4 Stability Margin Monitoring . . . . . . . . . . . . . . . . . . . . 63
4.2 Scenario Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.2.1 User Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . .67
5 Conclusion 68
References 70

[1] World Health Organization. Neurological disorders: public health challenges. World Health Organization, 2006.
[2] Patrik Andlin-Sobocki, Bengt Jönsson, Hans-Ulrich Wittchen, and Jes Olesen. Cost of disorders of the brain in europe. European Journal of neurology, 12(s1):1–27, 2005.
[3] World Health Organization et al. Neurological disorders affect millions globally: Who report. Brussels/Geneva: World Health Organization, 2007.
[4] Caroline M Tanner and Samuel M Goldman. Epidemiology of parkinson’s disease. Neurologic clinics, 14(2):317–335, 1996.
[5] Michael Macht, R Schwarz, and H Ellgring. Patterns of psychological problems in parkinson’s disease. Acta Neurologica Scandinavica, 111(2):95–101, 2005.
[6] JB Gao. Analysis of amplitude and frequency variations of essential and parkinsonian tremors. Medical and Biological Engineering and Computing, 42(3):345–349, 2004.
[7] Heidemarie Zach, Michiel F Dirkx, Jaco W Pasman, Bastiaan R Bloem, and Rick C Helmich. Cognitive stress reduces the effect of levodopa on parkinson’s resting tremor. CNS neuroscience & therapeutics, 23(3):209–215, 2017.
[8] ATP Richard Simpson PhD and Michael L Boninger. Clinical evaluation of guido robotic walker. Journal of rehabilitation research and development, 45(9):1281, 2008.
[9] Geunho Lee, Takanori Ohnuma, and Nak Young Chong. Design and control of jaist active robotic walker. Intelligent Service Robotics, 3(3):125–135, 2010.
[10] J Paulo, L Garrote, C Premebida, A Asvadi, D Almeida, A Lopes, and P Peixoto. An innovative robotic walker for mobility assistance and lower limbs rehabilitation. In Bioengineering (ENBENG), 2017 IEEE 5th Portuguese Meeting on, pages 1–4. IEEE, 2017.
[11] Spencer A Murray, Kevin H Ha, Clare Hartigan, and Michael Goldfarb. An assistive control approach for a lower-limb exoskeleton to facilitate recovery of walking following stroke. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 23(3):441–449, 2015.
[12] Geunho Lee, Takanori Ohnuma, Nak Young Chong, and Soon-Geul Lee. Walking intent-based movement control for jaist active robotic walker. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 44(5):665–672, 2014.
[13] Gerolf Vanacker, José del R Millán, Eileen Lew, Pierre W Ferrez, Ferran Galán Moles, Johan Philips, Hendrik Van Brussel, and Marnix Nuttin. Context-based filtering for assisted brain-actuated wheelchair driving. Computational intelligence and neuroscience, 2007:3–3, 2007.
[14] Wei-Hao Mou, Ming-Fang Chang, Chien-Ke Liao, Yuan-Han Hsu, Shih-Huan Tseng, and Li-Chen Fu. Context-aware assisted interactive robotic walker for parkinson’s disease patients. In Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on, pages 329–334. IEEE, 2012.
[15] Carlos A Cifuentes, Camilo Rodriguez, Anselmo Frizera-Neto, Teodiano Freire Bastos-Filho, and Ricardo Carelli. Multimodal human–robot interaction for walkerassisted gait. IEEE Systems Journal, 10(3):933–943, 2016.
[16] Virginia Ruiz Garate, Andrea Parri, Tingfang Yan, Marko Munih, Raffaele Molino Lova, Nicola Vitiello, and Renaud Ronsse. Walking assistance using artificial primitives: a novel bioinspired framework using motor primitives for locomotion assistance through a wearable cooperative exoskeleton. IEEE Robotics & Automation Magazine, 23(1):83–95, 2016.
[17] Andreas Wachaja, Pratik Agarwal, Mathias Zink, Miguel Reyes Adame, Knut Möller, and Wolfram Burgard. Navigating blind people with a smart walker. In Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on, pages 6014–6019. IEEE, 2015.
[18] Chung Dial Lim, Chia-Ming Wang, Ching-Ying Cheng, Yen Chao, Shih-Huan Tseng, and Li-Chen Fu. Sensory cues guided rehabilitation robotic walker realized by depth image-based gait analysis. IEEE Transactions on Automation Science and Engineering, 13(1):171–180, 2016.
[19] J d R Millán, Rüdiger Rupp, Gernot R Müller-Putz, Roderick Murray-Smith, Claudio Giugliemma, Michael Tangermann, Carmen Vidaurre, Febo Cincotti, Andrea Kübler, Robert Leeb, et al. Combining brain–computer interfaces and assistive technologies: state-of-the-art and challenges. Frontiers in neuroscience, 4, 2010.
[20] Gerard Lacey and Kenneth M Dawson-Howe. The application of robotics to a mobility aid for the elderly blind. Robotics and Autonomous Systems, 23(4):245–252, 1998.
[21] Jiwon Shin, Andrey Rusakov, and Bertrand Meyer. Smartwalker: An intelligent robotic walker. Journal of Ambient Intelligence and Smart Environments, 8(4):383–398, 2016.
[22] Marco Andreetto, Stefano Divan, Daniele Fontanelli, and Luigi Palopoli. Passive robotic walker path following with bang-bang hybrid control paradigm. In Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on, pages 1054–1060. IEEE, 2016.
[23] Skye McLachlan, Jaimal Arblaster, OK Liu, Jaime Valls Miro, and L Chenoweth. A multi-stage shared control method for an intelligent mobility assistant. In Rehabilitation Robotics, 2005. ICORR 2005. 9th International Conference on, pages 426–429. IEEE, 2005.
[24] Yanan Li, Keng Peng Tee, Wei Liang Chan, Rui Yan, Yuanwei Chua, and Dilip Kumar Limbu. Continuous role adaptation for human–robot shared control. IEEE Transactions on Robotics, 31(3):672–681, 2015.
[25] Milad Geravand, Christian Werner, Klaus Hauer, and Angelika Peer. An integrated decision making approach for adaptive shared control of mobility assistance robots. International Journal of Social Robotics, 8(5):631–648, 2016.
[26] Alexander Huntemann, Eric Demeester, Gerolf Vanacker, Dirk Vanhooydonck, Johan Philips, Hendrik Van Brussel, and Marnix Nuttin. Bayesian plan recognition and shared control under uncertainty: Assisting wheelchair drivers by tracking fine motion paths. In Intelligent Robots and Systems, 2007. IROS 2007. IEEE/RSJ International Conference on, pages 3360–3366. IEEE, 2007.
[27] Sin-Yi Jiang, Chen-Yang Lin, Ko-Tung Huang, and Kai-Tai Song. Shared control design of a walking-assistant robot. IEEE Transactions on Control Systems Technology, 2017.
[28] Cunjun Huang, Glenn Wasson, Majd Alwan, Pradip Sheth, and Alexandre Ledoux. Shared navigational control and user intent detection in an intelligent walker. In AAAI Fall 2005 Symposium, 2005.
[29] Milad Geravand and Angelika Peer. Safety constrained motion control of mobility assistive robots. In Biomedical Robotics and Biomechatronics (2014 5th IEEE RAS & EMBS International Conference on, pages 1073–1078. IEEE, 2014.
[30] Wenxia Xu, Jian Huang, Yongji Wang, Chunjing Tao, and Lei Cheng. Reinforcement learning-based shared control for walking-aid robot and its experimental verification. Advanced Robotics, 29(22):1463–1481, 2015.
[31] Chung Dial Lim, Ching-Ying Cheng, Chia-Ming Wang, Yen Chao, and Li-Chen Fu. Depth image based gait tracking and analysis via robotic walker. In Robotics and Automation (ICRA), 2015 IEEE International Conference on, pages 5916–5921. IEEE, 2015.
[32] Shuo Yang, Ping Luo, Chen Change Loy, Kenneth W Shum, Xiaoou Tang, et al. Deep representation learning with target coding. In AAAI, pages 3848–3854, 2015.
[33] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997.
[34] Yue Deng, Zhiquan Ren, Youyong Kong, Feng Bao, and Qionghai Dai. A hierarchical fused fuzzy deep neural network for data classification. IEEE Transactions on Fuzzy Systems, 2016.
[35] Jens Kober, J Andrew Bagnell, and Jan Peters. Reinforcement learning in robotics: A survey. The International Journal of Robotics Research, 32(11):1238–1274, 2013.
[36] Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction, volume 1. MIT press Cambridge, 1998.
[37] Leslie Pack Kaelbling, Michael L Littman, and Anthony R Cassandra. Planning and acting in partially observable stochastic domains. Artificial intelligence, 101(1):99–134, 1998.
[38] Martin L Puterman. Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons, 2014.
[39] Christopher John Cornish Hellaby Watkins. Learning from delayed rewards. PhD thesis, King’s College, Cambridge, 1989.
[40] Christopher JCH Watkins and Peter Dayan. Q-learning. Machine learning, 8(3-4):279–292, 1992.
[41] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. Nature, 518(7540):529–533, 2015.
[42] Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971, 2015.
[43] Maria Isabel Ribeiro and Pedro Lima. Kinematics models of mobile robots. Instituto de Sistemas e Robotica, pages 1000–1049, 2002.
[44] Jose L Pons. Wearable robots: biomechatronic exoskeletons. John Wiley & Sons,2008.
[45] Alexandre Alahi, Kratarth Goel, Vignesh Ramanathan, Alexandre Robicquet, Li Fei-Fei, and Silvio Savarese. Social lstm: Human trajectory prediction in crowded spaces. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 961–971, 2016.
[46] Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton. On the importance of initialization and momentum in deep learning. In International conference on machine learning, pages 1139–1147, 2013.
[47] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.