臺灣博碩士論文加值系統

本文的目的是提出一種社交型協作機器人應用情境的涵意,學習並預測人類 的想法,進而提供「剛剛好的服務」。為了在人類社交環境中與人友善的互動, 機器人應具有情境感知瞭解人類社交技巧的能力並且表現出得體的行為。
在本文中,情境式上下文著重在讓機器人感知他人是否需要幫助,根據預測 的人類想法,機器人提供剛剛好的服務。剛剛好的概念來自鼎泰豐餐廳的董事長, 他說:「服務不足,是怠慢;殷勤過頭,變成打擾,『剛剛好的服務』是鼎泰豐 團隊努力追求的目標」。在服務業方面,當顧客需要幫助時,服務員主動提供服 務是非常暖心的。換句話說,當顧客不需要幫助時,不去打擾他們是很體貼的。
我們提出兩個深度學習模型,作為機器人的情境式上下文感知,並從人機互 動中觀察並學習判斷人類的意圖。基於深度學習模型,我們賦予機器人感知人的 意向的能力。因此,機器人可以基於預測的人類心理狀態,做出適當的社交行為。 實驗結果表明,與常規分類器相比,我們提出的深度學習模型可以使機器人顯著 提高預測人類思維的準確性。此外,在判斷人是否需要幫忙的任務上,基於情境 式上下文的預測結果與服務業人士的意見保持高度一致。

The objective of this thesis is to develop a social co-robot for provision of “just-good services” using situational context perception for learning and predicting human’s mentation. To interact with humans in Human Social Environments (HSEs), robots are expected to possess the ability of situational context perception and behave appropriately.
In this paper, we employ the concept of situational context to our work, which mainly focus on making robots perceive others’ needing assistance and provide “just- good service”. The just-good concept is stem from the owner of Din Tai Fung restaurant, and he says: Inadequate service is neglecting; too diligent become disturbing, just-good
service is the goal Ding Tai Fung team pursue.” In service industry, it is indeed friendly to help others as they need. In other words, it is actually considerate not to bother others when they don’t need help.
We propose two deep learning models, as situational context perception of robot, to learn from observations of human-robot interaction. Based on these models, we endow robot the capability of perceiving human’s mentation. Thus, the appropriate social behaviors can be performed by the robot with respect to human’s mental state. The experimental results demonstrate that robot can significantly improve the accuracy of predicting a person’s mentation through the proposed deep learning models by comparison with conventional classifiers. Furthermore, the prediction of our situational context perception keep highly consistent with the opinion made by people who work in service industry.

TABLE OF CONTENTS
誌謝 I
中文摘要 II
Abstract III
TABLE OF CONTENTS IV
LIST OF FIGURES VI
LIST OF TABLE VIII
Chapter 1 INTRODUCTION 1
1.1 MOTIVATION 1
1.2 OBJECTIVE 3
1.3 LITERATURE REVIEW 4
1.4 THESIS STRUCTURE 5
Chapter 2 SYSTEM STRUCTURE 6
2.1 HARDWARE STRUCTURE 6
2.1.1 RenBo-S Service Robot 6
2.1.2 Kinect RGB-D Camera 7
2.2 SOFTWARE STRUCTURE 9
2.2.1 Point Cloud Library (PCL) 9
2.2.2 Open Source Computer Vision Library (OpenCV) 10
2.2.3 Scikit-Learn 12
2.2.4 KERAS 14
2.2.5 API.AI 16
2.2.6 Robot Operating System (ROS) 18
Chapter 3 BACKGROUND and INITIAL WORK 23
3.1 UNDERSTANDING and USING CONTEXT 23
3.1.1 Definition of Context 24
3.1.2 Definition of Context-Aware 24
3.2 JUST-GOOD SERVICES and ROBOT’S APPROPRIATE BEHAVIORS 25
3.2.1 Definition of Just-Good 25
3.2.2 Robot’s Appropriate Behaviors 26
3.3 INITIAL WORK 27
3.3.1 Data Collection 27
Chapter 4 SITUATIONAL CONTEXT PERCEPTION for JUST-GOOD SERVICES 29
4.1 DEFINITION OF SITUATIONAL CONTEXT PERCEPTION 29
4.2 ANALYSIS and TRAINING METHDOLOGY 30
4.3 FEATURE EXTRACTION 31
4.3.1 Handcraft Feature 31
4.3.2 Convolutional Neural Networks Auto-encoder 33
4.4 CLASSIFIER IMPLEMENTATION 50
4.4.1 Deep Learning Based Classifiers 50
4.4.2 Conventional Classifiers 55
4.5 SOCIAL CO-ROBOT VERSUS PEOPLE in SERVICE INDUSTRY 56
Chapter 5 EXPERIMENTAL RESULTS 58
5.1 DEEP LEARNING MODELS EVALUATION 58
5.1.1 K-fold Cross-Validation 58
5.1.2 Features Comparison 58
5.1.3 Classifier Appropriateness 60
5.1.4 Multi-feature Fusion 62
5.1.5 Deep Learning Models Comparison 63
5.2 SITUATIONAL CONTEXT PERCEPTION EVALUATION 64
5.2.1 Results and Discussion 66
Chapter 6 CONCLUSION, CONTRIBUTIONS and FUTURE WORKS 69
6.1 CONCLUSIONS 69
6.2 CONTRIBUTIONS 70
6.3 FUTURE WORKS 70
References 71
VITA 76

[1]T. Pire, T. Fischer, J. Civera, P. De Cristforis and J. J. Berlles, “Stereo parallel tracking and mapping for robot localization,” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, 2015, pp. 1373-1378.

[2]K. Qiu, F. Zhang and M. Liu, “Visible Light Communication-based indoor localization using Gaussian Process,” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, 2015, pp. 3125-3130.

[3]R. C. Luo, V. W. S. Ee and C. K. Hsieh, “3D point cloud based indoor mobile robot in 6-DoF pose localization using Fast Scene Recognition and Alignment approach,” IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), Baden-Baden, Germany, 2016, pp. 470-475.

[4]H. Kikkeri, G. Parent, M. Jalobeanu and S. Birchfield, “An inexpensive method for evaluating the localization performance of a mobile robot navigation system,” IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, 2014, pp. 4100-4107.

[5]R. L. Riek, “The Social Co-Robotics Problem Space: Six Key Challenges,” Robotics Challenges and Vision (RCV2013), 2014.

[6]C. R. Raymundo, C. G. Johnson and P. A. Vargas, “An architecture for emotional and context-aware associative learning for robot companions,” IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Kobe, 2015, pp. 31-36.

[7]P. Lison, C. Ehrler and G. J. M. Kruijff, “Belief modelling for situation awareness in human-robot interaction,” International Symposium in Robot and Human Interactive Communication, Viareggio, 2010, pp. 138- 143.

[8]S. H. Tseng, J. H. Hua, S. P. Ma and L. e. Fu, “Human awareness based robot performance learning in a social environment,” IEEE International Conference on Robotics and Automation, Karlsruhe, 2013, pp. 4291- 4296.

[9]Situational Context, https://www.alleydog.com/glossary/psychology-glossary.php [Online; accessed 1-March-2017]
[10]Nigam and L. D. Riek, “Social context perception for mobile robots,”
IEEE/RSJ International Conference on Intelligent Robots and Systems
(IROS), Hamburg, 2015, pp. 3621-3627.

[11]H. Qureshi, Y. Nakamura, Y. Yoshikawa and H. Ishiguro, “Robot
gains social intelligence through multimodal deep reinforcement learning,” IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids), Cancun, 2016, pp. 745-751.

[12]Aly and A. Tapus, “A model for synthesizing a combined verbal and nonverbal behavior based on personality traits in human-robot interaction,” ACM/IEEE International Conference on Human-Robot Interaction (HRI), Tokyo, 2013, pp. 325-332.

[13]J.Mumm and B.Mutlu, “Human-robotproxemics : Physical and psychological distancing in human-robot interaction,” ACM/IEEE International Conference on Human-Robot Interaction (HRI), Lausanne, 2011, pp. 331-338.

[14]T. Kitade, S. Satake, T. Kanda and M. Imai, “Understanding suitable locations for waiting,” ACM/IEEE International Conference on Human- Robot Interaction (HRI), Tokyo, 2013, pp. 57-64.

[15]PCL, http://pointclouds.org/ [Online; accessed 15-July-2017]
[16]OpenCV, http://opencv.org/ [Online; accessed 15-July-2017]
[17]Scikit-Learn, http://scikit-learn.org/stable/ [Online; accessed 15-July-2017]
[18]Keras, https://keras.io/ [Online; accessed 15-July-2017]
[19]API.AI, https://api.ai/ [Online; accessed 15-July-2017]
[20]ROS, http://www.ros.org/ [Online; accessed 15-July-2017]
[21]N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR''05), San Diego, CA, USA, 2005, pp. 886-893 vol. 1.

[22]Optical Flow, http://docs.opencv.org/trunk/d7/d8b/tutorial_py_lucas_kanade.html [Online; accessed 15-July-2017]
[23]Autoencoders, https://blog.keras.io/building-autoencoders-in-keras.html [Online; accessed 15-July-2017]
[24]Category classification by CNNs, https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/ [Online; accessed 15-July-2017]
[25]Activation function, https://en.wikibooks.org/wiki/Artificial_Neural_Networks/-Print_Version [Online; accessed 15-July-2017]
[26]Gradient descent, http://sebastianruder.com/optimizing-gradient-descent/ [Online; accessed 15-July-2017]
[27]Early stopping, https://deeplearning4j.org/earlystopping [Online; accessed 15-July-2017]
[28]K. Sasaki, H. Tjandra, K. Noda, K. Takahashi and T. Ogata, “Neural network based model for visual-motor integration learning of robot’s drawing behavior: Association of a drawing motion from a drawn image,” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, 2015, pp. 2736-2741.
[29]V. Veeriah, N. Zhuang and G. J. Qi, “Differential Recurrent Neural Networks for Action Recognition,” IEEE International Conference on Computer Vision (ICCV), Santiago, 2015, pp. 4041-4049.
[30]Q.Li,X.ZhaoandK.Huang,“Learningtemporallycorrelatedrepresentations using lstms for visual tracking,” IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA, 2016, pp. 1614-1618.
[31]Y. Bengio, P. Simard and P. Frasconi, “Learning long-term dependencies with gradient descent is difficult,” IEEE Transactions on Neural Networks, vol. 5, no. 2, pp. 157-166, 1994.
[32]S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation, vol. 9, no. 8, pp. 1735-1780, 1997.
[33]SVM, http://diggdata.in/post/94066544971/support-vector-machine-without-tears [Online; accessed 15-July-2017]
[34]X. Yuan, L. Cai-nian, X. Xiao-liang, J. Mei and Z. Jian-guo, “A two- stage hog feature extraction processor embedded with SVM for pedes- trian detection,” IEEE International Conference on Image Processing (ICIP), Quebec City, QC, 2015, pp. 3452-3455.
[35]Y. Benabbas, N. Ihaddadene, T. Yahiaoui, T. Urruty and C. Djeraba, “Spatio-Temporal Optical Flow Analysis for People Counting,” IEEE International Conference on Advanced Video and Signal Based Surveil- lance, Boston, MA, 2010, pp. 212-217.
[36]J. Ba and D. Kingma, “Adam: a Method for Stochastic Optimization,” In- ternational Conference on Learning Representations, San Diego, 2015.