臺灣博碩士論文加值系統

In order to bring back a sense of connectedness and enhance social engagement for users, in a context of rising popularity of VR concert (due to rising concerns over large-scale gathering of people), we developed I See You See Me, a new kind of eye-gaze-based interaction allowing a user (audience) to exchange eye-contacts with a virtual singer during a virtual concert. With this implicit interaction, we aim to improve the users’ feeling of social presence, even in a context where they are interacting with a computer-controlled performer. To develop our interaction, we conducted two main user experiments, one to help us confirm the significant impact of eye-based interaction on the level of social presence, while helping us determine if factors such as Frequency, Duration, and Trigger (i.e., who can trigger the interaction) of gaze from the virtual singer could affect natural aspect of the interaction and consequently affect the level of social presence for users. Our second user study was designed to understand how we could strengthen the interaction to over-come challenging factors (e.g., distance, users’ eyesight limitations, display quality, etc.) with visual feedbacks. Results supported the effectiveness of the visual feedbacks in communicating the virtual singer’s attention towards the users (when it was difficult for them to observe with precision the singer’s gaze direction) but only to a certain extent, as they also emphasized their limitations (e.g., distracting).

In order to bring back a sense of connectedness and enhance social engagement for users, in a context of rising popularity of VR concert (due to rising concerns over large-scale gathering of people), we developed I See You See Me, a new kind of eye-gaze-based interaction allowing a user (audience) to exchange eye-contacts with a virtual singer during a virtual concert. With this implicit interaction, we aim to improve the users’ feeling of social presence, even in a context where they are interacting with a computer-controlled performer. To develop our interaction, we conducted two main user experiments, one to help us confirm the significant impact of eye-based interaction on the level of social presence, while helping us determine if factors such as Frequency, Duration, and Trigger (i.e., who can trigger the interaction) of gaze from the virtual singer could affect natural aspect of the interaction and consequently affect the level of social presence for users. Our second user study was designed to understand how we could strengthen the interaction to over-come challenging factors (e.g., distance, users’ eyesight limitations, display quality, etc.) with visual feedbacks. Results supported the effectiveness of the visual feedbacks in communicating the virtual singer’s attention towards the users (when it was difficult for them to observe with precision the singer’s gaze direction) but only to a certain extent, as they also emphasized their limitations (e.g., distracting).

ABSTRACT i
Acknowledgements iii
Table of Contents iv
List of Tables vi
List of Figures vii
Chapter 1 - INTRODUCTION 1
1.1 Background & Motivation 1
1.2 Research goal 5
1.3 Research scope 5
Chapter 2 - RELATED WORKS 7
2.1 Presence and engagement 7
2.1.1 Presence as physical presence and place presence 7
2.1.2 Social presence and co-presence 9
2.2 Interacting with virtual characters 15
2.2.1 Representation of virtual characters 16
2.2.2 Designing interaction with virtual characters 18
2.3 Eye-gaze-based interaction 19
2.3.1 Effects of eye-contact 20
2.3.2 Eye-tracking in HCI 22
Chapter 3 - APPROACH 27
3.1 Design considerations 27
3.2 Design proposal: I See You See Me 28
3.3 System implementation 30
Chapter 4 - EVALUATION 33
4.1 Pilot Study: Simulating Gaze 33
4.1.1 Experiment design 33
4.1.2 Tasks 38
4.1.3 Participants and procedure 41
4.1.4 Results 43
4.2 User Study 1: Simulating Gaze 50
4.2.1 Experiment design: Frequency, Duration and Trigger 50
4.2.2 Task 51
4.2.3 Participants and procedure 54
4.2.4 Results 56
4.3 User Study 2: Confirming Gaze 75
4.3.1 Experiment design: Visual Feedbacks 75
4.3.2 Task 78
4.3.3 Participants and procedure 78
4.3.4 Results 79
Chapter 5 - DISCUSSION 86
5.1 Simulating Gaze: Do’s and Don’ts 86
5.2 Confirming Gaze: Alternative to Visual Feedbacks 87
Chapter 6 - CONCLUSION 88
Chapter 7 - LIMITATIONS & FUTURE DIRECTIONS 89
7.1 Limitations 89
7.2 Future directions 91
References 94

[1] Hashizume, S., Sakamoto, S., Suzuki, K., & Ochiai, Y. (2018, July). Livejacket: Weara-ble music experience device with multiple speakers. In International Conference on Dis-tributed, Ambient, and Pervasive Interactions (pp. 359-371). Springer, Cham.
[2] Riva, G., et al. (2007). Affective interactions using virtual reality: the link between pres-ence and emotions. Cyberpsychology & behavior, 10(1), 45-56.
[3] Diemer, J., Alpers, G. W., Peperkorn, H. M., Shiban, Y., & Mühlberger, A. (2015). The impact of perception and presence on emotional reactions: a review of research in virtual reality. Frontiers in psychology, 6, 26.
[4] Minsky, M. (1980). Telepresence.
[5] Steuer, J. (1992). Defining virtual reality: Dimensions determining telepresence. Journal of communication, 42(4), 73-93.
[6] Sheridan, T. B. (1992). Musings on telepresence and virtual presence. Presence Tele-operators Virtual Environ., 1(1), 120-125.
[7] Slater, M., Usoh, M., & Steed, A. (1994). Depth of presence in virtual environments. Presence: Teleoperators & Virtual Environments, 3(2), 130-144.
[8] Heater, C. (1992). Being there: The subjective experience of presence. Presence Tele-operators Virtual Environ., 1(2), 262-271.
[9] Biocca, F. (1997). The cyborg's dilemma: Progressive embodiment in virtual environ-ments. Journal of computer-mediated communication, 3(2), JCMC324.
[10] Biocca, F., Burgoon, J., Harms, C., & Stoner, M. (2001). Criteria and scope conditions for a theory and measure of social presence. Presence: Teleoperators and virtual environ-ments.
[11] Lee, K. M. (2004). Presence, explicated. Communication theory, 14(1), 27-50.
[12] Short, J., Williams, E., & Christie, B. (1976). The social psychology of telecommunica-tions. Toronto; London; New York: Wiley.
[13] Argyle, M., & Dean, J. (1965). Eye-contact, distance and affiliation. Sociometry, 289-304.
[14] Wiener, M., & Mehrabian, A. (1968). Language within language: Immediacy, a channel in verbal communication. Ardent Media.
[15] Gunawardena, C. N. (1995). Social presence theory and implications for interaction and collaborative learning in computer conferences. International journal of educational tele-communications, 1(2), 147-166.
[16] Garrison, D. R. (2009). Communities of inquiry in online learning. In Encyclopedia of distance learning, Second edition (pp. 352-355). IGI Global.
[17] Nowak, K. L., & Biocca, F. (2003). The effect of the agency and anthropomorphism on users' sense of telepresence, copresence, and social presence in virtual environments. Pres-ence: Teleoperators & Virtual Environments, 12(5), 481-494.
[18] Biocca, F., Harms, C., & Gregg, J. (2001, May). The networked minds measure of social presence: Pilot test of the factor structure and concurrent validity. In 4th annual interna-tional workshop on presence, Philadelphia, PA (pp. 1-9).
[19] Goffman, E. (2008). Behavior in public places. Simon and Schuster (pp. 17-22).
[20] Gunawardena, C. N., & Zittle, F. J. (1997). Social presence as a predictor of satisfaction within a computer‐mediated conferencing environment. American journal of distance edu-cation, 11(3), 8-26.
[21] Bulu, S. T. (2012). Place presence, social presence, co-presence, and satisfaction in vir-tual worlds. Computers & Education, 58(1), 154-161.
[22] Hassell, M., Goyal, S., Limayem, M., & Boughzala, I. (2009). Being there: An empiri-cal look at learning outcomes in 3D virtual worlds. In AMCIS 2009 Proceedings, 733.
[23] Mansour, S., El-Said, M., & Bennett, L. (2010). Does the use of Second Life affect stu-dents’ feeling of social presence in e-learning. 8th Education and Information Systems, Technologies and Applications: EISTA.
[24] Zhang, C. (2010). Using virtual world learning environment as a course component in both distance learning and traditional classroom: Implications for technology choice in course delivery. In Proceedings of the Southern Association for Information Systems Con-ference, March 26th-27th, Atlanta, GA, USA.
[25] Lowenthal, P. R. (2010). Social presence. In Social computing: Concepts, methodolo-gies, tools, and applications (pp. 129-136). IGI global.
[26] Lowenthal, P. R. (2010). The evolution and influence of social presence theory on online learning. In Social computing: Concepts, methodologies, tools, and applications (pp. 113-128). IGI Global.
[27] Biocca, F., Harms, C., & Burgoon, J. K. (2003). Toward a more robust theory and measure of social presence: Review and suggested criteria. Presence: Teleoperators & vir-tual environments, 12(5), 456-480.
[28] Schroeder, R. (2002, October). Copresence and interaction in virtual environments: An overview of the range of issues. In Presence 2002: Fifth international workshop (pp. 274-295).
[29] Nowak, K. (2000). The influence of anthropomorphism on mental models of agents and avatars in social virtual environments. Michigan State University.
[30] Blascovich, J. (2002). Social influence within immersive virtual environments. In The social life of avatars (pp. 127-145). Springer, London.
[31] Carter, E. J., Hyde, J., Williams, D. L., & Hodgins, J. K. (2016, May). Investigating the influence of avatar facial characteristics on the social behaviors of children with autism. In Proceedings of the 2016 CHI conference on human factors in computing systems (pp. 140-151).
[32] Alcorn, A., et al. (2011, June). Social communication between virtual characters and children with autism. In International conference on artificial intelligence in education (pp. 7-14). Springer, Berlin, Heidelberg.
[33] Yakura, H., & Goto, M. (2020, November). Enhancing participation experience in vr live concerts by improving motions of virtual audience avatars. In 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) (pp. 555-565). IEEE.
[34] Slater, M., & Steed, A. (2002). Meeting people virtually: Experiments in shared virtual environments. In The social life of avatars (pp. 146-171). Springer, London.
[35] Conty, L., et al. (2010). The mere perception of eye contact increases arousal during a word-spelling task. Social neuroscience, 5(2), 171-186.
[36] Helminen, T. M., Kaasinen, S. M., & Hietanen, J. K. (2011). Eye contact and arousal: The effects of stimulus duration. Biological Psychology, 88(1), 124-130.
[37] Jarick, M., & Bencic, R. (2019). Eye contact is a two-way street: Arousal is elicited by the sending and receiving of eye gaze information. Frontiers in Psychology, 10, 1262.
[38] Knight, D. J., Langmeyer, D., & Lundgren, D. C. (1973). Eye-contact, distance, and af-filiation: The role of observer bias. Sociometry, 390-401.
[39] Argyle, M., & Dean, J. (1965). Eye-contact, distance and affiliation. Sociometry, 289-304.
[40] Martin, W. W., & Rovira, M. L. (1981). An experimental analysis of discriminability and bias in eye-gaze judgment. Journal of Nonverbal Behavior, 5(3), 155-163.
[41] Martin, W. W., & Jones, R. F. (1982). The accuracy of eye‐gaze judgement: A signal detection approach. British Journal of Social Psychology, 21(4), 293-299.
[42] Scarl, D. L. (1985). Eye contact perception at distances up to six meters. Dissertations and Theses. Paper 3693.
[43] Sibert, L. E., & Jacob, R. J. (2000, April). Evaluation of eye gaze interaction. In Pro-ceedings of the SIGCHI conference on Human Factors in Computing Systems (pp. 281-288).
[44] Tanriverdi, V., & Jacob, R. J. (2000, April). Interacting with eye movements in virtual environments. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (pp. 265-272).
[45] Jacob, R. J., & Karn, K. S. (2003). Eye tracking in human-computer interaction and us-ability research: Ready to deliver the promises. In The mind's eye (pp. 573-605). North-Holland.
[46] Fono, D., & Vertegaal, R. (2005, April). EyeWindows: evaluation of eye-controlled zooming windows for focus selection. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 151-160).
[47] Poole, A., & Ball, L. J. (2006). Eye tracking in HCI and usability research. In Encyclo-pedia of human computer interaction (pp. 211-219). IGI global.
[48] Qian, Y. Y., & Teather, R. J. (2017, October). The eyes don't have it: an empirical com-parison of head-based and eye-based selection in virtual reality. In Proceedings of the 5th Symposium on Spatial User Interaction (pp. 91-98).
[49] Eid, M. A., Giakoumidis, N., & El Saddik, A. (2016). A novel eye-gaze-controlled wheelchair system for navigating unknown environments: case study with a person with ALS. IEEE Access, 4, 558-573.
[50] Ramloll, R., Trepagnier, C., Sebrechts, M., & Finkelmeyer, A. (2004, March). A gaze contingent environment for fostering social attention in autistic children. In Proceedings of the 2004 symposium on Eye tracking research & applications (pp. 19-26).
[51] Esteves, A., Velloso, E., Bulling, A., & Gellersen, H. (2015, November). Orbits: Gaze interaction for smart watches using smooth pursuit eye movements. In Proceedings of the 28th annual ACM symposium on user interface software & technology (pp. 457-466).
[52] De Luca, A., Weiss, R., & Drewes, H. (2007, November). Evaluation of eye-gaze inter-action methods for security enhanced PIN-entry. In Proceedings of the 19th Australasian conference on computer-human interaction: Entertaining user interfaces (pp. 199-202).
[53] Bulling, A., Roggen, D., & Tröster, G. (2008, October). Eyemote–towards context-aware gaming using eye movements recorded from wearable electrooculography. In Inter-national Conference on Fun and Games (pp. 33-45). Springer, Berlin, Heidelberg.
[54] Ekman, I. M., Poikola, A. W., & Mäkäräinen, M. K. (2008). Invisible eni: using gaze and pupil size to control a game. In CHI'08 extended abstracts on Human factors in compu-ting systems (pp. 3135-3140).
[55] Velloso, E., & Carter, M. (2016, October). The emergence of eyeplay: a survey of eye interaction in games. In Proceedings of the 2016 annual symposium on computer-human interaction in play (pp. 171-185).
[56] Piumsomboon, T., Lee, G., Lindeman, R. W., & Billinghurst, M. (2017, March). Explor-ing natural eye-gaze-based interaction for immersive virtual reality. In 2017 IEEE sympo-sium on 3D user interfaces (3DUI) (pp. 36-39). IEEE.
[57] Hirzle, T., Gugenheimer, J., Geiselhart, F., Bulling, A., & Rukzio, E. (2019, May). A de-sign space for gaze interaction on head-mounted displays. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1-12).
[58] Atienza, R., Blonna, R., Saludares, M. I., Casimiro, J., & Fuentes, V. (2016, May). In-teraction techniques using head gaze for virtual reality. In 2016 IEEE Region 10 Symposi-um (TENSYMP) (pp. 110-114). IEEE.
[59] Pfeuffer, K., Mayer, B., Mardanbegi, D., & Gellersen, H. (2017, October). Gaze+ pinch interaction in virtual reality. In Proceedings of the 5th symposium on spatial user interac-tion (pp. 99-108).
[60] Zeleznik, R. C., Forsberg, A. S., & Schulze, J. P. (2005). Look-that-there: Exploiting gaze in virtual reality interactions. Brown Univ., Providence, RI, USA, Tech. Rep. CS-05.
[61] Patney, A., et al. (2016). Towards foveated rendering for gaze-tracked virtual reality. ACM Transactions on Graphics (TOG), 35(6), 1-12..
[62] Marwecki, S., Wilson, A. D., Ofek, E., Gonzalez Franco, M., & Holz, C. (2019, Octo-ber). Mise-unseen: Using eye tracking to hide virtual reality scene changes in plain sight. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Tech-nology (pp. 777-789).
[63] Sidenmark, L., Clarke, C., Zhang, X., Phu, J., & Gellersen, H. (2020, April). Outline pursuits: Gaze-assisted selection of occluded objects in virtual reality. In Proceedings of the 2020 chi conference on human factors in computing systems (pp. 1-13).
[64] Binetti, N., Harrison, C., Coutrot, A., Johnston, A., & Mareschal, I. (2016). Pupil dila-tion as an index of preferred mutual gaze duration. Royal Society Open Science, 3(7), 160086.