臺灣博碩士論文加值系統

隨著頭戴式顯示器（HMD）在虛擬實境（VR）領域中的進步，VR開發者能夠為各種領域產生不同的應用，如教育、訓練、虛擬旅遊、娛樂等。當使用者沉浸在虛擬世界時，雖然藉由VR-HMD使用者可以獲得視覺與聽覺上的回饋，但卻無法感受到其他感官上的回饋，這使得現實世界和虛擬世界之間仍然存在著一些差異。為了提供更好的使用者體驗，利用觸覺回饋技術是一種最直接的方法來增強沉浸式體驗，而藉由高沉浸感的VR體驗，能帶給VR應用有更好的表現。然而觸覺體驗的設計並不像我們想像的那麼容易，不同的觸覺刺激需要利用不同技術來重現。在虛擬世界中重建所有觸覺刺激是昂貴且耗電的，且在不同應用中不見得需要模擬所有的觸覺刺激。這不僅涉及到技術問題，也需要更深入地了解人類的感知能力，進而能最大限度地降低成本並專注於應用上的需求。在本論文中，我們的研究目標是藉由觸覺技術以增強身臨其境的沉浸式體驗，而這些具有不同潛力及優使性的觸覺技術，能發揮在不同VR應用當中。透過深入了解人類的觸覺及認知，我們可以為VR應用開發出合適的混合式觸覺系統。本研究提出了四種潛在的技術來增強沉浸式體驗，這之中包含了混合式觸覺系統的設計考量、設計準則及系統原型。藉由這些觸覺技術，使用者可以在虛擬實境中同時獲得手部及上半身的觸覺體驗，並與真實的椅子互動。本研究之成果不僅能益於為混合式觸覺系統之發展，亦助於未來的互動設計師和觸覺設計師能通過觸覺技術設計出更好的沉浸式體驗。

With the advances of head-mounted displays (HMD) for immersive virtual reality (VR), VR designers are able to produce VR application for many purposes such as education, virtual touring, games, etc. Although the users can perceive visual and auditory feedback with modern VR-HMD, the other human senses are missing when the user immersed in the virtual world. This leads the users to recognize that there is still a difference between the real world and the virtual world. To provide a better user experience, adding tactile sensation is a useful approach that can directly enhance the immersive experience, which is one of the human sense that influenced human perception. Additionally, with higher immersion, the VR application could have better performance. However, designing the haptics experience is not as easy as we think. Different tactile sensation needs several ways to reproduce. To recreate all of the tactile sensations in the virtual world is expensive and power consuming. Besides, we may not need all of it in an application. It is not only about the technical issue but also required to understand human perception, so we can minimize the cost and focus on the application needs. In this dissertation, we aim to enhance the immersive experience via haptic techniques. Those haptic techniques are potential and have different utilities for VR applications. By understanding the tactile sensations and human perception, we can develop a hybrid-haptics system for VR applications. We propose four kinds of potential techniques for enhancing the immersive experience, which contributes the design consideration, guidelines and prototype systems for the addressed hybrid-haptics system. With our haptic techniques, the users can perceive partial body and upper body experience simultaneously, and seamless interaction with a physical chair. Our research results and outcome not only benefit the hybrid-haptics system but also assist future designers and hapticans to design better immersive experience via haptic techniques.

論文口試委員審定書 i
致謝 ii
摘要 iii
ABSTRACT iv
CONTENTS vi
LIST OF FIGURES xii
LIST OF TABLES xvi
Chapter 1 Introduction 1
1.1 Background and Motivation 1
1.2 Problem and Approach 1
Chapter 2 Related Works 4
2.1 Enhanced Virtual Environment with Haptic Feedbacks 4
2.2 Enhanced Immersive Interaction with Haptic Feedbacks 5
2.3 Seamless Interaction with Physical World 6
2.4 Haptic Feedback 8
2.4.1 Motion-platform 8
2.4.2 Wearable 9
2.4.3 Prop-based 10
2.4.4 Controller 11
2.4.5 Stationary (Environment-based) 11
2.5 Summary 12
Chapter 3 Enhancing the Immersive Experience with Multiple Tactile Sensation 14
3.1 Sensory Modality of Touch 14
3.2 Challenge of Designing Tactile Display 15
3.3 Design and Implementation 15
3.3.1 Orb of Elemments 16
3.3.2 Sword of Elements 17
3.3.3 Bits of Elements 18
3.3.4 Spider Silk 19
3.3.5 Area of Elements and Area of Elements Plus 20
3.4 Application, Tool, and Simulation 23
3.4.1 Entertainment Application 23
3.4.2 Education Application 25
3.4.3 Brainstorming Tool for Designing VR Game 26
3.4.4 Super Hero Simulator 27
3.4.5 Weather Simulation 28
3.5 Exploratory Observation Study 30
3.5.1 Handheld Device 30
3.5.2 Stationary Device 31
3.6 Summary 32
Chapter 4 Enabling Multiple Tactile Sensations for Immersive Environment and Interaction in Virtual Reality 33
4.1 Virtual Reality with Multiple Tactile Sensations 34
4.2 Design Consideration 35
4.2.1 Contact / Non-contact Tactile Sensation 36
4.2.2 Fully / Partial Body Experience 36
4.2.3 Stationary / Portable 38
4.2.4 Scalable 38
4.3 Haptic Feedback System 39
4.3.1 Ubiquitous-based Haptics 39
4.3.2 Controller-based Haptics 42
4.4 Implementation 43
4.5 Interaction Design with Hybrid-Haptics 44
4.5.1 Blending Two Haptic Feedback with Full Body Experience. 44
4.5.2 Extending the Effect Area by the Controller. 45
4.5.3 Enhancing the Haptic Feedback from the Controller by the Stationary Device 45
4.5.4 Blending Extended Haptic with Stationary Device. 46
4.6 User Study 46
4.7 Result 48
4.8 Discussion 50
4.8.1 Hybrid-Haptic Feedbacks 50
4.8.2 Haptic Modules and Multisensory Feedback 51
4.8.3 Enhanced Immersive Experience 52
4.9 Limitation and Challenges 53
4.9.1 Area of Effect and Efficiency 53
4.9.2 Universal Controller in VR 54
4.10 Summary 55
Chapter 5 Enabling Seamless Interaction for a Physical Chair in Virtual Reality with Prop-based Haptic 56
5.1 Interacting with the Physical World 57
5.2 Design Consideration 58
5.2.1 Alignment 58
5.2.2 Integrity 59
5.2.3 Size 59
5.2.4 Softness 59
5.2.5 Location 59
5.3 Methods 60
5.3.1 Apparatus 60
5.3.2 Implementation 60
5.3.3 Calibration 61
5.4 User Study 1: Integrity, Size, and Softness 61
5.4.1 Hypotheses 61
5.4.2 Experimental Design 62
5.4.3 Data Analysis 63
5.4.4 Results and Discussion 64
5.5 User Study 2: Location 67
5.5.1 Hypotheses 67
5.5.2 Experimental Design 67
5.5.3 Data Analysis 68
5.5.4 Results and Discussion 68
5.6 Limitations and Future Work 69
5.7 Summary 70
Chapter 6 Enabling the Weathers Simulation for Walking Around in Immersive Environment with Haptics Feedback 72
6.1 Locomotion Experience with Multiple Tactile Sensations 73
6.2 Design Considerations 75
6.2.1 Temperature and Apparent temperature 76
6.2.2 Wind Speed and Direction 76
6.2.3 Rain, Humidity and Moisture 77
6.3 Multiple Tactile Display 77
6.3.1 Haptic Module System 77
6.3.2 Steerable Structure 79
6.3.3 Mechanism and Tracking 79
6.4 Weather Simulation 80
6.5 User Study 81
6.6 Result 82
6.7 Discussion 85
6.7.1 Thermal, Wind and Humidity Stimulation 85
6.7.2 Multi-sensory Feedback 85
6.7.3 The Clue from the Past Experience 86
6.8 Limitation and Challenge 87
6.8.1 The Condition of the Physical Space 87
6.8.2 Extreme Weather 88
6.9 Summary 88
Chapter 7 Conclusion and Future Work 89
7.1 Summary of the Thesis 89
7.2 Future Direction 89
REFERENCE 91

Azmandian, M., Hancock, M., Benko, H., Ofek, E., & Wilson, A. D. (2016, May). Haptic retargeting: Dynamic repurposing of passive haptics for enhanced virtual reality experiences. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (pp. 1968-1979). ACM.
Benko, H., Holz, C., Sinclair, M., & Ofek, E. (2016, October). NormalTouch and TextureTouch: High-fidelity 3D Haptic Shape Rendering on Handheld Virtual Reality Controllers. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (pp. 717-728). ACM.
Biocca, F., & Delaney, B. (1995). Immersive virtual reality technology. Communication in the age of virtual reality, 15, 32.
Cardin, S., Thalmann, D., & Vexo, F. (2007). Head mounted wind. In proceeding of the 20th annual conference on Computer Animation and Social Agents (CASA2007) (No. VRLAB-CONF-2007-136, pp. 101-108).
Chen, Y. S., Han, P. H., Hsiao, J. C., Lee, K. C., Hsieh, C. E., Lu, K. Y., ... & Hung, Y. P. (2016, October). SoEs: Attachable Augmented Haptic on Gaming Controller for Immersive Interaction. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (pp. 71-72). ACM.
Cheng, L. P., Lühne, P., Lopes, P., Sterz, C., & Baudisch, P. (2014, April). Haptic turk: a motion platform based on people. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems (pp. 3463-3472). ACM.
Cheng, L. P., Marwecki, S., & Baudisch, P. (2017, October). Mutual Human Actuation. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (pp. 797-805).
Cheng, L. P., Roumen, T., Rantzsch, H., Köhler, S., Schmidt, P., Kovacs, R., ... & Baudisch, P. (2015, November). Turkdeck: Physical virtual reality based on people. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (pp. 417-426). ACM.
Choi, I., Hawkes, E. W., Christensen, D. L., Ploch, C. J., & Follmer, S. (2016, October). Wolverine: A wearable haptic interface for grasping in virtual reality. In Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on (pp. 986-993). IEEE.
Deligiannidis, L., & Jacob, R. J. (2006, March). The vr scooter: Wind and tactile feedback improve user performance. In 3D User Interfaces, 2006. 3DUI 2006. IEEE Symposium on (pp. 143-150). IEEE.
Filingeri, D., & Havenith, G. (2015). Human skin wetness perception: psychophysical and neurophysiological bases. Temperature, 2(1), 86-104.
Fitzgerald, D., & Ishii, H. (2018, April). Mediate: A Spatial Tangible Interface for Mixed Reality. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (p. LBW625). ACM.
Forsberg, A. S., LaViola, J. J., Markosian, L., & Zeleznik, R. C. (1997). Seamless interaction in virtual reality. IEEE Computer Graphics and Applications, 17(6), 6-9.
Fujinawa, E., Yoshida, S., Koyama, Y., Narumi, T., Tanikawa, T., & Hirose, M. (2017, November). Computational design of hand-held VR controllers using haptic shape illusion. In Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology (p. 28). ACM.
Green, B. G. (1977). Localization of thermal sensation: An illusion and synthetic heat. Perception & Psychophysics, 22(4), 331-337.
Han, P. H., Chen, Y. S., Hsieh, C. E., Huang, Y. J., Lin, H. C., Tong, P. W., ... & Hung, Y. P. (2016, July). OoEs: playing in the immersive game with augmented haptics. In ACM SIGGRAPH 2016 VR Village (p. 15). ACM.
Han, P. H., Chen, Y. S., Yang, K. T., Chuan, W. S., Chang, Y. T., Yang, T. M., ... & Chou, C. H. (2017, November). BoEs: attachable haptics bits on gaming controller for designing interactive gameplay. In SIGGRAPH Asia 2017 VR Showcase(p. 3). ACM.
Han, P. H., Hsieh, C. E., Chen, Y. S., Hsiao, J. C., Lee, K. C., Ko, S. F., ... & Hung, Y. P. (2017, July). AoEs: enhancing teleportation experience in immersive environment with mid-air haptics. In ACM SIGGRAPH 2017 Emerging Technologies (p. 3). ACM.
Han, P. H., Huang, D. Y., Tsai, H. R., Chen, P. C., Hsieh, C. H., Lu, K. Y., ... & Hung, Y. P. (2015, July). Moving around in virtual space with spider silk. In ACM SIGGRAPH 2015 Emerging Technologies (p. 19). ACM.
Hettiarachchi, A., & Wigdor, D. (2016, May). Annexing reality: Enabling opportunistic use of everyday objects as tangible proxies in augmented reality. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (pp. 1957-1967). ACM.
Hoffman, H. G. (1998, March). Physically touching virtual objects using tactile augmentation enhances the realism of virtual environments. In Virtual Reality Annual International Symposium, 1998. Proceedings., IEEE 1998 (pp. 59-63). IEEE.
Hülsmann, F., Mattar, N., Fröhlich, J., & Wachsmuth, I. (2014). Simulating wind and warmth in virtual reality: conception, realization and evaluation for a cave environment. JVRB-Journal of Virtual Reality and Broadcasting, 11(10).
Israr, A., Kim, S. C., Stec, J., & Poupyrev, I. (2012, May). Surround haptics: tactile feedback for immersive gaming experiences. In CHI''12 Extended Abstracts on Human Factors in Computing Systems (pp. 1087-1090). ACM.
Israr, A., & Poupyrev, I. (2010, April). Exploring surround haptics displays. In CHI''10 Extended Abstracts on Human Factors in Computing Systems (pp. 4171-4176). ACM.
Israr, A., Zhao, S., McIntosh, K., Kang, J., Schwemler, Z., Brockmeyer, E., ... & Mahler, M. (2015, July). Po2: augmented haptics for interactive gameplay. In ACM SIGGRAPH 2015 Emerging Technologies (p. 21). ACM.
Jain, D., Sra, M., Guo, J., Marques, R., Wu, R., Chiu, J., & Schmandt, C. (2016, October). Immersive Scuba Diving Simulator Using Virtual Reality. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (pp. 729-739). ACM.
Katsumoto, Y. (2016, November). Bottomless joystick. In SIGGRAPH ASIA 2016 Emerging Technologies (p. 6). ACM.
Kojima, Y., Hashimoto, Y., & Kajimoto, H. (2009, October). A novel wearable device to present localized sensation of wind. In Proceedings of the International Conference on Advances in Computer Enterntainment Technology (pp. 61-65). ACM.
Konishi, Y., Hanamitsu, N., Outram, B., Minamizawa, K., Mizuguchi, T., & Sato, A. (2016, July). Synesthesia suit: the full body immersive experience. In ACM SIGGRAPH 2016 VR Village (p. 20). ACM.
Konyo, M., Tadokoro, S., Yoshida, A., & Saiwaki, N. (2005, August). A tactile synthesis method using multiple frequency vibrations for representing virtual touch. In Intelligent Robots and Systems, 2005.(IROS 2005). 2005 IEEE/RSJ International Conference on (pp. 3965-3971). IEEE.
Lai, C. J., Han, P. H., Wang, H. L., & Hung, Y. P. (2016, November). Exploring Manipulation Behavior on Video See-Through Head-Mounted Display with View Interpolation. In Asian Conference on Computer Vision (pp. 258-270). Springer, Cham.
Lécuyer, A. (2009). Simulating haptic feedback using vision: A survey of research and applications of pseudo-haptic feedback. Presence: Teleoperators and Virtual Environments, 18(1), 39-53.
Lin, J. W., Han, P. H., Lee, J. Y., Chen, Y. S., Chang, T. W., Chen, K. W., & Hung, Y. P. (2017, July). Visualizing the keyboard in virtual reality for enhancing immersive experience. In ACM SIGGRAPH 2017 Posters (p. 35). ACM.
Linder, N., & Maes, P. (2010, October). LuminAR: portable robotic augmented reality interface design and prototype. In Adjunct proceedings of the 23nd annual ACM symposium on User interface software and technology (pp. 395-396). ACM.
Lopes, P., Ion, A., & Baudisch, P. (2015, November). Impacto: Simulating physical impact by combining tactile stimulation with electrical muscle stimulation. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (pp. 11-19). ACM.
Lopes, P., You, S., Cheng, L. P., Marwecki, S., & Baudisch, P. (2017, May). Providing Haptics to Walls & Heavy Objects in Virtual Reality by Means of Electrical Muscle Stimulation. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (pp. 1471-1482). ACM.
McGill, M., Boland, D., Murray-Smith, R., & Brewster, S. (2015, April). A dose of reality: overcoming usability challenges in vr head-mounted displays. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 2143-2152). ACM.
Meehan, M., Insko, B., Whitton, M., & Brooks Jr, F. P. (2002). Physiological measures of presence in stressful virtual environments. Acm transactions on graphics (tog), 21(3), 645-652.
Moon, T., & Kim, G. J. (2004, November). Design and evaluation of a wind display for virtual reality. In Proceedings of the ACM symposium on Virtual reality software and technology (pp. 122-128). ACM.
Murakami, T., Person, T., Fernando, C. L., & Minamizawa, K. (2017, July). Altered touch: miniature haptic display with force, thermal and tactile feedback for augmented haptics. In ACM SIGGRAPH 2017 Emerging Technologies (p. 2). ACM.
Nilsson, N. C., Peck, T., Bruder, G., Hodgson, E., Serafin, S., Whitton, M., ... & Rosenberg, E. S. (2018). 15 Years of Research on Redirected Walking in Immersive Virtual Environments. IEEE computer graphics and applications, 38(2), 44-56.
Peiris, R. L., Peng, W., Chen, Z., Chan, L., & Minamizawa, K. (2017, May). Thermovr: Exploring integrated thermal haptic feedback with head mounted displays. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (pp. 5452-5456). ACM.
Ranasinghe, N., Jain, P., Thi Ngoc Tram, N., Koh, K. C. R., Tolley, D., Karwita, S., ... & Yen, C. C. (2018, April). Season Traveller: Multisensory Narration for Enhancing the Virtual Reality Experience. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (p. 577). ACM.
Ranasinghe, N., Jain, P., Tolley, D., Karwita, S., Yilei, S., & Do, E. Y. L. (2016, October). AmbioTherm: Simulating Ambient Temperatures and Wind Conditions in VR Environments. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (pp. 85-86). ACM.
Ranasinghe, N., Jain, P., Thi Ngoc Tram, N., Koh, K. C. R., Tolley, D., Karwita, S., ... & Yen, C. C. (2018, April). Season Traveller: Multisensory Narration for Enhancing the Virtual Reality Experience. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (p. 577). ACM.
Rheiner, M. (2014, July). Birdly an attempt to fly. In ACM SIGGRAPH 2014 Emerging Technologies (p. 3). ACM.
Roumen, T., Perrault, S. T., & Zhao, S. (2015, April). Notiring: A comparative study of notification channels for wearable interactive rings. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 2497-2500). ACM.
Shapira, L., & Freedman, D. (2016, September). Reality Skins: Creating Immersive and Tactile Virtual Environments. In Mixed and Augmented Reality (ISMAR), 2016 IEEE International Symposium on (pp. 115-124). IEEE.
Simeone, A. L., Mavridou, I., & Powell, W. (2017). Altering user movement behaviour in virtual environments. IEEE transactions on visualization and computer graphics, 23(4), 1312-1321.
Simeone, A. L., Velloso, E., & Gellersen, H. (2015, April). Substitutional reality: Using the physical environment to design virtual reality experiences. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 3307-3316). ACM.
Sodhi, R., Glisson, M., & Poupyrev, I. (2013, October). AIRREAL: tactile interactive experiences in free air. In Proceedings of the adjunct publication of the 26th annual ACM symposium on User interface software and technology (pp. 25-26). ACM.
Sra, M., Garrido-Jurado, S., Schmandt, C., & Maes, P. (2016, November). Procedurally generated virtual reality from 3D reconstructed physical space. In Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology (pp. 191-200). ACM.
Suma, E. A., Lipps, Z., Finkelstein, S., Krum, D. M., & Bolas, M. (2012). Impossible spaces: Maximizing natural walking in virtual environments with self-overlapping architecture. IEEE Transactions on Visualization and Computer Graphics, 18(4), 555-564.
Sun, Q., Wei, L. Y., & Kaufman, A. (2016). Mapping virtual and physical reality. ACM Transactions on Graphics (TOG), 35(4), 64.
Unity. https://unity3d.com/jp/learn/tutorials/topics/virtual-reality/movement-vr
Wilson, A., Benko, H., Izadi, S., & Hilliges, O. (2012, October). Steerable augmented reality with the beamatron. In Proceedings of the 25th annual ACM symposium on User interface software and technology (pp. 413-422). ACM.
Whitmire, E., Benko, H., Holz, C., Ofek, E., & Sinclair, M. (2018, April). Haptic Revolver: Touch, Shear, Texture, and Shape Rendering on a Reconfigurable Virtual Reality Controller. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (p. 86). ACM.
Zerroug, A., Cassinelli, A., & Ishikawa, M. (2011, April). Invoked computing: Spatial audio and video AR invoked through miming. In Proceedings of Virtual Reality International Conference (pp. 31-32).