臺灣博碩士論文加值系統

本研究提出一套在虛擬實境中，基於雙手手勢辨識之手拉坯模擬系統，建立在互動式三維建模與網格編輯技術上，透過輕便的體感攝影裝置截取使用者雙手的深度影像，分析手指各關節的活動資訊以進行手勢辨識，實現在虛擬實境世界中以手勢進行手拉坯訓練的目標，為手拉坯初學者提供簡易且低成本的練習環境，使其不需要煩惱相關設備與材料的準備問題，並能隨時隨地進行練習。在手勢和其相對應變形方法的設計上，為了使其更貼近實際使用情形，本系統參考傳統手拉坯創作所遵循的工藝知識，包含製作流程與捏塑手法，配合進階網格編輯技術，發展出擬真的陶坯變形手勢，並能在編輯後即時於使用者眼前呈現變形結果。而在使用者體驗方面，本研究主要採用燈號及文字標籤和手影提示等視覺方法實作即時回饋，於操作過程中給予充分引導，讓使用者得知目前手勢的正確與否，並能依據提示立即修正錯誤手勢，不僅能以正確的手部操作完成三維模型雕塑，也在無形中提高了使用者的學習體驗。此外，系統也提供了真實的手勢示範動畫，讓使用者能對各種編輯手勢進行更深入的認識，藉此在練習過程中得到更真實的體驗，同時提高學習成效。

Our research present a virtual reality pottery simulation system based on hands gesture recognition. We extend the pottery learning to virtual reality environment, and handled the gesture recognition through a lightweight depth sensor. Our system supports a series of interactive gestures which were designed by observing the traditional process of pottery production. With these gesture set and its corresponding deformation effect, user can apply the mesh deformation to the model in an intuitive way. To further improve the user experience. We designed a vision-based user feedback to guide the user using our system. By using the label, spot light, laser, and gesture shadow, the feedback system can also give an error message instantly in a variety ways to help the user to correct the gesture. This feature was helpful for the pottery learner, especially the beginner to learn the usage of gestures. It also improve the learning efficiency of our system.

摘 要 i
Abstract ii
致 謝 iii
表目錄 iv
圖目錄 v
目 錄 vii
第一章 緒論 1
1.1 研究背景 1
1.2 研究動機與目標 1
1.3 論文結構 2
1.4 論文貢獻 3
第二章 相關文獻與技術 4
2.1 表面變形技術 4
2.2 手勢辨識與設計 7
2.3 使用者引導與回饋 9
2.4 類似研究之比較 12
第三章 系統概觀 14
3.1 系統架構與特色 14
3.2 輔助子系統 14
3.3 系統操作流程 15
第四章 系統實作方法 16
4.1 手勢定義與辨識 16
4.1.1 深度體感裝置 16
4.1.2 手部資訊分析 16
4.1.3 系統基礎編輯手勢 17
4.1.4 手勢功能切換 20
4.2 網格變形技術 21
4.2.1 變形區域 21
4.2.2 Falloff數學模型 22
4.2.3 類高斯數學模型 22
4.2.4 網格變形方程式 25
4.3 使用者回饋設計 25
4.3.1 回饋分類 26
4.3.2 回饋型式 26
4.4 模型表面著色 29
4.4.1 著色系統 29
4.4.2 貼圖座標轉換 30
第五章 互動實驗設計 31
5.1 實驗設定 31
5.1.1 實驗裝置 31
5.1.2 受測人員 31
5.2 實驗A 31
5.2.1 步驟一 31
5.2.2 步驟二 32
5.3 實驗B 32
5.4 系統成果展示 35
5.5 回饋問卷統計 36
第六章 結論與未來展望 37
6.1 結論 37
6.2 系統限制 38
6.3 未來展望 39
參考文獻 41
附錄一 43
附錄二 44

1.Leap Motion, https://www.leapmotion.com/
2.HTC VIVE, https://www.htcvive.com/US/
3.Pei-Ying Chiang and Wei-Yu Li. Augmented reality paper clay making based on hand gesture recognition. In Proceedings of the 2nd ACM symposium on Spatial user interaction (SUI 14). ACM, New York, NY, USA, 2014. 144-144.
4.Jaebong Lee, Gabjong Han, and Seungmoon Choi. Haptic Pottery Modeling Using Circular Sector Element Method. In Proceedings of the 6th international conference on Haptics: Perception, Devices and Scenarios (EuroHaptics 08), Manuel Ferre (Ed.). Springer-Verlag, Berlin, Heidelberg, 2008. 668-674.
5.Gautam Kumar, Naveen Kr. Sharma, Partha Bhowmick. Wheel-throwing in digital space using number-theoretic approach. 2011.
6.Vinayak, Murugappan, S., Liu, H.R., Ramani, K.: Shape-it-up: hand gesture based creative expression of 3D shapes using intelligent generalized cylinders. Comput. Aided Des. 45,2013, 277–287.
7.Vinayak and Karthik Ramani, "Extracting Hand Grasp & Motion for Intent Expression in Mid-Air Shape Deformation: A Concrete & Iterative Exploration through a Virtual Pottery Application", Computers & Graphics, Volume 55, April 2016, Pages 143-156
8.Kinect, https://msdn.microsoft.com/en-us/library/hh438998.aspx
9.H. Kim, G. Albuquerque, S. Havemann & D. Fellner. 3D Modelling with Hand Gesture Interaction in a Semi-Immersive Environment. Technical Report TUBSCG-2004-01, Institute of Computer Graphics, TU Braunschweig, 2004.
10.Sungmin Cho, Yunsil Heo, and Hyunwoo Bang. Turn: a virtual pottery by real spinning wheel. In ACM SIGGRAPH 2012 Emerging Technologies (SIGGRAPH 12). ACM, New York, NY, USA, , Article 25 , 2012. 1 pages.
11.Vinayak, Karthik Ramani, Kevin Lee, Jr., and Raja Jasti. zPots: a virtual pottery experience with spatial interactions using the leap motion device. In CHI 14 Extended Abstracts on Human Factors in Computing Systems (CHI EA 14). ACM, New York, NY, USA, 2014. 371-374.
12.Vinayak et al., "A Gesture-free Geometric Approach for Mid-Air Expression of Design Intent in 3D Virtual Pottery", 2015.
13.SoftKinect, http://www.softkinetic.com/
14.Takeo Igarashi and John F. Hughes. A suggestive interface for 3D drawing. In Proceedings of the 14th annual ACM symposium on User interface software and technology (UIST 01). ACM, New York, NY, USA, 2001. 173-181.
15.Steve Tsang, Ravin Balakrishnan, Karan Singh, and Abhishek Ranjan. A suggestive interface for image guided 3D sketching. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 04). ACM, New York, NY, USA, 2004. 591-598.
16.Patrick Paczkowski, Julie Dorsey, Holly Rushmeier, and Min H. Kim. Paper3D: bringing casual 3D modeling to a multi-touch interface. In Proceedings of the 27th annual ACM symposium on User interface software and technology (UIST 14). ACM, New York, NY, USA, 2014. 23-32.
17.Ross T. Smith, Bruce H. Thomas, and Wayne Piekarski. Digital foam interaction techniques for 3D modeling. In Proceedings of the 2008 ACM symposium on Virtual reality software and technology (VRST 08). ACM, New York, NY, USA, 2008. 61-68.
18.R. Deshayes and T. Mens. Statechart modelling of interactive gesture-based applications. In Intel Workshop on Combining Design and Engineering of Interactive Systems through Models and Tools (ComDeisMoto), satellite event of INTERACT 2011, 2011.
19.K. Renaud and R. Cooper, "Feedback in Human-Computer Interaction - Characteristics and Recommendations," presented at the Annual Research Conference, South Africa, 2000.
20.Schwaller, M., Kühni, J., Angelini, L. and Lalanne, D. Improving in-game gesture learning with visual feedback. Human-Computer Interaction. Applications and Services. 8512 LNCS, PART 3 (2014), 2014. 643--653.
21.Realtime Texture Mapping, http://codeartist.mx/tutorials/dynamic-texture-painting/
22.Oculus, https://www.oculus.com/en-us/
23.Meta Glass, https://www.metavision.com/