( 您好!臺灣時間:2021/05/08 15:32
字體大小: 字級放大   字級縮小   預設字形  


研究生(外文):Chung-Hsuan Sun
論文名稱(外文):Portable AR Piano Training System
指導教授(外文):Pei-Ying Chiang
外文關鍵詞:Mobile devicePianoAssisted learning
  • 被引用被引用:1
  • 點閱點閱:144
  • 評分評分:系統版面圖檔系統版面圖檔系統版面圖檔系統版面圖檔系統版面圖檔
  • 下載下載:0
  • 收藏至我的研究室書目清單書目收藏:1
本研究提出一套可以在 android 手機平台上讓使用者學習鋼琴的教學系統,透過微型投影機、UVC Cam 以及非同步投影構築出可互動的投影鋼琴,使得使用者能以較低廉的價格與不受實體鋼琴空間的限制來達到鋼琴學習的效果。此系統提供了演示、學習、練習三種模式,在演示模式中,系統會將範例的譜直接演奏並將當前被彈奏琴鍵變色顯示於投影鍵盤上,而在教學模式中系統會將當前應彈奏的音符標示在五線譜上,而使用者則須依標示彈奏出正確的音符以達到練習的目標,最後在練習模式裡,使用者可隨意地彈奏與檢視自己的成果。
We propose a portable AR piano training system that helps the user to learn piano with affordable cost and no space limitations. During the performance, the training system indicates which note is playing and which corresponding piano key should be pressed on the keyboard. Our system also detects which keys are pressed by the user and give sound/visual feed-back to correct the user. With no requirement of a physical piano and tutor, our system extends a great range of time and space for the user who wants to learn piano.
摘 要 i
Abstract ii
致 謝 iii
目 錄 iv
表目錄 vi
圖目錄 vii
第一章 緒論 1
1.1 前言 1
1.2 研究動機與目的 3
1.3 論文架構 3
1.4 論文貢獻 4
第二章 相關文獻與研究 5
2.1 過去之相關研究 5
2.1.1 投影輔助學習 5
2.1.2 頭戴式顯示器輔助學習 8
2.1.3 穿戴式裝置輔助學習 8
2.1.4 手持投影系統 9
2.1.5 其他樂器之輔助學習 11
2.2 影像處理相關技術 13
2.2.1 灰階與二值化 13
2.2.2 尋找邊緣 15
2.2.3 凸包(Convexhull) 15
2.2.4 凸包面積 16
2.2.5 輪廓角點判別 17
2.2.6 YCbCr色彩空間 17
2.2.7 膚色偵測 18
第三章 系統架構 20
3.1 硬體架構 20
3.2 UVC Camera 20
3.3 非同步投影 22
3.4 Android影像處理 23
第四章 系統流程 25
4.1 預處理-鍵盤定位 25
4.1.1 琴鍵邊緣偵測 25
4.1.2 琴鍵角點 27
4.1.3 琴鍵判定點內插 29
4.2 實時處理-彈奏 30
4.2.1 膚色偵測 30
4.2.2 觸發判斷 30
4.2.3 視覺回饋 31
第五章 使用者體驗實驗 32
5.1 實驗目的 32
5.2 實驗方法 32
5.2.1 實驗設備 32
5.2.2 實驗對象 33
5.2.3 實驗內容 33
5.3 實驗結果 35
5.4 使用者回饋 36
第六章 結論與未來展望 37
6.1 結論 37
6.2 系統限制 37
6.3 未來展望 37
參考文獻 38
1.Carolina Cruz-Neira, Daniel J. Sandin, and Thomas A. DeFanti. "Surround-screen projection-based virtual reality: the design and implementation of the CAVE." In Proceedings of the 20th annual conference on Computer graphics and interactive techniques, New York, 1993.
2.P. Roman, M. Lazarov, and A. Majumder, "A Scalable distributed paradigm for multi-user interaction with tiled rear projection display walls," IEEE Transactions on Visualization and Computer Graphics, vol. 16, no. 6,2010, pp. 1623–1632.
3.Hrvoje Benko, Ricardo Jota, and Andrew Wilson. "MirageTable: freehand interaction on a projected augmented reality tabletop." In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 12), New York, 2010.
4.Hrvoje Benko, Andrew D. Wilson, and Federico Zannier. 2014. "Dyadic projected spatial augmented reality." In Proceedings of the 27th annual ACM symposium on User interface software and technology (UIST 14), New York, 2014.
5.Susan Hallam. "The power of music: Its impact on the intellectual, social and personal development of children and young people." International Journal of Music Education 28, 3 , 2010, 269--289.
6.Jutras, P. J. "The benefits of adult piano study as self-reported by selected adult piano students." Journal of Research in Music Education 54, 2 , 2006, 97--110.
7.Yoshinari Takegawa, "Tsutomu Terada. A Piano Learning Support System Considering Rhythm." The International Catholic Migration Commission, 2012.
8.Qi Yang and Georg Essl. "Visual Associations in Augmented Keyboard Performance." The International Conference on New Interfaces for Musical Expression, 2013.
9.Linsey Raymaekers, Jo Vermeulen, Kris Luyten, and Karin Coninx. "Game of tones: learning to play songs on a piano using projected instructions and games." In CHI 14 Extended Abstracts on Human Factors in Computing Systems (CHI EA 14). New York, 2014, 411-414.
10.Katja Rogers, Amrei Röhlig, Matthias Weing, Jan Gugenheimer, Bastian Könings, Melina Klepsch, Florian Schaub, Enrico Rukzio, Tina Seufert, and Michael Weber. "P.I.A.N.O.: Faster Piano Learning with Interactive Projection." In Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces (ITS 14), New York, 2014, 149-158.
11.Jonathan Chow, Haoyang Feng, Robert Amor, and Burkhard C. Wünsche. "Music education using augmented reality with a head mounted display." In Proceedings of the Fourteenth Australasian User Interface Conference (AUIC 13), Australia, 2013, 73-79.
12.Nien-Chi Chen, "PLSOH: An Augmented Reality Based Piano Learning System Using Optical See-Through Head Mounted Display." B.Eng. Thesis, National Taiwan University, 2015.
13.Kevin Huang, Thad Starner, Ellen Do, Gil Weiberg, Daniel Kohlsdorf, Claas Ahlrichs, and Ruediger Leibrandt. "Mobile music touch: mobile tactile stimulation for passive learning." In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 10). New York, 2010, 791-800.
14.Ramesh Raskar, Jeroen van Baar, Paul Beardsley, Thomas Willwacher, Srinivas Rao, and Clifton Forlines. "iLamps: geometrically aware and self-configuring projectors." In ACM SIGGRAPH 2003 Papers (SIGGRAPH 03). New York, 2003, 809-818.
15.Johannes Schöning, Michael Rohs, Sven Kratz, Markus Löchtefeld, and Antonio Krüger. "Map torchlight: a mobile augmented reality camera projector unit." In CHI 09 Extended Abstracts on Human Factors in Computing Systems (CHI EA 09). New York, 2009, 3841-3846.
16.Markus Löchtefeld, Sven Gehring, Ralf Jung, and Antonio Krüger. "guitAR: supporting guitar learning through mobile projection." In CHI 11 Extended Abstracts on Human Factors in Computing Systems (CHI EA 11). New York, 2011, 1447-1452.
17.Jia-Sheng You. "Interactive Music Learning Aids by Combination Tablet Computer and Glockenspiel." B.Eng. Thesis, Tamkang University, 2015.
18.Yingxue Zhang, Siqi Liu, Lu Tao, Chun Yu, Yuanchun Shi, and Yingqing Xu. "ChinAR: Facilitating Chinese Guqin Learning through Interactive Projected Augmentation." In Proceedings of the Third International Symposium of Chinese CHI (Chinese CHI 15). New York, 2015, 23-31.
19.Suzuki, Satoshi, and KeiichiA be. "Topological Structural Analysis of Digitized Binary Images by Border Following." Computer Vision, Graphics, and Image Processing ,1985, 32–46.
20.C. Bradford Barber, David P. Dobkin, and Hannu Huhdanpaa. "The quickhull algorithm for convex hulls." ACM Trans. Math. Softw. 22, 4, 1996, 469-483.
21.P. S. Hiremath and A. Danti, "Detection of Multiple Faces in an Image Using Skin Color Information and Lines-of-separability Face Model." International Journal of Pattern Recognition and Artificial Intelligence, vol. 20, no. 01, 2006, pp. 39–61.
22.Saki. https://github.com/saki4510t/UVCCamera.
23.完美鋼琴. https://play.google.com/store/apps/details?id=com.gamestar.perfectpiano.
電子全文 電子全文(網際網路公開日期:20210824)
註: 此連結為研究生畢業學校所提供,不一定有電子全文可供下載,若連結有誤,請點選上方之〝勘誤回報〞功能,我們會盡快修正,謝謝!
第一頁 上一頁 下一頁 最後一頁 top
系統版面圖檔 系統版面圖檔