跳到主要內容

臺灣博碩士論文加值系統

(44.192.79.149) 您好!臺灣時間:2023/06/02 23:20
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果 :::

詳目顯示

: 
twitterline
研究生:彭敘豪
研究生(外文):PENG, HSU-HAO
論文名稱:點對點即時虛擬人物線上會議系統
論文名稱(外文):Point-To-Point Real-time Avatar Online Meeting System
指導教授:謝東儒謝東儒引用關係
指導教授(外文):HSIGH, TUNG-JU
口試委員:葉士青張陽郎謝東儒
口試委員(外文):YEH, SHIH-CHINGCHANG, YANG-LANGHSIGH, TUNG-JU
口試日期:2022-06-23
學位類別:碩士
校院名稱:國立臺北科技大學
系所名稱:資訊工程系
學門:工程學門
學類:電資工程學類
論文種類:學術論文
論文出版年:2022
畢業學年度:110
語文別:中文
論文頁數:27
中文關鍵詞:臉部追蹤虛擬人物線上會議網頁開發
外文關鍵詞:Face TrackingVirtual AvatarOnline MeetingWeb Development
相關次數:
  • 被引用被引用:0
  • 點閱點閱:184
  • 評分評分:
  • 下載下載:30
  • 收藏至我的研究室書目清單書目收藏:0
近年來,線上會議的需求日益增加,人們也越來越習慣利用網路視訊方式與他人互動,例如遠距離教學、遠距離辦公等等。但人們通常因為隱私考量而不願意將視訊鏡頭開啟,影響了線上教學或是會議的互動性;如果想要利用虛擬人物來發起或是參加會議的話,通常需要購買特定軟體或是器材才能達成,對於單純想要發起會議的使用者來說非常不方便。基於上述原因,我們提出一個利用Live2D Cubism WebSDK、ReactJS、MediaPipe FaceMesh,整合臉部辨識、2D虛擬人物以及即時視訊以及語音通話的網頁系統。使用者可以利用虛擬人物,在不需要露面的前提下保持會議的互動性。本系統利用可在使用者端網頁執行的AI臉部追蹤(Face Alignment),將所偵測到的資料套用至可自由變形的2D虛擬人物,以多媒體串流(MediaStream)的方式直接傳送到參加者的電腦上。
The need for online meetings nowadays is rising as people are used to interacting with others remotely. However, due to privacy concerns, people are usually reluctant to reveal their faces unless demanded, hindering the interactiveness of the meeting. We propose a real-time meeting web application that integrates facial recognition techniques and morphable anime characters, utilizing Live2D Cubism WebSDK, ReactJS and Mediapipe FaceMesh. This application enables users to show their expressions through 2D avatars while preventing them from revealing their real faces and surrounding areas, retaining the quality of communication in online meetings and privacy. This system utilizes a web-executable AI face alignment model to detect face position and expressions, then parse the extracted facial data to apply to 2D morphable characters. After these, send the rendered character image to other users directly via MediaStream.
中文摘要 i
英文摘要 ii
致謝 iv
目 錄 v
圖 目 錄 vi
表 目 錄 vii
1 導論 1
1.1 研究背景 1
1.2 研究動機 3
1.3 論文貢獻 5
2 相關文獻討論 6
3 虛擬人物控制 9
3.1 Cubism 9
3.1.1 ArtMesh 10
3.1.2 Deformer 11
3.1.3 參數控制 11
3.1.4 WebSDK 13
3.2 MediaPipe 13
3.2.1 Canonical Face Model 14
3.3 特徵點轉換 15
3.4 雜訊過濾 18
4 虛擬人物會議軟體 20
5 結論 23
5.1 結論 23
5.2 未來展望 24
參 考 文 獻 25
[1] Gather Presence, Inc. Gather. https://www.gather.town/, 2022.
[2] Hololive Productions. Chloe ch. 沙花叉クロヱ https://www.youtube.com/watch?v=5DJo5Wz700U, 2022.
[3] Xiangyu Zhu, Xiaoming Liu, Zhen Lei, and Stan Z. Li. Face alignment in full pose range:A 3d total solution. CoRR, abs/1804.01005, 2018.
[4] Valentin Bazarevsky, Yury Kartynnik, Andrey Vakunov, Karthik Raveendran, and Matthias Grundmann. Blazeface: Sub-millisecond neural face detection on mobile gpus. CoRR, abs/1907.05047, 2019.
[5] Yury Kartynnik, Artsiom Ablavatski, Ivan Grishchenko, and Matthias Grundmann. Realtime facial surface geometry from monocular video on mobile gpus. CoRR, abs/1907.06724, 2019.
[6] Artsiom Ablavatski, Andrey Vakunov, Ivan Grishchenko, Karthik Raveendran, and Matsvei Zhdanovich. Real-time pupil tracking from monocular video for digital puppetry. CoRR, abs/2006.11341, 2020.
[7] Google. Mediapipe iris, https://google.github.io/mediapipe/solutions/iris.html, 2020.
[8] Scott Schaefer, Travis McPhail, and Joe Warren. Image deformation using moving least squares. ACM Trans. Graph., 25(3):533–540, jul 2006.
[9] Google. Mediapipe face mesh, https://google.github.io/mediapipe/solutions/face_mesh.html, 2020.
[10] Playboard. Most super chatted in worldwide. https://playboard.co/en/youtube-ranking/most-superchatted-all-channels-inworldwide-total, 2022.
[11] Hololive Production. 潤羽るしあ, https://www.youtube.com/channel/UCl_gCybOJRIgOXw6Qb4qJzQ, 2022.
[12] Hololive Production. 桐 生 コ コ, https://www.youtube.com/channel/UCS9uQI-jC3DE0L4IpXyvr6w, 2021.
[13] Hololive Production. 兎 田 ぺ こ ら, https://www.youtube.com/channel/UC1DCedRgGHBdm81E1llLhOQ, 2022.
[14] Hololive Production. 宝 鐘 マ リ ン, https://www.youtube.com/channel/UCCzUftO8KOVkV4wQG1vkUvg, 2022.
[15] Live2D Inc. 標準パラメータリスト, https://docs.live2d.com/cubismeditor-manual/standard-parametor-list/?locale=ja, 2010.
[16] ASUS. Asus webcam c3, https://www.asus.com/tw/Accessories/Streaming-Kits/All-series/ASUS-Webcam-C3/, 2021.
[17] Valve. Valve index https://store.steampowered.com/valveindex?l=tchinese, 2021.
[18] Muhammed Kocabas, Nikos Athanasiou, and Michael J. Black. Vibe: Video inference for human body pose and shape estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
[19] Mark Sandler, Andrew G. Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Inverted residuals and linear bottlenecks: Mobile networks for classification, detection and segmentation. CoRR, abs/1801.04381, 2018.
[20] Live2D Inc. Live2d cubism, https://www.live2d.com/en/, 2010.
[21] Google. Mediapipe, https://google.github.io/mediapipe/, 2020.
[22] Tensorflow. Face landmarks detection, https://github.com/tensorflow/tfjs-models/tree/master/face-landmarks-detection, 2022.
[23] Géry Casiez, Nicolas Roussel, and Daniel Vogel. 1 € filter: A simple speed-based low-pass filter for noisy input in interactive systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’12, page 2527–2530, New York, NY, USA, 2012. Association for Computing Machinery.
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top