臺灣博碩士論文加值系統

這篇論文提出一個叫做 MidasTouch 的穿戴式指套裝置,用於對日 常生活物品操作手勢之感測。此裝置包含了無線射頻辨識 (RFID) 讀 取器、無線射頻辨識 (RFID) 天線、慣性測量單元 (IMU)。透過無線射 頻辨識 (RFID) 讀取器以及貼在物品上面的無線射頻辨識 (RFID) 標籤, 能夠取得關於物品的資訊,藉由一對安置在指甲與指根部位的慣性測 量單元 (IMU),可以還原使用者對物品操作的手勢。感測距離約兩公 分的無線射頻辨識 (RFID) 天線安置在食指指甲的部位,可以用作讀取 物品的資訊以及作為手勢的起始終結判斷。手勢判斷的部分利用兩個 分別安置在指甲以及指根的慣性測量單元在三維空間中的旋轉作為判 斷依據,使用支持向量機 (Support Vector Machine) 來辨識不同的手勢。 使用者可以透過穿戴這個裝置在背景記錄下每天日常生活中對於物品 的操作,或是能夠用這些物品建立與物聯網物品的關係。我們進行了 兩個各包含 10 位使用者的實驗,實驗一包含八個手勢的辨識以及六個 日常生活物品,實驗一結果顯示我們的系統在辨識對日常生活用品的 手勢是可行的,在每位使用者自己的模型中可以達到平均 98.85% 的 辨識率,使用不同使用者的模型有 75.2% 的辨識率。實驗二包含兩種 手勢在五種功能相同的物體上操作,結果顯示使用不同物體的模型有 96.6% 的辨識率。

This work presents MidasTouch, a finger-worn RFID-inertia-sensor for sensing gestural interactions on everyday objects. The fingerstall-like device comprising an RFID reader and a couple of motion sensors for recognizing the tagged objects and the hand gestures that users are performing on them. The short-range (∼2cm) RFID sensor mounted on the index fingertip functions as a robust feature for object identification and inertia sensor data segmentation, allowing the users wearing the device to directly associate their intentions with tagged objects by touching them or performing natural gestures on them. By wearing the device with the embedded sensing, users can easily log the daily activities related to the everyday things, or control smart things using every- day things. Results of two 10-user studies suggest that the proposed system is feasible of recognizing gestures from everyday objects, and supports reliably uses by each user in a personalized train-and-use basis.

口試委員審定書 i
誌謝 ii
中文摘要 iii
Abstract iv
Contents v
List of Figures viii
1 Introduction 1
1.1 Motivation.................................. 1
1.2 MidasTouch................................. 3
2 Related Works 4
2.1 SensingFromEnvironment......................... 4
2.2 SensingFromObject ............................ 5
2.3 SensingFromWearableCamera ...................... 6
2.4 RFID-motionSensing............................ 8
3 Design and Implementation 9
3.1 DesigningConsiderations.......................... 9
3.2 PrototypingHardware............................ 10
3.2.1 Designing RFID Antenna for Short-Range Sensing . . . . . . . . 11
3.2.2 Designing IMU Array for Index-Finger Posture Sensing . . . . . 12
3.2.3 Using Short-Range RFID for Gesture Segmentation . . . . . . . . 13
3.3 SoftwareImplementation.......................... 13
3.3.1 FeaturePreprocessing ....................... 14
3.3.2 Training and Predicting Tool.................... 14
3.4 ApplicationExamples............................ 16
3.4.1 Connecting Everyday Things and Smart Things . . . . . . . . . . 16
3.4.2 Logging Daily Activities on Everyday Things . . . . . . . . . . . 16
4 EVALUATION 18
4.1 Study1:Gestures Recognition on Everyday objects . . . . . . . . . . . . 18
4.1.1 Data Collection........................... 18
4.1.2 Results and Discussion....................... 20
4.2 Study 2: Gestures Recognition on Objects in Similar Form Factors . . . . 21
4.2.1 Data Collection........................... 21
4.2.2 Results and Discussion....................... 22
5 Discussion 23
5.1 Detect the Repetition within Gesture.................... 23
5.2 Editor.................................... 25
5.2.1 Web-based System ......................... 26
5.2.2 Editing Interface .......................... 27
5.2.3 Connect to Smart Things...................... 28
5.2.4 Display Devices .......................... 29
5.3 Predict Daily Activities........................... 30
5.4 Semi-supervised learning.......................... 31
5.5 Limitation.................................. 32
5.5.1 Same Location in Each Interaction................. 32
5.5.2 Yaw Drifting Problem ....................... 32
5.5.3 RFID on Metal Object ....................... 33
6 CONCLUSION AND FUTURE WORK 34
7 Appendix 35
7.1 confusion matrix of study1......................... 35
Bibliography 41

[1] K. P. Fishkin, M. Philipose, and A. Rea. Hands-on rfid: wireless wearables for detecting use of objects. In Proc. IEEE ISWC ’05, pages 38–41, 2005.
[2] Assaf Feldman, Emmanuel Munguia Tapia, Sajid Sadi, Pattie Maes, and Chris Schmandt. Reachmedia: On-the-move interaction with everyday objects. In Proc. IEEE ISWC ’05, pages 52–59, 2005.
[3] Eugen Berlin, Jun Liu, Kristof van Laerhoven, and Bernt Schiele. Coming to grips with the objects we grasp: Detecting interactions with efficient wrist-worn sensors. In Proc. TEI ’10, pages 57–64, 2010.
[4] J. C. Becker and N. V. Thakor. A study of the range of motion of human fingers with application to anthropomorphic designs. IEEE Transactions on Biomedical Engineering, pages 110–117, 1988.
[5] Kai-Yin Cheng, Rong-Hao Liang, Bing-Yu Chen, Rung-Huei Laing, and Sy-Yen Kuo. icon: Utilizing everyday objects as additional, auxiliary and instant tabletop controllers. In Proc. ACM CHI ’10, pages 1155–1164, 2010.
[6] Hanchuan Li, Can Ye, and Alanson P. Sample. Idsense: A human object interaction detection system based on passive uhf rfid. In Proc. ACM CHI ’15, pages 2555– 2564, 2015.
[7] Michael Buettner, Richa Prasad, Matthai Philipose, and David Wetherall. Recogniz- ing daily activities with rfid-based sensors. In Proceedings of the 11th International Conference on Ubiquitous Computing, UbiComp ’09, pages 51–60, New York, NY, USA, 2009. ACM.
[8] Munehiko Sato, Ivan Poupyrev, and Chris Harrison. Touche: Enhancing touch in- teraction on humans, screens, liquids, and everyday objects. In Proc. ACM CHI ’12, pages 483–492, 2012.
[9] Makoto Ono, Buntarou Shizuki, and Jiro Tanaka. Touch & activate: Adding inter- activity to existing objects using active acoustic sensing. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, UIST ’13, pages 31–40, New York, NY, USA, 2013. ACM.
[10] L. Dipietro, A. M. Sabatini, and P. Dario. A survey of glove-based systems and their applications. IEEE Transactions on Systems, Man, and Cybernetics, Part C, 38(4): 461–482, 2008.
[11] M. Fiala. Artag, a fiducial marker system using digital techniques. In Proc. IEEE CVPR ’05), volume 2, pages 590–596, 2005.
[12] Xing-Dong Yang, Tovi Grossman, Daniel Wigdor, and George Fitzmaurice. Magic finger: Always-available input through finger instrumentation. In Proc. ACMUIST ’12, pages 147–156, 2012.
[13] Liwei Chan, Yi-Ling Chen, Chi-Hao Hsieh, Rong-Hao Liang, and Bing-Yu Chen. Cyclopsring: Enabling whole-hand and context-aware interactions through a fisheye ring. In Proc. ACM UIST ’15, pages 549–556, 2015.
[14] David Kim, Otmar Hilliges, Shahram Izadi, Alex D. Butler, Jiawen Chen, Iason Oikonomidis, and Patrick Olivier. Digits: Freehand 3d interactions anywhere using a wrist-worn gloveless sensor. In Proc. ACM UIST ’12, pages 167–176, 2012.
[15] Xing-Dong Yang, Khalad Hasan, Neil Bruce, and Pourang Irani. Surround-see: En- abling peripheral vision on smartphones during active use. In Proc. ACM UIST ’13, pages 291–300, 2013.
[16] Chris Harrison, Hrvoje Benko, and Andrew D. Wilson. Omnitouch: Wearable mul- titouch interaction everywhere. In Proc. ACM UIST ’11, pages 441–450, 2011.
[17] Sean Gustafson, Daniel Bierwirth, and Patrick Baudisch. Imaginary interfaces: Spa- tial interaction with empty hands and without visual feedback. In Proc. ACM UIST ’10, pages 3–12, 2010.
[18] Gilles Bailly, Jörg Müller, Michael Rohs, Daniel Wigdor, and Sven Kratz. Shoe- sense: A new perspective on gestural interaction and wearable applications. In Proc. ACM CHI ’12, pages 1239–1248, 2012.
[19] Jason Hong. Considering privacy issues in the context of google glass. Commun. ACM, 56(11):10–11, November 2013.
[20] http://www.proxinc.co.jp/index.jsp.
[21] Chih-Chung Chang and Chih-Jen Lin. Libsvm: A library for support vector ma-
chines. ACM Trans. Intell. Syst. Technol., 2(3):27:1–27:27, May 2011.
[22] https://en.wikipedia.org/wiki/gimbal_lock.
[23] Dan Morris, T. Scott Saponas, Andrew Guillory, and Ilya Kelner. Recofit: Using a wearable sensor to find, recognize, and count repetitive exercises. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’14, pages 3225–3234, New York, NY, USA, 2014. ACM.
[24] B. Atal and L. Rabiner. A pattern recognition approach to voiced-unvoiced-silence classification with applications to speech recognition. IEEE Transactions on Acous- tics, Speech, and Signal Processing, 24(3):201–212, Jun 1976.
[25] N. V. Thakor and Y. S. Zhu. Applications of adaptive filtering to ecg analysis: noise cancellation and arrhythmia detection. IEEE Transactions on Biomedical Engineer- ing, 38(8):785–794, Aug 1991.
[26] Paul Boersma. Accurate short-term analysis of the fundamental frequency and the harmonics-to-noise ratio of a sampled sound. In IFA Proceedings 17, pages 97–110, 1993.
[27] Valentin Heun, James Hobin, and Pattie Maes. Reality editor: Programming smarter objects. In Proceedings of the 2013 ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication, UbiComp ’13 Adjunct, pages 307–310, New York, NY, USA, 2013. ACM.
[28] D.J.Patterson,D.Fox,H.Kautz,andM.Philipose.Fine-grainedactivityrecognition by aggregating abstract object usage. In Ninth IEEE International Symposium on Wearable Computers (ISWC’05), pages 44–51, Oct 2005.
[29] http://www.invensense.com/products/motion-tracking/9-axis/mpu-9250/.
[30] MarkPedley.Tiltsensingusingathree-axisaccelerometer.FreescaleSemiconductor
Application Note, pages 2012–2013, 2013.
[31] Freescale Semiconductor. Implementing a tilt-compensated ecompass using ac- celerometer and magnetometer sensors. Freescale Semiconductor Application Note, AN, 4248, 2012.
[32] Sebastian OH Madgwick. An efficient orientation filter for inertial and inertial/mag- netic sensor arrays. Report x-io and University of Bristol (UK), 2010.
[33] https://www.atlasrfidstore.com/metal-mount-rfid-tags/.