臺灣博碩士論文加值系統

本研究與語言治療師實際訪談後，發現有許多需要進行口腔功能訓練的身心障礙者，但由於語言治療師人手不足、協助的照服員或家長未能接受專業訓練，導致無法負荷口腔功能訓練的需求。因此本研究嘗試使用深度學習的技術，訓練口腔動作辨識模型，進而開發一個能進行口腔辨識的APP，期能在進行口腔功能訓練時，協助檢查是否確實執行每個動作。
在進行臉部偵測實作時，可分為以影像為基礎(image-based)或以特徵為基礎(feature-based)兩種方式。前者是指以臉部表象的影像，直接進行辨識模型訓練；而後者則是使用臉部的特徵點標註來進行訓練。本研究將比較兩種作法，探討何者更適合口腔動作訓練系統。
本研究以口腔功能訓練、深度學習技術及特徵萃取作為研究基礎。在口腔功能訓練方面，常見的作法包括感覺異常訓練及動作異常訓練。深度學習技術則因不同的模型架構不斷推陳出新，本研究主要探討AlexNet、VGG16等較為經典的模型，由於本研究需將模型應用到行動裝置中，因此最後選擇使用MobileNetV3。至於在特徵萃取方面，則蒐集了色彩特徵、紋路特徵與形狀特徵等文獻。
本研究的資料集是自行拍攝影片，轉成影格後每個動作等間隔取 3 張進行臉部偵測，被辨識出來的影像為 508張，藉由資料擴增後合計 7112 張。進行模型訓練時，再將整個資料分割為80%的訓練資料集以及20%的測試資料集。
由於MobileNetV3分成Large及Small兩種架構，且不同的寬度乘數會影響模型的複雜度及參數數量，本研究透過實驗，選擇了寬度乘數為 0.75的 MobileNetV3-Small，能讓複雜度較小且參數數量也較少。接著使用此模型比較兩種臉部偵測方式，結果顯示直接使用口腔原始影像，優於進行口腔特徵點標註影像進行訓練的成效。
本研究最後採用成效最好的模型，來進行口腔動作辨識APP的開發，不僅能協助服務對象自行進行口腔動作異常訓練，還能讓語言治療師、照服員、家長等從旁提供口腔感覺異常訓練。

In our research, we interviewed speech therapists and found that there are many physically and mentally handicapped people who need oral function training. Due to the shortage of speech therapists and the lack of professional training for the assisting caregivers and parents, they are unable to cope with the demand for oral function training. Therefore, we tried to use deep learning techniques to train an oral motor recognition model, and then developed an oral health app that can assist in checking whether or not each oral motor is actually performed.
Face detection can be divided into image-based or feature-based approaches. The former refers to the direct training of the recognition model with the image of the face, while the latter uses the facial landmark for training. In our research, we will compare the two approaches and investigate which one is more suitable for oral motor training system.
This study is based on oral function training, deep learning techniques and feature extraction. In oral function training, common practices include sensory abnormality training and motor abnormality training. For deep learning techniques, the model frameworks are constantly evolving, and this study mainly focuses on classical models such as AlexNet and VGG16. Finally MobileNetV3 is adopted because of the need to apply the model to mobile devices. As for feature extraction, we collected literatures on color features, texture features, and shape features.
The dataset of our research is a self-recorded video, which is converted into frames and then three images are taken at equal intervals of each action for face detection. 508 images are recognized, and the total number of images is 7112 after data augmentation. For model training, the whole data is divided into 80% of the data set and 20% of the test set.
Since MobileNetV3 can be divided into two architectures, Large and Small, different width multipliers may affect the complexity and the number of parameters of the model. In our research, MobileNetV3-Small with a width multiplier of 0.75 was chosen to minimize the complexity and the number of parameters. Then, the model is used to compare the two face detection methods, and the results show that the direct use of the original images is more effective than the training of the oral cavity feature landmark images.
In this study, the most effective model was used for the development of the oral motor recognition app, which not only assisted the clients to perform the oral motor abnormalities training on their own, but also allowed the speech therapists, caregivers, and parents to perform the oral sensory abnormalities training.

中文摘要 i
英文摘要 ii
誌謝 iv
目錄 v
圖目錄 vii
表目錄 viii
第一章、 緒論 1
1.1 研究背景 1
1.2 研究動機及目的 2
第二章、 文獻探討 4
2.1 口腔功能訓練 4
2.1.1 感覺異常訓練 4
2.1.2 動作異常訓練 4
2.2 深度學習(Deep Learning) 5
2.2.1. AlexNet 5
2.2.2. VGG16 6
2.2.3. MobileNet 6
2.2.4. 結論 8
2.3 特徵萃取(Feature Extraction) 8
2.3.1 色彩特徵(Color Feature) 9
2.3.2 紋路特徵(Texture Feature) 9
2.3.3 形狀特徵(Shape Feature) 9
2.3.4 結論 9
第三章、 研究方法 10
3.1. 口腔動作影像資料庫 10
3.1.1 影像前處理 13
3.1.2 資料擴增(Data Augmentation) 13
3.1.3 臉部特徵點標註 14
3.2. 深度學習 15
3.3. 行動應用軟體開發 16
3.3.1 動作異常訓練 16
3.3.2 感覺異常訓練 17
第四章、 研究結果 18
4.1 不同影像訓練成效 18
4.2 口腔動作辨識APP 19
第五章、 結論 23
5.1 研究結果與實務意涵 23
5.1.1. 技術層面 23
5.1.2. 使用者層面 23
5.2 研究限制 24
5.3 未來研究方向 24
參考文獻 26

Almabdy, S. and Elrefaei L., "Deep Convolutional Neural Network-Based Approaches for Face Recognition", Applied Sciences ,vol 9, 20, pp. 4397, 2019.
Alqahtani, Z. R., Sunar, M. S., and Ali, A., "Landmark Localization in Occluded Faces Using Deep Learning Approach", Innovative Systems for Intelligent Health Informatics, Vol 72, pp. 1023–1029, May 2021.
Arnold, D., Li, X., and Lin, Y., Zhonghui Wang, Won-Jae Yi, Jafar Saniie, "IoT Framework for 3D Body Posture Visualization", IEEE International Conference on Electro Information Technology, pp. 117-120, 2020.
Boiron, M., Nobrega, L. D., Roux, S., Henrot, A., and Saliba, E.,"Effects of oral stimulation and oral support on non-nutritive sucking and feeding performance in preterm infants", Developmental medicine and child neurology, vol. 49, 6, pp. 439-444, June 2007.
Du-Harpur, X., Watt, F. M., Luscombe, N. M., & Lynch, M. D., "What is AI? Applications of artificial intelligence to dermatology", The British journal of dermatology, 183(3), pp. 423–430, September 2020.
Gupta, N., "A Literature Survey on Artificial Intelligence", INTERNATIONAL JOURNAL OF ENGINEERING RESEARCH & TECHNOLOGY, Volume 5, Issue 19, 2017.
Hjelmås, E. and Low, B. K. , "Face Detection: A Survey", Computer Vision and Image Understanding, Volume 83, Issue 3, pp. 236-274, 2021.
Howard, A. G., Zhu, M., Chen B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H., "MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications", Computer Science Computer Vision and Pattern Recognition, Apr 2017.
Howard, A., Sandler, M., Chu, G., Chen, L. C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., Vasudevan, V., Le, Q. V., and Adam, H.,"Searching for MobileNetV3", Computer Science Computer Vision and Pattern Recognition, May 2019
Hubel, D. H. and Wiesel, T. N., "Receptive fields, binocular interaction and functional architecture in the cat's visual cortex", The Journal of physiology, vol. 160, 1 pp. 106-54, 1962.
Keyvanpour, M. R., Vahidiansadegh, S., and Mirzakhani, Z., "An analytical review of texture feature extraction approaches", International Journal of Computer Applications in Technology, vol. 65, pp. 118, 2021.
Khalid, S., Khalil, T., and Nasreen, S., "A survey of feature selection and feature extraction techniques in machine learning", 2014 Science and Information Conference, pp. 372-378, London, UK, 2014.
Krizhevsky, A., Sutskever, I. ,and Hinton, G. E., "ImageNet classification with deep convolutional neural networks", In Proceedings of the 25th International Conference on Neural Information Processing Systems, Volume 1, pp. 1097–1105, December 2012.
Kulikajevas, A., Maskeliunas, R., and Damaševičius, R., "Detection of sitting posture using hierarchical image composition and deep learning", PeerJ. Computer science, vol. 7, March 2021.
Lee, C. and Landgrebe, D.A., "Feature extraction based on decision boundaries,", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 15, no. 4, pp. 388-400, April 1993.
Medjahed, S. A., "A Comparative Study of Feature Extraction Methods in Images Classification", International Journal of Image, Graphics and Signal Processing, vol. 7, pp. 16-23, 2015.
P.S, S. K. and Vs, D., "Extraction of Texture Features using GLCM and Shape Features using Connected Regions", International Journal of Engineering and Technology, vol. 8, 6, pp. 2926-2930, December 2016.
Prakash, R. M., Thenmoezhi, N., and Gayathri, M., "Face Recognition with Convolutional Neural Network and Transfer Learning", International Conference on Smart Systems and Inventive Technology, pp. 861-864, 2019.
Salau, A. O. and Jain, S., "Feature Extraction: A Survey of the Types, Techniques, Applications", 2019 International Conference on Signal Processing and Communication , pp. 158-164, NOIDA, India, 2019.
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L. C., "MobileNetV2: Inverted Residuals and Linear Bottlenecks", Computer Science Computer Vision and Pattern Recognition, Jan 2018
Simonyan, K. and Zisserman, A., "Very Deep Convolutional Networks for Large-Scale Image Recognition", 3rd International Conference on Learning Representations, pp. 1–14, 2015.
Tian, D., "A review on image feature extraction and representation techniques", International Journal of Multimedia and Ubiquitous Engineering, vol. 8, pp.385-395, 2013.
Venkateswarlu, I. B., Kakarla, J., and Prakash, S., "Face mask detection using MobileNet and Global Pooling Block," 2020 IEEE 4th Conference on Information & Communication Technology ,pp. 1-5, Chennai, India, 2020.
Zhang, X., Cui, J., Wang, W., and Lin, C., "A Study for Texture Feature Extraction of High-Resolution Satellite Images Based on a Direction Measure and Gray Level Co-Occurrence Matrix Fusion Algorithm", Sensors, vol. 17, no. 7, pp.1474, 2017.
何怡慧、劉秀月、黃純德，「復健期腦中風病患咀嚼吞嚥障礙盛行率及初步篩檢徵象之探討」，護理雜誌，第61卷，第2期，頁54-62，民國103年。
施博文，「基於機器訓練機器技術建構姿態與深度估算的多任務深度學習網路模型」，國立交通大學資訊科學與工程研究所，碩士論文，民國109年。
夏至賢、陳昭和、郭景明、李永祥，「多混合特徵法於自主投籃姿勢辨識」，NCS 2017 全國計算機會議，頁443-448，台北，民國107年。
陳旭萱，「發展障礙兒童口腔動作障礙的形成與處理」，職能治療學會雜誌，第7卷，頁61-72，民國78年。
陳志嘉，「口腔健康促進之介入對高齡者口腔機能及生活品質的成效-以苗栗縣為例」，高雄醫學大學，碩士在職專班學位論文，民國105年。
陳昭明，深度學習 最佳入門邁向AI專題實戰，深智數位股份有限公司，台北，民國111年6月。
衛生福利部統計處，身心障礙統計專區，https://dep.mohw.gov.tw/dos/cp-5224-62359-113.html ，2024。
蕭素燕、林嘉德、鄭元凱、郭憲文、蔡銘修，「合併口運動功能異常與構音障礙兒童口功能訓練之成效」，Mid-Taiwan Journal of Medicine，第9卷，第s_1期，頁44-52，民國93年。
戴邦地，「基於深度學習之復健動作辨識系統」，國立中央大學，碩士論文，民國106年。