臺灣博碩士論文加值系統

本論文建立一水下生物異常行為系統，其可協助養殖業者提早發現生物異常行為、啟動早期救援措施、與後期死亡原因之判定，以達到減少人力與生物之耗損、促進養殖、育種選種、生態保育教育之發展。系統開發過程中所建立之資料庫，日後可允許使用者增加生物物種種類、並更新與之對應的生物模板。
至目前為止，關於水下生物行為偵測之研究並不多見，尤其在應用物件偵測[1,2,3,4]技術於生物快速移動和突然運動、姿態的劇烈變化、重疊等相關文獻更是寥寥無幾。針對此一問題，吾人結合物件追蹤[12] 及DTW(Dynamic Time Warping)[11] ，當追蹤目標與其他物件重疊時進行預測，當下一幀原目標出現時將與預測座標進行匹配是為同一目標，則繼續逕行追蹤; 否則，結束追蹤並與行為模板進行DTW計算最短距離以決定行為分類。
本論文提出兩種演算法：姿態分類演算法與行為分類演算法，其具體之實現主要係利用深度學習物件偵測及專家知識來完成。其中，姿態分類演算法係利用深度學習推論模型偵測一移動目標物的複數元件，將其記錄為有向圖[8] ，再搭配專家建議之姿態模板進行匹配以獲得姿態分類。姿態可初分正常與異常，若為異常姿態則繼續進行行為分類。行為分類演算法係將第一筆異常姿態(cue)記錄並利用一追蹤演算法追蹤目標物直至失去目標為止，此時獲得一姿態序列(state sequence)，接著搭配感測器數據對該姿態序列與專家建議之行為模板進行匹配，以獲得該姿態序列之最終行為分類結果，其準確率達86.73%。感測器係用於收集環境參數，針對不同物種收集其姿態、環境參數及行為分類，可以提供養殖業、研究者隨時進行監控，更可以為水下生物建立行為資料庫大數據。

The present thesis aims to establish an underwater biological abnormal behavior system, which is useful for researchers and the breeding industry to discover biological abnormal behavior early, start early rescue measures, and determine the cause of death in the later period, so as to reduce the loss, promote breeding, facilitate breeding selection, as well as ecological conservation education. The database created during the system development process allows users to increase the types of biological species and update the corresponding biological templates in the future.
So far, research on underwater biological behavior detection is rare, especially very few literatures can be found in the application of object detection [1,2,3,4] techniques to catch the rapid movement of organisms and sudden movements, dramatic changes in posture, shadowing and overlap. To solve this problem, we combined object tracking [12] and DTW (Dynamic Time Warping) [11], and finally used template matching to classify and judge the covered objects, which can achieve the purpose of detecting biological anomalies.
In this thesis, two algorithms are proposed: pose algorithm and behavior algorithm. Their implementations are mainly done by deep learning object detection and expert knowledge. Among them, the pose algorithm is to use a deep learning inference model to detect the plural parts of a moving target, record them as a directed graph [8], and then match it with a pose template suggested by experts. The pose can be classified as either normal or abnormal, and if it is abnormal pose, continue to conduct behavior classification. The behavior algorithm records the first abnormal pose (cue) and uses a tracking algorithm to track the target until it loses the target. At this time, a state sequence is obtained and combined with sensor data to match with the expert-suggested templates to obtain the final behavior classification result. Sensors are used to collect environmental parameters. By collecting posture data, environmental parameters, and behavior classification for different species, we can provide monitoring for the breeding industry and researchers at any time, and these data together constitute a set of behavior database big data for underwater creatures.

摘要 I
Abstract II
目次 III
圖目次 IV
表目次 VI
第一章 緒論 1
1.1 研究背景 1
1.2 研究動機與目的 1
1.3 論文架構 2
第二章 文獻回顧 3
2.1 Faster R-CNN 3
2.2 圖論(Graph theory) 5
2.3 動態時間規整 (Dynamic Time Warping, DTW) 5
2.3.1 動態規劃(Dynamic Programming, DP) 6
2.3.2 動態時間規整 (Dynamic Time Warping, DTW) 7
2.4 物件追蹤 9
2.4.1卡爾曼濾波器(Kalman filter) 9
2.4.2 Underwater target tracking via object detection and stereo vision 10
第三章 研究方法 11
3.1物件偵測 13
3.2基於元件相互位置之姿態編碼 14
3.3行為分類演算法 17
3.3.1追蹤演算法 20
第四章 實驗結果及比較 21
4.1實驗環境 22
4.2姿態編碼 23
4.2.1物件偵測 Faster R-CNN 23
4.2.2姿態分類演算法 25
4.3 行為編碼與分類 26
第五章 結論與未來展望 30
參考文獻 31

[1] S. Ren, K. He, R. Girshick and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks, ”Advances in neural information processing systems, pp.91-99, 2015.
[2] J. Uijlings, K. van de Sande, T. Gevers, and A. Smeulders, “Selective search for object recognition, ”IJCV, 2013
[3]K. He, X. Zhang, S. Ren, and J. Sun, “Spatial pyramid pooling in deep convolutional networks for visual recognition, ”in European Conference on Computer Vision (ECCV), 2014.
[4] R. Girshick, “Fast r-cnn, ”Proceedings of the IEEE international conference on computer vision, pp.1440-1448, 2015.
[5] J. Zbontar, Y. LeCun, “Stereo Matching by Training a Convolutional Neural Network to compare Image Patches, ”Computer Vision and Pattern Recognition. arXiv:1510.05970v2, 2016.
[6] M. D. Zeiler and F. Rob, “Visualizing and understanding convolutional networks, ”European conference on computer vision. Springer, Cham, 2014.
[7] K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition, ”Computer Science, 2014.
[8] C. Berge, “Théorie des graphes et ses applications, ”1966.
[9] J. R. Munkres, “Topology, ”Vol. 2. Upper Saddle River. Prentice Hall. 2000.
[10] L. Euler, “Solutio problematis ad geometriam situs pertinentis, ”Commentarii academiae scientiarum Petropolitanae, pp.128-140, 1741.
[11] D. J. Berndt and C. James, “Using dynamic time warping to find patterns in time series. ” KDD workshop. Vol. 10. No. 16. 1994.
[12] Z. W. Pylyshyn, R. W. Storm, “Tracking multiple independent targets: Evidence for a parallel tracking mechanism, ”Spatial Vision. pp. 179–197,1988.
[13] R. E. Kalman, “A new approach to linear filtering and prediction problems, ”pp.35-45, 1960.
[14] S. J. Julier, and K. U. Jeffrey, “Unscented filtering and nonlinear estimation, ”Proceedings of the IEEE 92.3 pp. 401-422. 2004.
[15] E. A. Wan, and V. D. M. Rudolp, “The unscented Kalman filter for nonlinear estimation, ”Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Communications, and Control Symposium (Cat. No. 00EX373). IEEE, 2000.
[16] L. R. Rabiner, “A tutorial on hidden Markov models and selected applications in speech recognition, ”Proceedings of the IEEE, 77(2), pp. 257-286, 1989.
[17] R. J. Huang and J. H. Wang, “Underwater target tracking via object detection and stereo vision, ”in preparation for submission to IEEE Access, 2020.
[18] Y. T. Peng, M. H. Lin, C. L. Tang, and C. H. Wu, “Image Denoising Based on Overlapped and Adaptive Gaussian Smoothing and Convolutional Refinement Networks, ”2019 IEEE Intnl. Symp. on Multimedia (ISM), Vol. 1, pp. 136-1363, 2019.
[19] A. Bochkovskiy, C. Y. Wang, and H. Y. Mark Liao, “YOLOv4: Optimal Speed and Accuracy of Object Detection, ”arXiv:2004.10934, 2020.
[20] R. J. Huang, C. Y. Tsao, Y. P. Kuo, Y. C. Lai, C. C. Liu, Z. W. Tu, J. H. Wang, and C. C. Chang, “Fast Visual Tracking based on Convolutional Networks, ”Sensors, Vol. 18, no. 2, 2405, 2018.
[21] V. Niennattrakul, D. Srisai, and C. A. Ratanamahatana, “Shape-based template matching for time series data, ”Knowledge-Based Systems, Vol. 26, pp. 1-8, 2012.
[22] J. Redmon, S. Divvala, R. Girshick and A. Farhadi, “You only look once: Unified, real-time object detection, ”Proceedings of the IEEE conference on computer vision and pattern recognition, pp.779-788, 2016.
[23] J. Redmon and A. Farhadi, “YOLO9000: better, faster, stronger, ”Proceedings of the IEEE conference on computer vision and pattern recognition, pp.7263-7271, 2017.
[24] J. Redmon and A. Farhadi, “Yolov3: An incremental improvement, ”arXiv preprint arXiv:1804.02767, 2018.
[25] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Y. Fu, and A. C. Berg, “Ssd: Single shot multibox detector, ”In European conference on computer vision, Springer, Cham. , pp. 21-37.
[26] K. He, G. Gkioxari, P. Dollár, and R. Girshick. “Mask R-CNN, ”In Proceedings of the International Conference on Computer Vision (ICCV), 2017.
[27] B. Cigdem and F. Robert, “Detecting abnormal fish trajectories using clustered and labeled data, ”2013 IEEE International Conf. on Image Processing (ICIP), 2013.
[28] M. Thida, H. l. Eng, and B. F. Chew, “Automatic Analysis of Fish Behaviors and Abnormality Detection, ”Proceedings of the 11th IAPR Conf. on Machine Vision Applications (IAPR MVA), pp. 8-18, 2009.
[29] Y. C. Lai, “Dynamic Action Recognition Based on Deep Learning, ”National Taiwan Ocean University, Master thesis, 2018.
[30] Y. C. Lai, “Underwater biological abnormal behavior detection based on direction of movement ”, in preparation for submissionuv to IEEE Access, 2020.