臺灣博碩士論文加值系統

為了完成人機互動的任務，如何確實地偵測到指定人士(Specific Human (SH))就變得非常重要與關鍵。在本論文中，研發了整合Single-Shot Detection (SSD)、FaceNet (FN)及核化相關濾波器(Kernelized Correlation Filter (KCF))的深度學習法則：SSD-FN-KCF，以達成搜尋(指定)人士及追隨指定人士的任務。實驗一開始，在8公尺的距離內，使用SSD搭配輸出解析度320×240的RGB-D鏡頭進行人士偵測，之後命令全方位移動機器人(ODMR)移動至3公尺附近，使得深度檢測器可以準確地偵測到人體姿態與深度訊息。然後命令ODMR移動至1公尺和相對於光軸0度的方位，並藉由FaceNet辨識他(她)是否為SH。為了減少FaceNet的運算時間並擴展性能以持續追蹤指定人士，並使用KCF追蹤SH以達成追隨之人機互動。緊接著，根據影像處理的資訊，以基於影像的適應有限時間之分層約束控制 (IB-AFTHCC) 達成ODMR的搜尋或追蹤(指定)人士所需的姿態。最後，藉由比較SH與ODMR之間的實驗，驗證本論文所提出的控制法則的效能與強健性。

To fulfill the tasks of human-robot interactions, how to detect the specific human (SH) becomes paramount. In this paper, the deep learning approach：SSD-FN-KCF by the integration of Single-Shot Detection(SSD), FaceNet(FN), and Kernelized Correlation Filter (KCF) is developed. From the outset, the SSD is employed to detect the human up to 8m using RGB-D camera with the resolution of After that, the omnidirectional mobile robot (ODMR) is commanded to the neighborhood of 3.0m such that the depth image can accurately estimate the detected human’s pose. Then the ODMR is commanded to the vicinity of 1.0m and 0 with respect to the optical axis to identify whether he/she is the SH by FaceNet. To reduce the computation time of FaceNet and extend the tracking of the SH, the KCF accomplished the goal for the human-robot interactions (e.g., human following). Based on the information of image processing, the required pose for searching or tracking (specific) human is also accomplished by the ODMR with the image-based adaptive finite-time hierarchical constraint control (IB-AFTHCC). Finally, compared experiments between SH and ODMR validate the effectiveness and robustness of the proposed control.

摘要 i
Abstract ii
目錄 iii
圖目錄 v
表目錄 viii
第一章 導論與文獻回顧 1
1.1 導論 1
1.2 文獻回顧 3
第二章 系統建構與任務陳述 6
2.1 系統建構 6
2.2 任務陳述 9
2.2.1 人士偵測 11
2.2.2 人士趨近 12
2.2.3 人臉辨識 12
2.2.4 人士追隨 13
第三章 SSD-FN-KCF深度學習法則 14
3.1 SSD人士偵測 14
3.1.1 預測 16
3.1.2 難分樣本挖掘 17
3.1.3 損失函數 18
3.1.4 性能測試 21
3.2 FaceNet指定人臉辨識 21
3.2.1 深度架構 23
3.2.2 L2范數歸一化 23
3.2.3 崁入 23
3.2.4 三元損失 24
3.2.5 歐式距離比較 26
3.2.6 性能測試 27
3.3 KCF追蹤器 28
3.3.1 循環矩陣 30
3.3.2 線性回歸與脊回歸 30
3.3.3 傅氏空間對角化 31
3.3.4 非線性回歸與核技巧 32
3.3.5 快速檢測 33
3.3.6 性能測試 34
第四章 指定人士與全方位移動機器人之互動 36
4.1 影像基礎的期望姿態 36
4.2 適應有限時間之分層約束控制 38
第五章 實驗結果與分析討論 46
5.1 實驗結果 46
5.2 分析討論 55
第六章 結論與未來建議 56
參考文獻 57

[1]N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 886-893, June 20th-25th, 2005.
[2]W. Chung, H. Kim, Y. Yoo, C. Moon, and J. Park, “The detection and following of human legs through inductive approaches for a mobile robot with a single laser range finder,” IEEE Trans. Ind. Electron., vol. 59, no. 8, pp. 3156–3166, Aug. 2012.
[3]L. Li, S. Yan, X. Yu, Y. K. Tan, and H. Li, “Robust multiperson detection and tracking for mobile service and social robots,” IEEE Trans. Syst., Man, Cybern., Part B, vol. 42, no. 5, pp. 1398-1412, Oct. 2012.
[4]N. Khamsemanan, C. Nattee, and N. Jianwattanapaisarn, “Human identification from freestyle walks using posture-based gait feature,” IEEE Trans. Information Forensics and Security, vol. 13, no. 1, pp. 119-128, Jan. 2018.
[5]J. Nie, L. Huang, W. Zhang, G. Wei, and Z. Wei, “Deep feature ranking for person re-identification,” IEEE Access, to be published, 2019.
[6]L. Chen, M. Li, W. Su, M. Wu, K. Hirota, and W. Pedrycz, “Adaptive feature selection-based AdaBoost-KNN with direct optimization for dynamic emotion recognition in human–robot interaction,” IEEE Trans. Emerging Topics in Computational Intelligence, to be published, 2019.
[7]S. K. Biswas and P. Milanfar, “One shot detection with Laplacian object and fast matrix cosine similarity,” IEEE Trans. Pattern Anal. and Mach. Intell., vol. 38, no. 3, pp. 546-562, Mar. 2016.
[8]S. Duffner and C. Garcia, “Fast pixelwise adaptive visual tracking of non-rigid objects,” IEEE Trans. Image Processing, vol. 26, no. 5, pp. 2368-2380, May 2017.
[9]Y. Liu, Q. Wang, Y. Zhuang, and H. Hu, “A novel trail detection and scene understanding framework for a quadrotor UAV with monocular vision,” IEEE. Sensors Journal, vol. 17, no. 20, pp. 6778- 6787, Oct. 2017.
[10]W. Zhang, Z. Wang, X. Liu, H. Sun, J. Zhou, Y. Liu, W. Gong, “Deep learning-based real-time fine-grained pedestrian recognition using stream processing,” IET Intell. Transp. Syst., vol. 12, no. 7, pp. 602-609, 2018.
[11]M. Liao, B. Shi, and X. Bai, “TextBoxes++: A single-shot oriented scene text detector,” IEEE Trans. Image Processing, vol. 27, no. 8, pp. 3676-3690, Aug. 2018.
[12]F.-J. Chu, R. Xu , and P. A. Vela, “Real-world multiobject, multigrasp detection,” IEEE Robotics and Autom. Lett., vol. 3, no. 4, pp. 3355-3362, Oct. 2018.
[13]C.-L. Hwang and Y. Lee, “Tracking design of an omni- direction autonomous ground vehicle by hierarchical enhancement using fuzzy 2nd order variable structure control,” ASME J. of Dynamic Systems, Measurement and Control, vol. 140, pp. 1-11, Sep. 2018.
[14]C.-L. Hwang, W. H. Hung, and Y. Lee, “Tracking design of omnidirectional drive service robot using hierarchical adaptive finite-time control,” IEEE IECON-2018, Washington D.C., USA, pp. 5680-5685, Oct. 21st -Oct. 23rd, 2018.
[15]F. Schroff, D. Kalenichenko, and J. Philbin, “FaceNet: A unified embedding for face recognition and clustering,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 815-823, June 7th -12th, 2015.
[16]N. Werghi, C. Tortorici, S. Berretti, and A. D. Bimbo, “Boosting 3D LBP-based face recognition by fusing shape and texture descriptors on the mesh,” 2016, IEEE Trans. Information Forensics and Security, vol. 11, no. 5, pp. 964-979, May 2016.
[17]A. Rikhtegar, M., Pooyan, and M. T. Manzuri-Shalmani, “Genetic algorithm- optimised structure of convolutional neural network for face recognition applications,” IET Computer Vision, vol. 10, no. 6, pp. 559-566, 2016.
[18]M. A. Abuzneid, and A. Mahmood, “Enhanced human face recognition using LBPH descriptor, Multi-KNN, and back-propagation neural network,” IEEE Access, vol. 6, pp. 20641-20651, 2018.
[19]H. B. Abebe and C.-L. Hwang, “RGB-D face recognition using LBP with suitable feature dimension of depth image,” IET Cyber Physical Systems: Theory & Applications, to be published, Feb. 2019.
[20]J. F. Henriques, R. Caseiro, P. Martins, and J. Batista , “High-speed tracking with kernelized correlation filters,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 37, no. 3, pp. 583-596, Mar. 2015.
[21]J. Leng and Y. Liu, “Real-time RGB-D visual tracking with scale estimation and occlusion handling,” IEEE Access, vol. 6, pp. 24256-24263, 2018.
[22]Q. Liu, G. Hu, and M. M. Isla, “Fast visual tracking with robustifying kernelized correlation filters,” IEEE Access, vol. 6, pp. 43302-43314, 2018.
[23]S. P. Bharati, Y. Wu, Y. Sui, C. Padgett, and G. Wang, “Real-time obstacle detection and tracking for sense-and -avoid mechanism in UAVs,” IEEE Trans. Intell. Veh., vol. 3, no. 2 , pp. 185-197, Jun. 2018.
[24]W. Tian, L. Chen, K. Zou, and M. Lauer, “Vehicle tracking at nighttime by kernelized experts with channel-wise and temporal reliability estimation,” IEEE Trans. Intell. Trans. Syst., vol. 19, no. 10, pp. 3159-3169, Oct. 2018.
[25]F. Fabrizio and A. D. Luca, “Real-time computation of distance to dynamic obstacles with multiple depth sensors,” IEEE Robotics and Autom. Lett., vol.. 2, no. 1, pp. 56-63, Jan. 2017.
[26]P. Long, W. Liu, and J. Pan, “Deep-learned collision avoidance policy for distributed multiagent navigation,” IEEE Robotics and Autom. Lett., vol.. 2, no. 2, pp. 656-663, Apr. 2017.
[27]C.-L. Hwang and H. H. Huang “Experimental validation for a car-like automated ground vehicle with trajectory tracking, obstacle avoidance, and target approach using hierarchical sliding mode tracking control,” IEEE IECON-2017, pp. 2858-2863, Beijing China, Oct. 29th- Nov. 1st, 2017.
[28]G. Plessen, D. Bernardini, H. Esen, and A. Bemporad, “Spatial-based predictive control and geometric corridor planning for adaptive cruise control coupled with obstacle avoidance,” IEEE Trans. Contr. Syst. Technol., vol. 26, no. 1, pp. 38-50, Jan. 2018.
[29]T. Zhao and R. Nevatia, “Tracking multiple humans in complex situations,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 9, pp. 1208-1221, Sep. 2004.
[30]C. Wojek, S. Walk, S. Roth, K. Schindler, and B. Schiele, “Monocular visual scene understanding: understanding multi-object traffic scenes,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 35, no. 4, pp. 882- 897, Apr. 2013.
[31]A. S. Rao, J. Gubbi, S. Marusic, and M. Palaniswami, “Crowd event detection on optical flow manifolds,” IEEE Trans. Cybern., vol. 46, no. 7, pp. 1524-1537, Jul. 2016.
[32]C.-L. Hwang and Y.-H. Chen, “Fuzzy fixed-time learning control with saturated input, nonlinear switching surface and switching gain to achieve null tracking error,” IEEE Trans. Fuzzy Syst., to be published, 2019.