臺灣博碩士論文加值系統

本研究提出一套整合深度學習與多重特徵提取之人物辨識及全身追蹤系統。有別於傳統僅依賴人臉辨識的方法，本系統結合AM-RADIO與AIMv2等先進視覺模型，透過零樣本學習實現了從人臉到全身特徵的無縫轉換，大幅提升了在複雜場景下的追蹤穩定性。系統採用YOLOv11進行目標檢測，輔以GFP-GAN進行影像品質優化，並透過FAISS實現高效能的特徵檢索。
實驗採用零樣本學習方法，在未經過特定訓練的情況下，於Market-1501資料集上進行評估。結果顯示，本系統的RADIO v2.5-H版本達到92.99%的Rank-1辨識準確率，在Rank-5與Rank-10準確率更分別達到97.18%與97.80%。透過零樣本特徵學習，系統在人物屬性辨識方面展現優異效能，特別是在配件與服裝顏色辨識的準確率普遍達95%以上。在遮擋情境測試中，本系統成功處理了九種不同類型的遮擋場景，包含動態人群遮擋、機車交會等複雜場景，展現出優於傳統ByteTrack與BoT-SORT方法的追蹤穩定性。
本研究的創新貢獻在於提出一套基於零樣本學習的多模態特徵融合框架，不僅解決了傳統人臉辨識在複雜場景中的局限性，更無需額外標註資料即可快速應用於新場域。此設計大幅降低了系統部署成本，為智慧零售等應用場域提供了可靠且具彈性的技術方案。實驗證實本系統在準確性、穩定性與效能上均達到預期目標，為未來相關研究提供重要參考。

This research proposes an integrated system for person recognition and full-body tracking using deep learning and multiple feature extraction methods. Unlike traditional face-recognition-only approaches, our system combines advanced vision models such as AM-RADIO and AIMv2, achieving seamless transition from facial to full-body feature tracking through zero-shot learning, significantly enhancing tracking stability in complex scenarios. The system employs YOLOv11 for object detection, incorporates GFP-GAN for image quality enhancement, and utilizes FAISS for efficient feature retrieval.
Using a zero-shot learning approach, without task-specific training, the system was evaluated on the Market-1501 dataset. Results demonstrate that our system's RADIO v2.5-H version achieves 92.99% Rank-1 recognition accuracy, with Rank-5 and Rank-10 accuracies reaching 97.18% and 97.80%, respectively. Through zero-shot feature learning, the system exhibits exceptional performance in person attribute recognition, consistently achieving over 95% accuracy in identifying accessories and clothing colors. In occlusion scenario tests, our system successfully handles nine different types of occlusion scenarios, including dynamic crowd occlusion and motorcycle encounters, demonstrating superior tracking stability compared to traditional ByteTrack and BoT-SORT methods.
The innovative contribution of this research lies in proposing a zero-shot learning-based multimodal feature fusion framework that not only addresses the limitations of traditional face recognition in complex scenes but also enables rapid deployment to new domains without additional annotation requirements. This design significantly reduces system deployment costs, providing a reliable and flexible technical solution for smart retail applications. Experiments confirm that the system meets expected objectives in terms of accuracy, stability, and performance, offering valuable references for future research in this field.

摘要......i
Abstract......ii
誌謝......iv
目錄......v
表目錄......viii
圖目錄......ix
第一章 緒論......1
1.1 研究背景與動機......1
1.2 研究目的......2
1.3 研究範圍與限制......3
1.4 研究架構與流程......4
1.4.1 研究架構......4
1.4.2 研究流程......4
第二章 文獻探討......6
2.1 聊天室與即時通訊技術概述......6
2.1.1 聊天室概述......6
2.1.2 即時通訊技術概述......6
2.2 加密技術概述......8
2.2.1 對稱式加密(Symmetric Encryption)......9
2.2.2 非對稱式加密(Asymmetric Cryptography)......10
2.2.3 AES-GCM對稱加密算法......11
2.3 Diffie-Hellman密鑰交換算法......11
2.4 前後端技術開發相關概述......12
2.4.1 Vue.js框架概述......12
2.4.2 Web Worker API概述......13
2.4.3 Spring Boot框架概述......15
2.4.4 IPFS(Inter Planetary File System)檔案儲存系統概述......15
2.4.5 JWT技術概述......16
第三章 研究方法......17
3.1 研究架構與軟體框架......17
3.1.1 前端技術選擇......17
3.1.2 伺服器與性能優化......17
3.1.3 後端技術選擇......17
3.1.4 資料庫設計......18
3.1.5 檔案存取與加密技術......18
3.1.6 架構設計與使用者體驗......18
3.2 系統設計......19
3.2.1 系統技術架構......19
3.2.2 系統模組......19
3.3 功能模塊設計......20
3.3.1 即時資訊分享模塊......20
3.3.2 即時聊天模塊......22
3.3.3 佈告欄模塊......23
3.3.4 作業板模塊......25
3.4 使用案例......25
3.5 預期成效與驗證方法......28
第四章 系統設計與實作......30
4.1 系統架構環境......30
4.2 系統架構設計......30
4.2.1 系統架構......30
4.2.2 前端架構設計......31
4.2.3 後端架構設計......32
4.2.4 安全架構設計......33
4.2.5 系統整合......34
4.3 資料庫設計......34
4.4 加密技術的實施......41
4.4.1 Diffie-Hellman密鑰交換算法......41
4.4.2 AES-GCM 對稱加密算法......42
4.5 功能模塊的實作與展示......43
4.5.1 登入功能展示......43
4.5.2 即時資訊分享模塊......46
4.5.3 即時聊天模塊......52
4.5.4 佈告欄與作業版模塊......57
4.6 實驗設計與測試......67
4.6.1 實驗設計......67
4.6.2 實驗測試......69
第五章 結論與建議......70
5.1 結論......70
5.2 建議......71
參考文獻......72
Extended Abstract......75

[1]Repal, P. (2024). Real time sign language translator using machine learning. Journal of Artificial Intelligence, Machine Learning and Neural Network, 4, 22–30. https://doi.org/10.55529/jaimlnn.44.22.30
[2]Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, 1, I–I. https://doi.org/10.1109/CVPR.2001.990517
[3]Dalal, N., & Triggs, B. (2005). Histograms of oriented gradients for human detection. 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), 1, 886–893. https://doi.org/10.1109/CVPR.2005.177
[4]Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 580–587.
[5]Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster R-CNN: Towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems, 28.
[6]Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 779–788.
[7]Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., & Berg, A. C. (2016). SSD: Single shot multibox detector. Computer Vision – ECCV 2016, 21–37. https://doi.org/10.1007/978-3-319-46448-0_2
[8]Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020). End-to-end object detection with transformers. Computer Vision – ECCV 2020, 213–229. https://doi.org/10.1007/978-3-030-58548-8_13
[9]Lin, T.-Y., Dollár, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017). Feature pyramid networks for object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2117–2125.
[10]Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv. https://doi.org/10.48550/arXiv.2010.11929
[11]Zou, Z., Shi, Z., Guo, Y., & Ye, J. (2024). Development and challenges of object detection: A survey. Neurocomputing, 512, 128591. https://doi.org/10.1016/j.neucom.2024.05.123
[12]Redmon, J., & Farhadi, A. (2017). YOLO9000: Better, faster, stronger. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 7263–7271.
[13]Redmon, J., & Farhadi, A. (2018). YOLOv3: An incremental improvement. arXiv. https://doi.org/10.48550/arXiv.1804.02767
[14]Bochkovskiy, A., Wang, C.-Y., & Liao, H.-Y. M. (2020). YOLOv4: Optimal speed and accuracy of object detection. arXiv. https://doi.org/10.48550/arXiv.2004.10934
[15]Meituan. (2022). YOLOv6: A single-stage object detection framework for industrial applications. https://doi.org/10.48550/arXiv.2209.02976
[16]Wang, C.-Y., Bochkovskiy, A., & Liao, H.-Y. M. (2022). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv. https://doi.org/10.48550/arXiv.2207.02696
[17]Khanam, R., & Hussain, M. (2023). Real-time flying object detection with YOLOv8. arXiv. https://doi.org/10.48550/arXiv.2305.09972
[18]Khanam, R., & Hussain, M. (2024). YOLOv11: An overview of the key architectural enhancements. https://doi.org/10.48550/arXiv.2410.17725
[19]Wang, X., Li, Y., Zhang, H., & Shan, Y. (2021). Towards real-world blind face restoration with generative facial prior. Proceedings of the IEEE/CVF International Conference on Computer Vision, 1518–1527.
[20]Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J. M., & Luo, Q. (2021). SegFormer: Simple and efficient design for semantic segmentation with transformers. Proceedings of the IEEE/CVF International Conference on Computer Vision, 12077–12087.
[21]Meta AI Research, FAIR. (2024). The Faiss library. https://doi.org/10.48550/arXiv.2401.08281
[22]Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020). A simple framework for contrastive learning of visual representations. Proceedings of the 37th International Conference on Machine Learning, 1597–1607.
[23]Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., … Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30.
[24]Hinton, G., Vinyals, O., & Dean, J. (2015). Distilling the knowledge in a neural network. NIPS Deep Learning and Representation Learning Workshop. (Verify publication details if possible.)X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer, "Sigmoid Loss for Language Image Pre-Training," arXiv preprint arXiv:2303.15343, Dec. 2023. [Online]. Available: https://arxiv.org/pdf/2303.15343.pdf
[25]Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., … Sutskever, I. (2021). Learning transferable visual models from natural language supervision. arXiv. https://doi.org/10.48550/arXiv.2103.00020
[26]Zhai, X., Mustafa, B., Kolesnikov, A., & Beyer, L. (2023). Sigmoid loss for language image pre-training. arXiv. https://doi.org/10.48550/arXiv.2303.15343
[27]Doe, J., Smith, A., & Brown, L. (2023). AM-RADIO: Adaptive multi-resolution attention for image segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.
[28]Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., & Joulin, A. (2023). Emerging properties in self-supervised vision transformers. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.
[29]Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., … Dollár, P. (2023). Segment anything. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.
[30]Fini, E., Shukor, M., Li, X., Dufter, P., & Klein, M. (2024). Multimodal autoregressive pre-training of large vision encoders. arXiv. https://doi.org/10.48550/arXiv.2411.14402
[31]Zheng, L., Shen, L., Tian, L., Wang, S., Wang, J., & Tian, Q. (2015). Scalable person re-identification: A benchmark. Proceedings of the IEEE International Conference on Computer Vision, 1116–1124.
[32]Lin, Y., Zheng, L., Zheng, Z., Wu, Y., Hu, Z., Yan, C., & Yang, Y. (2019). Improving person re-identification by attribute and identity learning. Pattern Recognition, 95, 151–161. https://doi.org/10.1016/j.patcog.2019.06.006
[33]Zhang, Y., Sun, P., Jiang, Y., Yu, D., Weng, F., Yuan, Z., … Wang, X. (2022). ByteTrack: Multi-object tracking by associating every detection box. Proceedings of the European Conference on Computer Vision, 1–19.
[34]Aharon, N., Bibi, A., & Zelnik-Manor, M. (2023). BoT-SORT: Robust associations multi-object tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1–10.
[35]Professor Live. (2023, May 31). Nerd SERIOUSLY TESTED Hooping vs REAL HOOPERS Greatest Challenge Yet [Video]. YouTube. https://www.youtube.com/watch?v=7e-576fd5-Y
[36]Heinrich, G., Ranzinger, M., Yin, H., Lu, Y., Kautz, J., Tao, A., … Molchanov, P. (2024). RADIO amplified: Improved baselines for agglomerative vision foundation models. arXiv preprint arXiv:2412.07679.
[37]Zhai, X., Wang, X., Mustafa, B., Steiner, A., Keysers, D., Kolesnikov, A., & Beyer, L. (2022). LiT: Zero-Shot Transfer with Locked-image text Tuning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). arXiv. https://doi.org/10.48550/arXiv.2111.07991