跳到主要內容

臺灣博碩士論文加值系統

(44.201.97.224) 您好!臺灣時間:2024/04/18 02:53
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果 :::

詳目顯示

我願授權國圖
: 
twitterline
研究生:江彥均
研究生(外文):Yan-Jun Jiang
論文名稱:基於自監督式學習之無源域域適應方法
論文名稱(外文):Source-Free Domain Adaptation Via Self-Supervised Learning
指導教授:林慧珍林慧珍引用關係
指導教授(外文):Hwei-Jen Lin
口試委員:顏淑惠凃瀞珽
口試日期:2023-07-18
學位類別:碩士
校院名稱:淡江大學
系所名稱:資訊工程學系碩士班
學門:工程學門
學類:電資工程學類
論文種類:學術論文
論文出版年:2023
畢業學年度:111
語文別:中文
論文頁數:57
中文關鍵詞:自監督式學習無源域域適應偽標籤
外文關鍵詞:Self-Supervised learningSource free domain adaptationPseudo label
DOI:10.6846/tku202300655
相關次數:
  • 被引用被引用:0
  • 點閱點閱:30
  • 評分評分:
  • 下載下載:0
  • 收藏至我的研究室書目清單書目收藏:0
非監督式域適應(Unsupervised Domain Adaptation, UDA)旨在利用已標籤的源域資料集學習到的知識來解決在未標籤目標域中的任務,以往的UDA方法在訓練過程中需要使用到源域資料,但由於個資法源域資料不易取得與資料傳輸成本等考量,啟發了無源域域適應(source-free domain adaptation),即訓練過程中需要使用到源域資料。本論文即著重於無源域域適應的研究,充分利用預訓練好的源域模型執行域適應訓練。源域模型包含包含特徵抽取模組(feature encoding module)與分類器模組(classifier module)。目標域模型則以源域模型做初始化,訓練時分類器模組則固定不動,只調整特徵抽取模組裡的參數。網路骨架(backbone)以ResNet50為基礎並加入Dropblock與IBN-Net等機制,使其有更好的正則化能力與穩健性,從而獲得更佳的域不變特徵,並引入旋轉角度分類器來共同訓練以強化目標域的分類效能。最後將目標域樣本一一輸入當前的目標域模型,以其輸出的熵(entropy)做為樣本之信心度評量標準,低熵值之樣本視為具高信心度,並給予標籤;而高熵值之樣本視為具低信心度,不給予標籤。之後以這些樣本作為訓練樣本,以半監督式訓練(semi-supervised training)方式再次訓練整個模型。
Unsupervised Domain Adaptation (UDA) aims to leverage knowledge learned from a labeled source domain dataset to address tasks in an unlabeled target domain. Traditional UDA methods typically require the use of source domain data during the training process. However, due to concerns such as the difficulty of acquiring source domain data and data transfer costs under privacy regulations, the concept of Source-Free Domain Adaptation has emerged, which means that source domain data is not required during the training process. This paper focuses specifically on the research of source-free domain adaptation, making full use of a pre-trained source domain model for domain adaptation training. The source domain model consists of a feature extraction module and a classifier module. The target domain model is initialized using the source domain model, and during training, the classifier module remains fixed while only adjusting the parameters within the feature extraction module. The network backbone is based on ResNet50 and incorporates mechanisms like Dropblock and IBN-Net to enhance its regularization capability and robustness, thus obtaining better domain-invariant features. Additionally, a rotation angle classifier is introduced for training together to enhance the classification performance of the target domain. The target domain samples are individually fed into the current target domain model. The entropy of the model's output serves as a confidence measure for each sample. Low entropy values indicate high confidence, leading to label assignment, while high entropy values suggest low confidence and result in no labels being assigned. These samples are then used as training data to retrain the entire model through a semi-supervised training mechanism.
目錄
目錄...IV
圖目錄...V
表目錄...VI
第一章 導論...1
第二章 相關研究...3
2.1 無監督式域適應...3
2.2 偽標籤...4
2.3 丟棄區塊...4
2.4 批次標準化與實例標準化網路...4
第三章 研究方法...6
3.1 生成源域模型...9
3.2 資料擴增與插值一致性...9
3.3 源域假設遷移...10
3.4 自監督式學習...12
3.5 旋轉分類器...13
3.6 半監督學習...13
第四章 實驗結果...18
4.1 實驗設定...18
4.2 準確率比較...19
4.3 消融實驗...20
第五章 結論...22
參考文獻...23
附錄: 英文論文...27

圖目錄
圖一 無源域域適應架構圖...7
圖二 訓練系統骨幹部分區塊...8
圖三 目標域模型訓練演算法...16
圖四 半監督式訓練演算法...17

表目錄
表一 在數字資料集測試結果比較...20
表二 在物件資料集測試結果比較...20
表三 消融實驗...21
[1] W. Wang, H. Li, Z. Ding, and Z. Wang, "Rethink Maximum Mean Discrepancy for Domain Adaptation," arXiv e-prints arXiv: 2007.00689, 2020.
[2] A. Gretton, K. M. Borgwardt, M. J. Rasch, B. Schölkopf, & A. Smola, "A kernel two-sample test, " The Journal of Machine Learning Research, 13(1), pp. 723-773, 2012.
[3] M. Long, H. Zhu, J. Wang, and M. I. Jordan, “Unsupervised Domain Adaptation with Residual Transfer Networks,” Workshop on Neural Information Processing Systems (NIPS), pp. 136–144, 2016.
[4] Yaroslav Ganin and Victor Lempitsky, "Unsupervised Domain Adaptation by Backpropagation," International conference on Machine Learning, pp. 1180-1189, 2015.
[5] Z. Cao, M. Long, J. Wang, and M. I. Jordan, "Partial Transfer Learning with Selective Adversarial Networks," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2724-2732, 2018.
[6] Youngeun Kim, Donghyeon Cho, and Sungeun Hong, "Towards Privacy-Preserving Domain Adaptation," IEEE Signal Processing Letters, Vol. 27, pp. 1675-1679, 2020.
[7] Z. Qiu, Y. Zhang, H. Lin, S. Niu, Y. Liu, Q. Du, and M. Tan, "Source-Free Domain Adaptation via Avatar Prototype Generation and Adaptation," arXiv e-prints, arXiv-2106, 2021.
[8] Haifeng Xia, Zhao Handong, and Ding Zhengming Ding, "Adaptive Adversarial Network for Source-Free Domain Adaptation," in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9010-9019, 2021.
[9] X. Pan, P. Luo, J. Shi, and X. Tang, "Two at Once: Enhancing Learning and Generalization Capacities via Ibn-Net," in Proceedings of the European Conference on Computer Vision (ECCV), pp. 464-479, 2018.
[10] Raghuraman Gopalan, Li Ruonan, and Chellappa Rama, "Domain Adaptation for Object Recognition: An Unsupervised Approach," 2011 International Conference on Computer Vision. IEEE, pp. 999-1006, 2011.
[11] M. Khodabandeh, A. Vahdat, M. Ranjbar, and W. G. Macready, "A Robust Learning Approach to Domain Adaptive Object Detection," in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 480-490, 2019.
[12] G. Matasci, M. Volpi, M. Kanevski, L. Bruzzone, and D. Tuia, "Semi-Supervised Transfer Component Analysis for Domain Adaptation in Remote Sensing Image Classification," IEEE Transactions on Geoscience and Remote Sensing, 53.7, pp. 3550-3564, 2015.
[13] Dong-Hyun Lee, "Pseudo-Label: The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks," Workshop on challenges in representation learning, pp. 896, 2013.
[14] Vincent Tinto, "Dropout from Higher Education: A Theoretical Synthesis of Recent Research," Review of educational research, 45.1, pp.89-125, 1975.
[15] Golnaz Ghiasi, Tsung-Yi Lin, and Quoc V. Le, "Dropblock: A Regularization Method for Convolutional Networks," arXiv:1810.12890v1 [cs.CV], 2018.
[16] K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[17] P. Chen, S. Liu, H. Zhao, and J. Jia, "Gridmask Data Augmentation. " arXiv e-prints, arXiv-2001, 2020.
[18] D. Berthelot, N. Carlini, I. Goodfellow, N. Papernot, A. Oliver, and C. A. Raffel, "Mixmatch: A Holistic Approach to Semi-Supervised Learning," arXiv:1905.02249v2 [cs.LG], 2019.
[19] H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lopez-Paz, "Mixup: Beyond Empirical Risk Minimization," arXiv e-prints, arXiv-1710, 2017.
[20] Rafael Müller, Kornblith Simon and E. Hinton Geoffrey, "When Does Label Smoothing Help?" Advances in Neural Information Processing Systems 32, 2019.
[21] V. Verma, K. Kawaguchi, A. Lamb, J. Kannala, A. Solin, Y. Bengio, and D. Lopez-Paz, "Interpolation Consistency Training for Semi-Supervised Learning," Neural Networks, Vol. 145, pp. 90-106, 2022.
[22] Samarth Mishra, Saenko Kate, and Saligrama Venkatesh, "Surprisingly Simple Semi-Supervised Domain Adaptation with Pretraining and Consistency," arXiv e-prints, arXiv-2101, 2021.
[23] Jian Liang, Hu Dapeng, and Feng Jiashi, "Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation," in Proceedings of Machine Learning Research, pp. 6028-6039, 2020.
[24] Andreas Krause, Perona Pietro, and Gomes Ryan, "Discriminative Clustering by Regularized Information Maximization," Advances in neural information processing systems, Vol. 23, pp. 1-9, 2010.
[25] M. Caron, P. Bojanowski, A. Joulin, and M. Douze, "Deep Clustering for Unsupervised Learning of Visual Features," in Proceedings of the European Conference on Computer Vision (ECCV), pp. 132-149, 2018.
[26] Bram Wallace and Hariharan Bharath, "Extending and Analyzing Self-Supervised Learning across Domains," in Proceedings 16th European Conference, Glasgow, UK, August 23-28, pp. 717-734, 2020.
[27] E. Tzeng, J. Hoffman, K. Saenko, and T. Darrell, "Adversarial Discriminative Domain Adaptation," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7167-7176, 2017.
[28] K. Saito, Y. Ushiku, T. Harada, and K. Saenko, "Adversarial Dropout Regularization," International Conference on Learning Representations, 2018.
[29] J. Hoffman, E. Tzeng, T. Park, J. Y. Zhu, P. Isola, K. Saenko, and T. Darrell, "Cycada: Cycle-Consistent Adversarial Domain Adaptation," in Proceedings of Machine Learning Research, pp. 1989-1998, 2018.
[30] C. Y. Lee, T. Batra, M. H. Baig, and D. Ulbricht, "Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10285-10295, 2019.
[31] Y. Zhang, T. Liu, M. Long, and M. Jordan, "Bridging Theory and Algorithm for Domain Adaptation," in Proceedings of Machine Learning Research, pp. 7404-7413, 2019
[32] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-Based Learning Applied to Document Recognition," in Proceedings of the IEEE, 86(11), pp. 2278-2324, 1998.
[33] Jonathan J. Hull, "A Database for Handwritten Text Recognition Research," IEEE Transactions on pattern analysis and machine intelligence, Vol. 16, No. 5, pp. 550-554, 1994.
[34] torchvision.datasets.svhn — Torchvision 0.15 documentation (pytorch.org)
[35] K. Saenko, B. Kulis, M. Fritz, and T. Darrell, "Adapting Visual Category Models to New Domains," 11th European Conference on Computer Vision, Greece, September 5-11, 2010.
連結至畢業學校之論文網頁點我開啟連結
註: 此連結為研究生畢業學校所提供,不一定有電子全文可供下載,若連結有誤,請點選上方之〝勘誤回報〞功能,我們會盡快修正,謝謝!
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top