臺灣博碩士論文加值系統

本文重點研究圖像分類問題，提出了一種半監督學習方法，用於處理只有少量標記數據但有大量未標記數據的情況。半監督學習是一個重要的機器學習研究問題，因為它結合了標記數據和未標記數據來學習預測模型。此外，深度學習在許多應用領域已經顯示出有希望的結果，但深度學習的一個重要要求是在模型訓練期間使用大量的訓練數據。深度學習的成功激勵我們使用深度學習技術來開發所提出的半監督學習算法。這項工作建議學習深度學習的嵌入空間，以便投射到新空間的數據可以更容易分開。為了充分利用未標記的數據，我們使用自我訓練技術和少量學習架構來開發模型。在實現中，幾射學習架構是三重網絡，它通過給出不同和相似對的三元組來學習距離函數。所提出的方法包括以下步驟。首先，我們在三聯網絡上進行監督學習並獲得良好的嵌入。我們將獲得的嵌入轉移到分類器。其次，我們使用自我訓練將未標記的數據迭代地包含到模型中並增加列車集的數據大小。由於該方法可以有效地使用未標記的數據，因此可以減少準備標記數據的工作量並減少深度學習技術的使用障礙。

This thesis focuses on image classification problem, and propose a semi-supervised learning method to deal with the situations where only a few labeled data but enormous unlabeled data are available. Semi-supervised learning is an important machine learning research problem as it combines labeled data as well as unlabeled data to learn a predictive model. Moreover, deep learning has shown promising results in many application domains, but one important requirement for deep learning is to use enormous training data during model training. The success of deep learning inspires us to use deep learning technique to develop the proposed semi-supervised learning algorithm. This work proposes to learn an embedding space with deep learning, so that the data projected onto the new space could be easier separated. To fully utilize unlabeled data, we use self-training technique and few-shot learning architecture to develop the model. In the implementation, the few-shot learning architecture is triplet network, which learns distance functions by giving a triplet of dissimilar and similar pairs. The proposed method comprises the following steps. First, we perform supervised learning on triplet network and obtain good embeddings. We transfer the obtained embedding to the classifier. Second, we use self-training to include unlabeled data into the model iteratively and increase data size of train set. Since this method can use unlabeled data effectively, it is possible to reduce the effort for preparation of labeled data and to reduce the barriers to the usage of deep learning technology.

中文提要 i
英文提要 ii
Acknowledgements iii
1 Introduction 1
1.1 Background................................... 1
1.2 Objective .................................... 3
2 Related Work 4
2.1 Few-shotlearning................................ 4
2.2 Semi-SupervisedLearning ........................... 6
2.3 SiameseNetworks................................ 7
2.4 DEEP METRIC LEARNING USING TRIPLET NETWORK . . . . . . . 10
3 Proposed Method 13
3.1 Stage1...................................... 15
3.1.1 Createtriplets ............................. 15
3.1.2 Samplingtriplets............................ 15
3.1.3 Supervisedlearning........................... 18
3.2 Stage2...................................... 18
4 Experiments 21
4.1 Datasets..................................... 21
4.2 EvaluationMetric................................ 21
4.3 ExperimentalSettings ............................. 25
4.3.1 ConvolutionalNeuralNetwork..................... 25
4.3.2 DecisionTreesandRandomForest .................. 26
4.3.3 Thresholdofhigh-con dence ..................... 27
4.4 ExperimentalResults.............................. 27
5 Discussion 29
5.1 ThresholdofHigh-con dence ......................... 29
5.2 E ectivenessofEmbeddingModel....................... 30
5.3 FixedEmbeddingversusNon- xedEmbedding . . . . . . . . . . . . . . . 31
5.4 Comparison of other semi-supervised learning method . . . . . . . . . . . . 32
6 Conclusion ...................................34
References 35

[1] Yi Sun et al. “Deep learning face representation by joint identi cation-veri cation”. In: Advances in neural information processing systems. 2014, pp. 1988–1996.
[2] Tom Young et al. “Recent trends in deep learning based natural language process- ing”. In: ieee Computational intelligenCe magazine 13.3 (2018), pp. 55–75.
[3] Dong Yu and Li Deng. “Deep learning and its applications to signal and information processing [exploratory dsp]”. In: IEEE Signal Processing Magazine 28.1 (2011), pp. 145–154.
[4] Wei Liu et al. “Ssd: Single shot multibox detector”. In: European conference on computer vision. Springer. 2016, pp. 21–37.
[5] Xavier Carreras and Lluis Marquez. “Boosting trees for anti-spam email ltering”. In: arXiv preprint cs/0109015 (2001).
[6] Kwok-Wai Cheung et al. “Mining customer product ratings for personalized mar- keting”. In: Decision Support Systems 35.2 (2003), pp. 231–243.
[7] Nitish Srivastava et al. “Dropout: a simple way to prevent neural networks from over tting”. In: The Journal of Machine Learning Research 15.1 (2014), pp. 1929– 1958.
[8] David Yarowsky. “Unsupervised word sense disambiguation rivaling supervised meth- ods”. In: 33rd annual meeting of the association for computational linguistics. 1995.
[9] Avrim Blum and Tom Mitchell. “Combining labeled and unlabeled data with co- training”. In: Proceedings of the eleventh annual conference on Computational learn- ing theory. ACM. 1998, pp. 92–100.
[10] Kamal Nigam et al. “Text classi cation from labeled and unlabeled documents using EM”. In: Machine learning 39.2-3 (2000), pp. 103–134.
35
[11] Xiaojin Zhu, Zoubin Ghahramani, and John D La erty. “Semi-supervised learning using gaussian elds and harmonic functions”. In: Proceedings of the 20th Interna- tional conference on Machine learning (ICML-03). 2003, pp. 912–919.
[12] Durk P Kingma et al. “Semi-supervised learning with deep generative models”. In: Advances in neural information processing systems. 2014, pp. 3581–3589.
[13] Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov. “Siamese neural networks for one-shot image recognition”. In: ICML Deep Learning Workshop. Vol. 2. 2015.
[14] Elad Ho er and Nir Ailon. “Deep metric learning using triplet network”. In: Inter- national Workshop on Similarity-Based Pattern Recognition. Springer. 2015, pp. 84– 92.
[15] Flood Sung et al. “Learning to compare: Relation network for few-shot learning”. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018, pp. 1199–1208.
[16] Oriol Vinyals et al. “Matching networks for one shot learning”. In: Advances in neural information processing systems. 2016, pp. 3630–3638.
[17] Chelsea Finn, Pieter Abbeel, and Sergey Levine. “Model-agnostic meta-learning for fast adaptation of deep networks”. In: Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org. 2017, pp. 1126–1135.
[18] Jake Snell, Kevin Swersky, and Richard Zemel. “Prototypical networks for few-shot learning”. In: Advances in Neural Information Processing Systems. 2017, pp. 4077– 4087.
[19] Yongqin Xian et al. “Zero-shot learning-a comprehensive evaluation of the good, the bad and the ugly”. In: IEEE transactions on pattern analysis and machine intelligence (2018).
[20] Xiaojin Zhu and Andrew B Goldberg. “Introduction to semi-supervised learning”. In: Synthesis lectures on arti cial intelligence and machine learning 3.1 (2009), pp. 1–130.
[21] Andrew Goldberg et al. “Multi-manifold semi-supervised learning”. In: Arti cial Intelligence and Statistics. 2009, pp. 169–176.
36
[22] Jane Bromley et al. “Signature veri cation using a” siamese” time delay neural network”. In: Advances in neural information processing systems. 1994, pp. 737– 744.
[23] Brenden M Lake, Ruslan R Salakhutdinov, and Josh Tenenbaum. “One-shot learn- ing by inverting a compositional causal process”. In: Advances in neural information processing systems. 2013, pp. 2526–2534.
[24] Online tiplets. url: https://omoindrot.github.io/triplet-loss.
[25] Confusion Matric in Machine learning. url: https://www.geeksforgeeks.org/
confusion-matrix-machine-learning/.
[26] scikit-learn. url: https://scikit-learn.org/stable/.
[27] Yann LeCun et al. “Handwritten digit recognition with a back-propagation net- work”. In: Advances in neural information processing systems. 1990, pp. 396–404.
[28] Improved GAN Pytorch. url: https://github.com/eli5168/improved_gan_ pytorch.
[29] Bruno Lecouat et al. Semi-Supervised Learning With GANs: Revisiting Manifold Regularization. 2018. url: https://openreview.net/forum?id=Hy5QRt1wz.