臺灣博碩士論文加值系統

極度多標籤分類問題是從極大的標籤空間中設法預測出多個標籤。標籤的數量和稀疏性的問題使得普通模型難以處理極度多標籤分類問題。在本研究中，我們提出了一個處理極度多標籤分類問題的方法。我們的方法可以有效地處理龐大的數據集，不管數據集有多大，實驗證明我們的方法都能有效地處理並預測出結果。此外，現在大多數機器學習演算法都被批評為“黑盒子”問題：模型無法說明它如何決定預測。在我們的方法中，透過特殊的非負參數的限制，我們的方法能夠提供可解釋的解釋。實驗證明，該方法能兼具不錯的預測精度並提供可理解的解釋。

Extreme multi-label learning is to seek most relevant subset of labels from an extreme large labels space. The problem of scalability and sparsity makes extreme multi-label hard to learn. In this paper, we propose a framework to deal with these problems. Our approach allows to deal with enormous dataset efficiently. Moreover, most algorithms nowadays are criticized for “black box” problem, which model cannot provide how it decides to make predictions. Through special non-negative constraint, our proposed approach is able to provide interpretable explanation. Experiments show that our method achieves both high prediction accuracy and understandable explanation.

論文審定書 i
摘要 ii
Abstract iii
List of Figures v
List of Table vi
1. Introduction 1
2. Background and Related Work 3
3. Proposed approach 9
4. Experimental result 14
4.1 Datasets 14
4.2 Evaluation Metrics 15
4.3 Performance Comparison 16
4.4 Interpretable Explanation 19
5. Conclusion 23
6. Reference 24

AnnexML: Approximate Nearest Neighbor Search for Extreme Multi-label Classification. (n.d.). Retrieved June 3, 2019, from https://www.kdd.org/kdd2017/papers/view/annexml-approximate-nearest-neighbor-search-for-extreme-multi-label-classif
Bengio, Y., Courville, A., & Vincent, P. (2012). Representation Learning: A Review and New Perspectives. ArXiv:1206.5538 [Cs]. Retrieved from http://arxiv.org/abs/1206.5538
Bengio, Y., & Delalleau, O. (2011). On the Expressive Power of Deep Architectures. In J. Kivinen, C. Szepesvári, E. Ukkonen, & T. Zeugmann (Eds.), Algorithmic Learning Theory (pp. 18–36). Springer Berlin Heidelberg.
Bhatia, K., Jain, H., Kar, P., Varma, M., & Jain, P. (2015). Sparse Local Embeddings for Extreme Multi-label Classification. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, & R. Garnett (Eds.), Advances in Neural Information Processing Systems 28 (pp. 730–738). Retrieved from http://papers.nips.cc/paper/5969-sparse-local-embeddings-for-extreme-multi-label-classification.pdf
Boutell, M. R., Luo, J., Shen, X., & Brown, C. M. (2004). Learning multi-label scene classification. Pattern Recognition, 37(9), 1757–1771. https://doi.org/10.1016/j.patcog.2004.03.009
Chen, M., Mao, S., & Liu, Y. (2014). Big Data: A Survey. Mobile Networks and Applications, 19(2), 171–209. https://doi.org/10.1007/s11036-013-0489-0
Chen, S.-F., Chen, Y.-C., Yeh, C.-K., & Wang, Y.-C. F. (2017). Order-Free RNN with Visual Attention for Multi-Label Classification. ArXiv:1707.05495 [Cs]. Retrieved from http://arxiv.org/abs/1707.05495
Durand, T., Mehrasa, N., & Mori, G. (2019). Learning a Deep ConvNet for Multi-label Classification with Partial Labels. ArXiv:1902.09720 [Cs]. Retrieved from http://arxiv.org/abs/1902.09720
Hinton, G. E. (2006). Reducing the Dimensionality of Data with Neural Networks. Science, 313(5786), 504–507. https://doi.org/10.1126/science.1127647
Hochreiter, S., & Schmidhuber, J. (1997). Long Short-Term Memory. Neural Comput., 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
Hoyer, P. O., & Hoyer, P. (n.d.). Non-negative Matrix Factorization with Sparseness Constraints. 13.
Jernite, Y., Choromanska, A., & Sontag, D. (2016). Simultaneous Learning of Trees and Representations for Extreme Classification and Density Estimation. ArXiv:1610.04658 [Cs, Stat]. Retrieved from http://arxiv.org/abs/1610.04658
Kalman, D. (n.d.). A Singularly Valuable Decomposition: The SVD of a Matrix. 27.
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet Classification with Deep Convolutional Neural Networks. In F. Pereira, C. J. C. Burges, L. Bottou, & K. Q. Weinberger (Eds.), Advances in Neural Information Processing Systems 25 (pp. 1097–1105). Retrieved from http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf
Lee, D. D., & Seung, H. S. (1999). Learning the parts of objects by non-negative matrix factorization. Nature, 401(6755), 788–791. https://doi.org/10.1038/44565
Lipton, Z. C. (2016). The Mythos of Model Interpretability. ArXiv:1606.03490 [Cs, Stat]. Retrieved from http://arxiv.org/abs/1606.03490
Liu, B., Sadeghi, F., Tappen, M., Shamir, O., & Liu, C. (2013). Probabilistic Label Trees for Efficient Large Scale Image Classification. 2013 IEEE Conference on Computer Vision and Pattern Recognition, 843–850. https://doi.org/10.1109/CVPR.2013.114
Liu, J., Chang, W.-C., Wu, Y., & Yang, Y. (2017). Deep Learning for Extreme Multi-label Text Classification. Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval - SIGIR ’17, 115–124. https://doi.org/10.1145/3077136.3080834
McAuley, J., & Leskovec, J. (2013). Hidden factors and hidden topics: Understanding rating dimensions with review text. Proceedings of the 7th ACM Conference on Recommender Systems - RecSys ’13, 165–172. https://doi.org/10.1145/2507157.2507163
Prabhu, Y., & Varma, M. (2014). FastXML: A fast, accurate and stable tree-classifier for extreme multi-label learning. Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD ’14, 263–272. https://doi.org/10.1145/2623330.2623651
Recipes—BBC Food. (n.d.). Retrieved June 3, 2019, from https://www.bbc.com/food/recipes.
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). “Why Should I Trust You?”: Explaining the Predictions of Any Classifier. ArXiv:1602.04938 [Cs, Stat]. Retrieved from http://arxiv.org/abs/1602.04938
Shlens, J. (n.d.). A Tutorial on Principal Component Analysis. 13.
Sorower, M. S. (2010). A Literature Survey on Algorithms for Multi-label Learning.
Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014). Striving for Simplicity: The All Convolutional Net. ArXiv:1412.6806 [Cs]. Retrieved from http://arxiv.org/abs/1412.6806
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., … Rabinovich, A. (2014). Going Deeper with Convolutions. ArXiv:1409.4842 [Cs]. Retrieved from http://arxiv.org/abs/1409.4842
The Extreme Classification Repository. (n.d.). Retrieved June 3, 2019, from http://manikvarma.org/downloads/XC/XMLRepository.html#Prabhu14
Tsai, C.-P., & Lee, H.-Y. (2018). Adversarial Learning of Label Dependency: A Novel Framework for Multi-class Classification. ArXiv:1811.04689 [Cs, Stat]. Retrieved from http://arxiv.org/abs/1811.04689
Tsoumakas, G., Katakis, I., & Vlahavas, I. (2010). Mining Multi-label Data. In O. Maimon & L. Rokach (Eds.), Data Mining and Knowledge Discovery Handbook (pp. 667–685). https://doi.org/10.1007/978-0-387-09823-4_34
Tsoumakas, G., Katakis, I., & Vlahavas, I. (n.d.). Effective and Efficient Multilabel Classification in Domains with Large Number of Labels. 15.
Tzanetakis, G., & Cook, P. (2002). Musical genre classification of audio signals. IEEE Transactions on Speech and Audio Processing, 10(5), 293–302. https://doi.org/10.1109/TSA.2002.800560
Yang, P., Sun, X., Li, W., Ma, S., Wu, W., & Wang, H. (2018). SGM: Sequence Generation Model for Multi-label Classification. ArXiv:1806.04822 [Cs]. Retrieved from http://arxiv.org/abs/1806.04822
Yen, I. E. H., Huang, X., Dai, W., Ravikumar, P., Dhillon, I., & Xing, E. (2017). PPDsparse: A Parallel Primal-Dual Sparse Method for Extreme Classification. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD ’17, 545–553. https://doi.org/10.1145/3097983.3098083
You, R., Dai, S., Zhang, Z., Mamitsuka, H., & Zhu, S. (2018). AttentionXML: Extreme Multi-Label Text Classification with Multi-Label Attention Based Recurrent Neural Networks. ArXiv:1811.01727 [Cs]. Retrieved from http://arxiv.org/abs/1811.01727
Zeiler, M. D., & Fergus, R. (2014). Visualizing and Understanding Convolutional Networks. In D. Fleet, T. Pajdla, B. Schiele, & T. Tuytelaars (Eds.), Computer Vision – ECCV 2014 (Vol. 8689, pp. 818–833). https://doi.org/10.1007/978-3-319-10590-1_53
Zhang, M., & Zhou, Z. (2014). A Review on Multi-Label Learning Algorithms. IEEE Transactions on Knowledge and Data Engineering, 26(8), 1819–1837. https://doi.org/10.1109/TKDE.2013.39
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016). Learning Deep Features for Discriminative Localization. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2921–2929. https://doi.org/10.1109/CVPR.2016.319