臺灣博碩士論文加值系統

近年來隨著閱讀習慣改變，越來越多的漫畫被電子化儲存，但針對 漫畫內容進行分析與自動標記的研究則尚未發展成熟。我們決定針對 角色身份的識別著手研究，試著將不同畫格中出現的相同角色進行關 聯，將整本漫畫中所有出現的人臉根據他們所屬的角色進行分群。在 本論文中我們提出了一種基於卷積類神經網路與聚類算法的漫畫角色 分群方法。我們設計並訓練了一個卷積類神經網路對漫畫中的人臉周 邊做特徵的抽取，並將抽取出來的特徵結合臉的位置資訊利用聚類演 算法進行分群。我們的方法在訓練集中未出現過的漫畫與角色上獲得 了 76.92% 的分群準確率。

As reading habits change, more and more manga books are digitized and can be read on tablet or smartphone. However, most of them are just scanned from the printed version and treated as normal image files. There are only few studies focus on extracting information from manga pages automatically. In this thesis, we try to cluster faces in manga books based on their identities. We proposed a method using convolutional neural network and clustering algorithm. We designed and trained a CNN network to extract features from manga character faces, and used the extracted features and spatial relations of faces to cluster them. Our method achieved 76.92% accuracy on unseen manga books and characters.

誌謝 i
摘要 ii
Abstract iii
1 Introduction 1
1.1 Background and Motivation ........................ 1
1.2 Research Purpose.............................. 2
1.3 Thesis Organization............................. 3
2 Related Work 4
2.1 Computer Vision for Manga ........................ 4
2.2 Convolutional Neural Network....................... 4
2.3 Clustering Algorithm ............................ 5
3 Proposed Method 7
3.1 Overview .................................. 7
3.2 Input Image about Character ........................ 9
3.3 Feature Extraction Network......................... 10
3.3.1 Network Architecture........................ 10
3.3.2 Data Selection and Training Detail................. 11
3.4 Clustering Algorithm ............................ 12
3.4.1 Agglomerative Clustering ..................... 12
3.4.2 Distance Measurement (Feature Vectors) . . . . . . . . . . . . . 12
3.4.3 Distance Measurement (Spatial Relations) . . . . . . . . . . . . . 13
4 Evaluation and Results 15
4.1 Evaluation Method ............................. 15
4.2 Dataset ................................... 16
4.3 Empirical Analysis ............................. 16
4.3.1 The size of input region ...................... 16
4.3.2 SpatialConnectivity ........................ 17
4.3.3 Comparison with other methods.................. 18
5 Conclusion 19
Bibliography 20

[1] W.-T. Chu and W.-W. Li. Manga facenet: Face detection in manga based on deep neural network. In Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval, pages 412–415. ACM, 2017.
[2] S. C. Johnson. Hierarchical clustering schemes. Psychometrika, 32(3):241–254, 1967.
[3] J. MacQueen et al. Some methods for classification and analysis of multivariate ob- servations. In Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, volume 1, pages 281–297. Oakland, CA, USA, 1967.
[4] Y. Matsui, K. Ito, Y. Aramaki, A. Fujimoto, T. Ogawa, T. Yamasaki, and K. Aizawa. Sketch-based manga retrieval using manga109 dataset. Multimedia Tools and Appli- cations, 76(20):21811–21838, 2017.
[5] R. Narita, K. Tsubota, T. Yamasaki, and K. Aizawa. Sketch-based manga retrieval using deep features. In Document Analysis and Recognition (ICDAR), 2017 14th IAPR International Conference on, volume 3, pages 49–53. IEEE, 2017.
[6] N.-V. Nguyen, C. Rigaud, and J.-C. Burie. Comic characters detection using deep learning. In Document Analysis and Recognition (ICDAR), 2017 14th IAPR Inter- national Conference on, volume 3, pages 41–46. IEEE, 2017.
[7] T. Ogawa, A. Otsubo, R. Narita, Y. Matsui, T. Yamasaki, and A. Kiyoharu. Object detection for comics using manga109 annotations. arXiv preprint arXiv:1803.08670, 2018.
[8] F. Schroff, D. Kalenichenko, and J. Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 815–823, 2015.
[9] H. Schütze, C. D. Manning, and P. Raghavan. Introduction to information retrieval, volume 39. Cambridge University Press, 2008.
[10] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna. Rethinking the incep- tion architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2818–2826, 2016.
[11] J. H. Ward Jr. Hierarchical grouping to optimize an objective function. Journal of the American statistical association, 58(301):236–244, 1963.
[12] L.Zhao,X.Li,J.Wang,andY.Zhuang.Deeply-learnedpart-alignedrepresentations for person re-identification. arXiv preprint arXiv:1707.07256, 2017.