臺灣博碩士論文加值系統

Public repositories provide unlimited e-learning resources for any educational and training purposes especially for leveraging human resources competency. At the same time it can also increase the reusability of e-learning resources, and in turn, will be able to reduce the cost significantly for the creation and maintenance on the e learning provider side. To facilitate sharing and reuse of LOs on global scale, IEEE LOM (Learning Object Metadata) has been developed for describing the pedagogical aspects of LOs, such as cataloging the LOs based on a taxonomic system of a particular knowledge domain for usage in any educational contexts. The metadata annotation of the LOs can be customized based on an organization need which can extend the usage coverage of the LOs for any particular organizations. This goal can be achieved by through semantically annotating the LO’s metadata.
In this research we aim to extend the reusability of the LOs by associating the LOs with subjects/topics within a pre-defined taxonomic tree. The similarity between topics is calculated using the Lexical Level Similarity approach with the WordNet as the lexical similarity base. This approach will categorize the similar topic within the LOs and a job task topic by measuring the similarity score between the textual meanings of LO’s title, descriptions and keywords with a particular job task’s title and description. Through this way, the LOs will have an association to a similar topic whenever the users browse a learning object repository in order to find the LOs of his interest. The similar topics, based on the similarity score, will be automatically annotated into the metadata of the related LO. This research is intended to evaluate the accuracy rate of the approach in associating the similar topic of LOs. Based on our experiment this approach gives high accuracy in inferring dissimilar topics rather than inferring the similar ones.

Public repositories provide unlimited e-learning resources for any educational and training purposes especially for leveraging human resources competency. At the same time it can also increase the reusability of e-learning resources, and in turn, will be able to reduce the cost significantly for the creation and maintenance on the e learning provider side. To facilitate sharing and reuse of LOs on global scale, IEEE LOM (Learning Object Metadata) has been developed for describing the pedagogical aspects of LOs, such as cataloging the LOs based on a taxonomic system of a particular knowledge domain for usage in any educational contexts. The metadata annotation of the LOs can be customized based on an organization need which can extend the usage coverage of the LOs for any particular organizations. This goal can be achieved by through semantically annotating the LO’s metadata.
In this research we aim to extend the reusability of the LOs by associating the LOs with subjects/topics within a pre-defined taxonomic tree. The similarity between topics is calculated using the Lexical Level Similarity approach with the WordNet as the lexical similarity base. This approach will categorize the similar topic within the LOs and a job task topic by measuring the similarity score between the textual meanings of LO’s title, descriptions and keywords with a particular job task’s title and description. Through this way, the LOs will have an association to a similar topic whenever the users browse a learning object repository in order to find the LOs of his interest. The similar topics, based on the similarity score, will be automatically annotated into the metadata of the related LO. This research is intended to evaluate the accuracy rate of the approach in associating the similar topic of LOs. Based on our experiment this approach gives high accuracy in inferring dissimilar topics rather than inferring the similar ones.

ABSTRACT iv
ACKNOWLEDGMENTS vi
CONTENTS vii
LIST OF FIGURES x
LIST OF TABLE xi
CHAPTER 1 INTRODUCTION 1
1.1. Motivation 1
1.2. Problem Description 2
1.3. Research Approach 5
1.4. Research Overview 6
CHAPTER 2 LITERATURE REVIEW 7
2.1. E-learning and Learning Objects Repository 7
2.2. IEEE LOM (Learning Object Metadata) specification 9
2.2.1. General category (1, IEEE LOM) elements 10
2.2.2. Classification category (9, IEEE LOM) elements 11
2.3. Content Packaging (CP) and IMS CP Information Model 13
2.4. Automatic Metadata Annotation Tool 14
2.5. WordNet 15
2.6. WordNet::Similarity 17
2.7. Text Categorization 19
2.8. Sentence Semantic Similarity Measures 20
2.9. Job Task Classification 22
CHAPTER 3 RESEARCH METHODOLOGY 24
3.1. Detail explanation of the Annotation Tool 24
3.2. Architecture of the Annotation Tool 28
3.2.1. Learning Object Repository (LOR) 28
3.2.2. Metadata analyzer and Annotation Module 29
3.3. Computing the sentence similarity 30
3.3.1. Pre-processing 30
3.3.2. Measuring Similarity 34
3.4. Experimental Design 36
3.4.1. Data Sets 36
3.4.2. Evaluation Criteria 37
3.4.3. Threshold determination 39
CHAPTER 4 EXPERIMENT AND IMPLEMENTATION 41
4.1. Experimental Setting 41
4.1.1. Pre-processing Phase 41
4.1.2. WordNet Similarity and LLM 44
4.2. A typical Example 44
4.2.1. LO metadata and Job Task 45
4.2.2. Pre-processing phase 46
4.2.3. LLM Similarity Comparison 49
4.3. Datasets 50
4.4. Performance Measure 52
4.5. Experimental Results 54
4.5.1. Training Datasets Performance 54
4.5.2. Testing Datasets Performance 55
4.5.3. Observation during the experiment 57
4.6. Result Analysis 58
4.6.1. Comparing Solely the Titles 58
4.6.2. More Information is better 59
4.6.3. Recall and Precision results 59
4.7. Automatic Tool Implementation 60
4.7.1. Scenario 1: A tool in authoring of the LO metadata 60
4.7.2. Demonstration of the tool in Scenario 1 62
4.7.3. Scenario 2: Searching the related LO’ topic 64
4.7.4. Demonstration of the tool in Scenario 2 65
4.7.5. Analysis of Algorithm complexity 66
4.7.6. Case-based Reasoning for Reducing Calculation Complexity 67
CHAPTER 5 CONCLUSIONS 69
5.1. Conclusions 69
5.2. Contributions 70
5.3. Limitations 70
5.4. Future Works 71
REFERENCES 72
APPENDICES 78

[1] www.bkn.go.id. (2012, December 25, 2012). Civil Servant's Distribution based on Type of Job Positions for Period of June 2012. Available: www.bkn.go.id
[2] M. J. Rosenberg, "E-learning: Strategies for Delivering Knowledge in the Digital Era," 2001.
[3] D. Churchill, "Towards a useful classification of learning objects," Educational Technology Research and Development, vol. 55, pp. 479-497, 2007/10/01 2007.
[4] J. Boneu, "Survey on Learning Content Management Systems," in Content Management for E-Learning, N. F. Ferrer and J. M. Alfonso, Eds., ed: Springer New York, 2011, pp. 113-130.
[5] K. Cardinaels, M. Meire, and E. Duval, "Automating metadata generation: the simple indexing interface," in Proceedings of the 14th international conference on World Wide Web, 2005, pp. 548-556.
[6] D. G. Sampson, "Competence-related metadata for educational resources that support lifelong competence development programmes," Educational Technology & Society, vol. 12, pp. 149-159, 2009.
[7] B. Chang, Y. Lee, S. Ko, and J. Cha, "Enhancing ontology-based educational content search service with competency," in Advanced Learning Technologies, 2008. ICALT'08. Eighth IEEE International Conference on, 2008, pp. 293-294.
[8] D. Roy, S. Sarkar, and S. Ghose, "A Comparative Study of Learning Object Metadata, Learning Material Repositories, Metadata Annotation & an Automatic Metadata Annotation Tool," Advances in Semantic Computing, vol. 2, 2010.
[9] E. Duval and W. Hodgins, "A LOM research agenda," in Proceedings of the twelfth international conference on World Wide Web, 2003, pp. 1-9.
[10] J. Greenberg, A. Crystal, W. D. Robertson, and E. Leadem, "Iterative design of metadata creation tools for resource authors," in International Conference on Dublin Core and Metadata Applications, 2003, pp. pp. 49-58.
[11] B. Simon, P. Dolog, Z. Miklós, D. Olmedilla, and M. Sintek, "Conceptualising smart spaces for learning," Journal of Interactive Media in Education, vol. 2004, 2004.
[12] S. Downes, "Resource profiles," Journal of Interactive Media in Education, vol. 2004, 2004.
[13] S. Ternier, E. Duval, D. Massart, A. Campi, S. Guinea, and S. Ceri, "Interoperability for searching learning object repositories: the ProLearn query language," D-Lib Magazine, vol. 14, p. 1, 2008.
[14] I. L. T. S. Committee, "Learning Object Metadata, Final Draft Standard, IEEE 1484.12. 1-2002," Available in: http://ltsc.ieee.org/wg12/20020612-Final-LOM-Draft.html, 2002.
[15] A. Gelbukh, G. Sidorov, and A. Guzman-Arenas, "Document comparison with a weighted topic hierarchy," in Database and Expert Systems Applications, 1999. Proceedings. Tenth International Workshop on, 1999, pp. 566-570.
[16] J. Jovanović, D. Gašević, and V. Devedžić, "Automating semantic annotation to enable learning content adaptation," in Adaptive Hypermedia and Adaptive Web-Based Systems, 2006, pp. 151-160.
[17] C. Fellbaum and G. Miller, "Wordnet: An electronic lexical database," ed: MIT Press, 1998.
[18] J. Boyd-Graber, C. Fellbaum, D. Osherson, and R. Schapire, "Adding dense, weighted connections to wordnet," in Proceedings of the Third International WordNet Conference, 2006, pp. 29-36.
[19] T. Pedersen, S. Patwardhan, and J. Michelizzi, "WordNet:: Similarity: measuring the relatedness of concepts," in Demonstration Papers at HLT-NAACL 2004, 2004, pp. 38-41.
[20] R. Navigli, "Word sense disambiguation: A survey," ACM Computing Surveys (CSUR), vol. 41, p. 10, 2009.
[21] Q. Do, D. Roth, M. Sammons, Y. Tu, and V. G. V. Vydiswaran, "Robust, light-weight approaches to compute lexical similarity," Computer Science Research and Technical Reports, University of Illinois. http://hdl. handle. net/2142/15462, 2010.
[22] A. Djamaris, A. B. Priyanto, and F. Jie, "Implementation of e-learning system readiness: Indonesia context," in Management of Innovation and Technology (ICMIT), 2012 IEEE International Conference on, 2012, pp. 314-319.
[23] H. W. Hodgins, "The future of learning objects," e-Technologies in Engineering Education: Learning Outcomes Providing Future Possibilities, p. 11, 2004.
[24] D. Wiley, "Connecting learning objects to instructional design theory: A definition, a metaphor, and a taxonomy in DA Wiley (Ed.), The Instructional Use of Learning Objects," pp. http://reusability. org/read/chapters/wiley. doc (accessed in March 2th de 2005), 2007.
[25] J. Minguillón, M.-A. Sicilia, and B. Lamb, "From Content Management to E-Learning Content Repositories," in Content Management for E-Learning, N. F. Ferrer and J. M. Alfonso, Eds., ed: Springer New York, 2011, pp. 27-41.
[26] H. Han, C. L. Giles, E. Manavoglu, H. Zha, Z. Zhang, and E. A. Fox, "Automatic document metadata extraction using support vector machines," in Digital Libraries, 2003. Proceedings. 2003 Joint Conference on, 2003, pp. 37-48.
[27] C. Jenkins and D. Inman, "Server-side automatic metadata generation using Qualified Dublin Core and RDF," in Digital Libraries: Research and Practice, 2000 Kyoto, International Conference on., 2000, pp. 262-269.
[28] DC-dot. (accessed January 6, 2013). Available: http://www.ukoln.ac.uk/metadata/dcdot/
[29] D. Roy, S. Sarkar, and S. Ghose, "Automatic extraction of pedagogic metadata from learning content," International Journal of Artificial Intelligence in Education, vol. 18, pp. 97-118, 2008.
[30] S. Dehors, C. Faron-zucker, J. Stromboni, and A. Giboin, "Semi-automated semantic annotation of learning resources by identifying layout features," in In Proceedings of the International Workshop on Applications of Semantic Web Technologies for E-Learning, 2005.
[31] D. Huynh, S. Mazzocchi, and D. Karger, "Piggy bank: Experience the semantic web inside your web browser," The Semantic Web–ISWC 2005, pp. 413-430, 2005.
[32] G. A. Miller, "WordNet: a lexical database for English," Communications of the ACM, vol. 38, pp. 39-41, 1995.
[33] A. Budanitsky and G. Hirst, "Semantic distance in WordNet: An experimental, application-oriented evaluation of five measures," in Workshop on WordNet and Other Lexical Resources, 2001.
[34] P. Resnik, "Using information content to evaluate semantic similarity in a taxonomy," arXiv preprint cmp-lg/9511007, 1995.
[35] D. Lin, "An information-theoretic definition of similarity," in Proceedings of the 15th international conference on Machine Learning, 1998, pp. 296-304.
[36] J. J. Jiang and D. W. Conrath, "Semantic similarity based on corpus statistics and lexical taxonomy," arXiv preprint cmp-lg/9709008, 1997.
[37] G. A. Miller, C. Leacock, R. Tengi, and R. T. Bunker, "A semantic concordance," in Proceedings of the workshop on Human Language Technology, 1993, pp. 303-308.
[38] C. Leacock and M. Chodorow, "Combining local context and WordNet similarity for word sense identification," WordNet: an electronic lexical database, vol. 49, pp. 265-283, 1998.
[39] Z. Wu and M. Palmer, "Verbs semantics and lexical selection," in Proceedings of the 32nd annual meeting on Association for Computational Linguistics, 1994, pp. 133-138.
[40] G. Hirst and D. St-Onge, "Lexical chains as representations of context for the detection and correction of malapropisms," WordNet: an electronic lexical database, vol. 305, pp. 305-332, 1998.
[41] S. Banerjee and T. Pedersen, "Extended gloss overlaps as a measure of semantic relatedness," in International Joint Conference on Artificial Intelligence, 2003, pp. 805-810.
[42] S. Patwardhan, S. Banerjee, and T. Pedersen, "Using measures of semantic relatedness for word sense disambiguation," Computational Linguistics and Intelligent Text Processing, pp. 241-257, 2003.
[43] K. Celik, "A Comprehensive Analysis of Using Wordnet, Part-Of-Speech Tagging, and Word Sense Disambiguation in Text Categorization," Bogaziçi University, 2012.
[44] P. Achananuparp, X. Hu, and X. Shen, "The evaluation of sentence similarity measures," Data Warehousing and Knowledge Discovery, pp. 305-316, 2008.
[45] A. Raviv and S. Markovitch, "Concept-Based Approach to Word-Sense Disambiguation," 2012.
[46] R. Navigli and P. Velardi, "Structural semantic interconnections: a knowledge-based approach to word sense disambiguation," Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 27, pp. 1075-1086, 2005.
[47] E. Agirre, O. L. De Lacalle, A. Soroa, and I. Fakultatea, "Knowledge-based WSD on specific domains: performing better than generic supervised WSD."
[48] M. Cuadros and G. Rigau, "Quality assessment of large scale knowledge resources," in Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, 2006, pp. 534-541.
[49] J. J. Rocchio, "Relevance Feedback in Information Retrieval," in The SMART Retrieval System - Experiments in Automatic Document Processing, G. Salton, Ed., ed: Prentice Hall, 1971.
[50] M. Lesk, "Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone," presented at the Proceedings of the 5th annual international conference on Systems documentation, Toronto, Ontario, Canada, 1986.
[51] H. Schütze, "Automatic word sense discrimination," Computational linguistics, vol. 24, pp. 97-123, 1998.
[52] G. Salton, A. Singhal, M. Mitra, and C. Buckley, "Automatic text structuring and summarization," Information Processing & Management, vol. 33, pp. 193-207, 1997.
[53] K. Papineni, "Why inverse document frequency?," presented at the Proceedings of the second meeting of the North American Chapter of the Association for Computational Linguistics on Language technologies, Pittsburgh, Pennsylvania, 2001.
[54] C.-Y. Lin and E. Hovy, "Automatic evaluation of summaries using n-gram co-occurrence statistics," in Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1, 2003, pp. 71-78.
[55] M. Lapata and R. Barzilay, "Automatic evaluation of text coherence: Models and representations," in International Joint Conference On Artificial Intelligence, 2005, p. 1085.
[56] I. Dagan, O. Glickman, and B. Magnini, "The pascal recognising textual entailment challenge," Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Tectual Entailment, pp. 177-190, 2006.
[57] R. Mihalcea, C. Corley, and C. Strapparava, "Corpus-based and knowledge-based measures of text semantic similarity," in Proceedings of the national conference on artificial intelligence, 2006, p. 775.
[58] B. Dolan, C. Quirk, and C. Brockett, "Unsupervised construction of large paraphrase corpora: Exploiting massively parallel news sources," in Proceedings of the 20th international conference on Computational Linguistics, 2004, p. 350.
[59] P. Achananuparp, X. Hu, X. Zhou, and X. Zhang, "Utilizing sentence similarity and question type similarity to response to similar questions in knowledge-sharing community," in 17th international conference on World Wide Web, 2008.
[60] Rumpun Jabatan Fungsional Pegawai Negeri Sipil, T. G. o. R. Indonesia No. 87 tahun 1999, 1999.
[61] J. Najjar, S. Ternier, and E. Duval, "User behavior in learning object repositories: an empirical analysis," in World Conference on Educational Multimedia, Hypermedia and Telecommunications, 2004, pp. 4373-4379.
[62] Y. Li, Q. Zhu, and Y. Cao, "Automatic metadata generation based on neural network," in Proceedings of the 3rd international conference on Information security, 2004, pp. 192-197.
[63] N. Rizzolo and D. Roth, "Learning based java for rapid development of nlp systems," Language Resources and Evaluation., 2010.
[64] D. Roth and D. Zelenko, "Part of speech tagging using a network of linear separators," in Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics-Volume 2, 1998, pp. 1136-1142.
[65] Y. Tu and D. Roth, "Learning English light verb constructions: contextual or statistical," ACL HLT 2011, p. 31, 2011.
[66] R. Kohavi and F. Provost, "Glossary of terms: Editorial for the Special Issue on Applications of Machine Learning and the Knowledge Discovery Process," Journal of Machine Learning, vol. 30, 1998.
[67] H. Hamilton. (2000). Computer Science 831: Knowledge Discovery in Databases.
[68] B. Spell. (2009, March 20, 2013). Java API for WordNet Searching (JAWS). Available: http://lyle.smu.edu/~tspell/jaws/index.html
[69] Technical Guide of Accessing Credit Point of Computer Administrator, S. C. Body 16/2008, 2008.
[70] S. Tasci and T. Gungor, "An evaluation of existing and new feature selection metrics in text categorization," in Computer and Information Sciences, 2008. ISCIS'08. 23rd International Symposium on, 2008, pp. 1-6.
[71] J. T.-Y. Kwok, "Automated text categorization using support vector machine," in In Proceedings of the International Conference on Neural Information Processing (ICONIP, 1998.
[72] F. Debole and F. Sebastiani, "Supervised term weighting for automated text categorization," in Text mining and its applications, ed: Springer, 2004, pp. 81-97.
[73] V. Adamchik, "Algorithmic Complexity," ed. http://www.cs.cmu.edu/~adamchik/15-121/lectures/Algorithmic%20Complexity/complexity.html, 2009.
[74] D. B. Leake, "CBR in context: The present and future," Case-based reasoning: experiences, lessons, and future directions, pp. 3-30, 1996.
[75] M. Lenz, A. Hübner, and M. Kunze, "Question answering with textual CBR," in Flexible Query Answering Systems, ed: Springer, 1998, pp. 236-247.