臺灣博碩士論文加值系統

資訊爆炸時代，如何短時間內由天文數字般的資訊中發掘有興趣資訊為一重要的研究議題。一般在進行資訊檢索前需牢記各搜尋引擎(search engine)的網址，適應不同搜尋引擎提供的搜尋界面，透過各獨立搜尋引擎設定關鍵字或查詢條件，在獲得各搜尋引擎回覆檢索結果後，需自行過濾高度重覆的檢索結果以獲取所需的資訊。
本研究發展一個有效率的搜尋機制以快速找尋需求的資訊，並實作之超連結與關鍵字頻分析(Hyperlink and Keyword Frequency Analysis, HKFA) 搜尋引擎包含五個模組為(1)查詢模組；(2)資訊檢索模組─內含分派演算法；(3)資訊萃取模組─內含特徵萃取演算法與超連結正規化演算法；(4)資訊過濾模組─內含個數與次數演算法、過濾超連結演算法及關鍵字頻與位置演算法；(5)資訊整合模組。藉由本研究所設計之搜尋引擎機制，所提供的資訊具有高精確度(precision)及高回憶度(recall)的特性，能有效率節省到各搜尋引擎的檢索與過濾資訊時間，並降低使用者的資訊負荷(information overhead)。

In this epoch, the growth of information technology has exploded. How to mine the information of people’s interest in a short time has become an important issue. In general, people need to record addresses of every search engine before they begin information retrieval and adapt to different interfaces that every search engine provides. After getting each retrieve results, people need to filter the information that has filled with high repetition from independent search engines which set keywords or query conditions.
This study develops an effective search mechanism to help people to find out the required information. It also aims to implement a Hyperlink and Keyword Frequency Analysis (HKFA) search engine. The HKFA search engine consists of five modules. There are (1)Query Module; (2)Information Retrieval Module─includes Dispatcher Algorithm; (3)Information Extraction Module─includes Feature Extraction Algorithm and Hyperlink Normal Form Algorithm; (4)Information Filtering Module─includes Occurrence Hit Algorithm、Filter Hyperlink Algorithm、Keyword Frequency and Position Algorithm; (5) Information Integration Module. The search engine mechanism designed in this study is to provide high precision and high recall information. Furthermore, this mechanism can not only effectively save people’s time spent on retrieving and filtering information but also reduce user’s information overhead.

中文摘要-------------------------------------------I
英文摘要-------------------------------------------II
誌謝-----------------------------------------------III
目錄-----------------------------------------------IV
圖目錄---------------------------------------------VI
表目錄---------------------------------------------X
第1章 緒論-----------------------------------------1
1.1 研究動機-------------------------------------1
1.2 研究目的-------------------------------------2
1.3 HKFA搜尋引擎---------------------------------3
第2章 文獻探討-------------------------------------5
2.1 搜尋引擎面臨之問題---------------------------5
2.2 搜尋引擎分類---------------------------------8
2.3 搜尋引擎相關技術-----------------------------11
2.3.1 關鍵字-------------------------------------11
2.3.2 獲取資訊---------------------------------12
2.3.3 代理人-----------------------------------14
2.4 目前搜尋引擎網站彙整-------------------------15
2.5 資料探勘-------------------------------------17
2.6 網頁探勘-------------------------------------21
第3章 HKFA搜尋引擎架構-----------------------------23
3.1 HKFA搜尋引擎架構-----------------------------23
3.2 查詢模組-------------------------------------26
3.3 資訊檢索模組---------------------------------31
3.4 資訊萃取模組---------------------------------32
3.5 資訊過濾模組---------------------------------34
3.6 資訊整合模組---------------------------------35
第4章 系統設計與演算法-----------------------------36
4.1 查詢模組之流程及演算法-----------------------41
4.2 資訊檢索模組之流程及演算法-------------------44
4.3 資訊萃取模組之流程及演算法-------------------54
4.4 資訊過濾模組之流程及演算法-------------------63
4.5 資訊整合模組之流程及演算法-------------------77
4.6 HKFA搜尋引擎各演算法之結果-------------------79
第5章 實驗結果-------------------------------------86
5.1 實驗設計-------------------------------------86
5.2 實驗結果-------------------------------------87
第6章 結論及建議-----------------------------------140
6.1 結論-----------------------------------------140
6.2 建議-----------------------------------------141
參考文獻-------------------------------------------142

-中文文獻-
1．江玉婷，”中文資訊檢索測試集設計與製作之研究”，台大圖資碩士論文，1998.
2．呂赫，”以模糊類神經網路方法實現資訊自動搜尋機制”，成大資工碩士論文，1998.
3．林癸宏，”多代理人在全球資訊網的資訊蒐集”，清大資工碩士論文，1998.
4．林盛康，”以關聯式索引典為基礎之互動式查詢擴展應用於網頁資訊檢索”，台科電子碩士論文，1999.
5．邱舜傑，” WWW網路資訊篩選智慧型代理器建構之研究”，大葉資管碩士論文，1997.
6．許志新，”分散式搜尋引擎之設計與實作”，中正資工碩士論文，1996.
7．黃承渠，”中文文件自動建立鏈結”，大同資工碩士論文，1997.
8．陳珍珮，”網頁篩選的索引結構”，清華資工碩士論文，1997.
9．潘信全，”以軟體代理程式為基礎之網際網路商品資訊搜尋”，交大資管碩士論文，1997.
10．鄭祺文，”個人資訊代理人之設計”，交大資工碩士論文，1999.
11．謝欣君，”個人化搜尋服務之設計與實作”，交大資科碩士論文，1998.
-英文文獻-
12．Adali, S., Bufi, C. and Temtanapat, Y., “Integrated Search Engine,” IEEE Conference on Knowledge and Data Engineering Exchange Workshop, November 1997, 140-147.
13．Agrawal, R., Imielinski, T. and Swami, A., “Mining association rules between sets of items in large database,” ACM SIGMOD Conference on Management of Data, Washington, D.C., May 1993, 207-216.
14．Agrawal, R. and Srikant, R., “Fast algorithm for mining association rules,” 20th Int’l Conference on Very Large Databases (VLDB), Santiago, Chile, Sept 1994, 487-199.
15．Agrawal, R. and Srikant, R., “Mining Sequential Patterns,” IEEE the 11th Int’l Conference on Data Enginering (ICDC), Taipai, Taiwan, March 1995, 3-14.
16．Arita, T., Shishibori, M. and Aoe, J. I., “An Efficient Algorithm for Full Text Retrieval for Multiple Keywords,” ELSEVIER Journal on Information Sciences, February 1998, 345-363.
17．Berson, A., Smith, S. and Thearling, K., “Building data mining application for CRM,” Proc. Of McGraw-Hill, 2000.
18．Budi, Y., and Dik, L. L., “WISE: A World Wide Web Resource Database System,” IEEE Journal on Knowledge and Data Engineering, Auguest 1996, 548-554
19．Chen, H., Chung, Y. M., Ramsey, M. and Yang, C. C., “An intelligent personal spider (agent) for dynamic Internet/Intranet searching,” ELSEVIER Journal on Decision Support System, May 1998, 41-58
20．Chen, M. S., Han, J. and Yu, P. S., “Data Mining: An Overview from Database Perpective,” IEEE Journal on Knowledge and Data Engineering, December 1996, 866-883
21．Cooley, R., Mobasher, B. and Srivastava, J., “Web mining : information and pattern discovery on the world wide web,” 9th IEEE International Conference on Tools with Artificial Intelligence (ICTAI’97), November 1997, 558-567.
22．Fu, Y., “Data mining,” IEEE Journal on IEEE Potentials, October/November 1997, 18-20.
23．Jenkins, C., Kackson, M., Burden, P. and Wallis, J., “Searching the world wide web : an evaluation of available tools and methodologies,” ELSEVIER Journal on Information and software technology, February 1998, 985-994.
24．Kolak, O. and Li, W. S., “On Ranking and Organizing Web Query Results,” IEEE Conference on Knowledge and Data Engineering Exchange, 1999. (KDEX ''99). Workshop on , 2000, 26-33.
25．McBryan, O. A., “GENVL and WWWW: tools for taming the Web,” The First International World Wide Web Conference (WWW1), May 1994.
26．Mendelzon, A. O., Mihaila, G. A. and Milo, T., “Querying the World Wide Web,” International Journal on Digital Libraries 1(1), 1997, 54-67.
27．Michael, J. S., Chandrasekar, S., Woo, T. G. and Michael E. W., “ Knowledge management and data mining for marketing,” ELSEVIER Journal on Decision Support Systems, May 2001, 127-137.
28．Ramkumar, G. D. and Swami, A., “Clustering Data without Distance Functions,” IEEE bulletin of technical committee on Data Engineering, March 1998, 9-14.
29．Sato, N., Uehara, M., Sakai, Y. and Mori, H., “Distributed Information Retrieval by using Cooperative Meta Search Engines,” International Conference on Distributed Computing Systems Workshop, April 2001, 345-350.
30．Spertus, E., “ParaSite: Mining Structural Information on the Web,” The Sixth International World Wide Web Conference (WWW6), April 1997, 1205-1215.
31．Tsoi, A. C., ”Structure of the Internet?” IEEE Conference on Intelligent Multimedia, Video and Speech Processing, 2001, 449-452.
32．VanRijsbergen, C. J., “Information Retrieval,” 1992, http://
www.dcs.glasgow.ac.uk/Keith/Preface.html.
33．Wang, W., Meng, W. and Yu, C., “Concept Hierarchy Based Text Database Categorization in Metasearch Engine Environment, ” the First International Conference on Web Information Systems Engineering (WISE''2000), Hong Kong, June 2000, 283-290.
34．Yang, C. C., Yen, J. and Chen, H., “Intelligent internet searching agent based on hybrid simulated annealing,” ELSEVIER Journal on Decision Support System, May 2000, 269-277.
35．Zamir, O. and Etzioni, O., “Grouper: A Dynamic Clustering Interface to Web Search Results”, The Eighth International World Wide Web Conference (WWW8), May 1999.
-相關網站-
36．The Gais Search Engine, http://gais.cs.ccu.edu.tw
37．The Google Search Engine, http://www.google.com/
38．The Openfind Search Engine, http://www.openfind.com.tw/
39．The PC Home Online Search Engine, http://www.pchome.
com.tw/
40．The Sina Search Engine, http://www.sina.com.tw
41．The Yahoo!奇摩 Search Engine, http://tw.yahoo.coXm
42．The Yam Search Engine, http://www.yam.com.tw