臺灣博碩士論文加值系統

隨著網際網路的盛行，電子郵件的使用日趨頻繁。然而垃圾郵件、商業廣告與病毒夾帶的郵件也因此層出不窮。一般的解決辦法是在使用者端執行郵件過濾管理功能，然而大多數的使用者並沒有太多的專業知識背景足以追查這些不請自來的郵件來源，甚至設定完善的過濾規則。即使啟動了過濾機制亦可能因設定錯誤造成本身使用上的不便。所以將過濾條件與控管機制交由伺服器端執行，才能避免上述問題的缺點。在此篇論文中，我們針對電子郵件記錄檔進行近似性資料探勘的研究，以協助系統管理員處理垃圾郵件與病毒郵件肆虐的問題。我們研究的重點在於利用兩階段資料探勘的方法，以近似模式進行關聯組合屬性分析，配合探索式規則以發掘電子郵件紀錄檔之中所蘊含的郵件傳送行為模式，目標是在即時有效的第一時間內偵測出大量傳送與異常發生的行為記錄分析結果。首先在第一階段，我們使用資料探勘前置處理流程，擷取記錄檔中重要的屬性欄位儲存至資料庫，作為後續研究的基礎來源。第二階段則是採用近似性資料探勘方法，針對大量異常與出現頻度較高的關聯組合進行分析。我們所獲得的分析結果將會記錄至關聯規則資料庫，進而提供系統管理員進行有效的管理決策參考依據，以防止類似行為的持續發生。

As e-mail service becomes popular on Internet, general problems such as UBE/UCE and virus mails have occurred more and more. Many client-side facilities have been developed to help users deal with such problems. However, since most users do not have enough resources and expertise to track theses abusing and make the necessary changes adaptively, few (if any) could benefit from applying these facilities. In general, if the filtering process could be done at the servers, these drawbacks could be avoided. In this thesis, we propose an approximation algorithm for mining e-mail logs to help deal with the anti-SPAM and anti-virus problems. The focus of our work is to apply the two-phase incremental mining processes with heuristic rules on e-mail logs for locating the embedded patterns of massively abnormal e-mail transactions in near real-time. In the first phase, we will make the ECTL preprocessing for extracting important attributes from the e-mail log and put them into the database for later use. In the second phase, we will apply the incremental mining algorithm with approximation to find the suspected outliers with massively anomalous e-mail transactions. The results could be integrated into the rule-base and utilized by the related system administrators for further preventing these kinds of abusing activities adaptively.

Abstract
Chapter 1 Introduction
1.1 Motivation
1.2 Issues
1.3 Countermeasures for Anti-SPAM & Anti-Virus
1.4. Overview of the Thesis
Chapter 2 Preliminaries
2.1 Traffic Monitoring Graph
2.2 Statistical Report
2.3 Mail Filtering
2.4 Clustering for Outlier Detection
2.5 Association Mining
Chapter 3 System Architecture
3.1 System Workflow
3.2 Phase 1 — ECTL Preprocessing
3.3 Phase 2 — Incremental Mining
Chapter 4 Phase 1: Preprocessing
4.1 Overview of Sendmail Log Entries
4.2 ECTL Process
4.3 Data Generalization
4.4 Generalized Relation Table
Chapter 5 Phase 2: Incremental Mining
5.1 Mining with Accuracy
5.2 Mining with Approximation
Chapter 6 Experimental Results
6.1 Experimental Environment
6.2 Mining Result
6.3 Performance Analysis
Chapter 7 Conclusion and Future Work
Reference

[1] R. Agrawal, T. Imielinski, and A. Swami. “Mining association rules between sets of items in large databases”, Proc. Of ACM SIGMOD, pages 207-217, May 1993.
[2] N. F. Ayan, A. U. Tansel and E. Arkun. “An efficient algorithm to update large itemsets with early pruning”. Fifth ACM SIGKDD International Conference on
Knowledge Discovery & Data, August 1999.
[3] Tim Bass and Alfredo Freyre, “E-Mail bombs and countermeasures”, IEEE Network, March/April 1998.
[4] Christian Borgelt , Association rule induction,
“http://fuzzy.cs.uni-magdeburg.de/~borgelt/software.html#assoc”, 2002.
[5] Q. Chen, U. Dayal, M.Hsu, "A distributed OLAP infrastructure for e-commerce”, Proceedings, Fourth IFCIS International Conference on Cooperative Information Systems, Sept. 1999.
[6] Her-Tsaan Cheng, An Intelligent E-mail Management System, Master Thesis, National Chiao Tung University, 1999.
[7] D. Cheung, S. Lee, B. Kao, “A general incremental technique for maintaining discovered association rules”, In Proceedings of the 5th International Conference on Database Systems for Advanced Applications (DASFAA), pp. 185-194, 1997.
[8] Kenjiro Cho, R. Kaizaki, A. Kato “Aguri, An aggregation-based traffic profiler”, http://www.csl.sony.co.jp/person/kjc/software.html#aguri
[9] Edward R. Fuller “Denial of service attack”,
http://www.sans.org/infosecFAQ/securitybasics/dos.htm , 2002.
[10] Jiawei Han and Micheline Kamber, Data Mining Concepts and Techniques, Morgan Kaufmann, 2001.
[11] C.-H. Lee, C.-R. Lin and M.-S. Chen, “Sliding-Window Filtering: An efficient algorithm for incremental mining”, Proc. of the ACM 10th Intern''l Conf. on Information and Knowledge Management, November 5-10, 2001.
[12] Gary C. Kessler, “Defenses against distributed denial of service attacks.” 29 Nov. 2000. URL: http://www.sans.org/infosecFAQ/threats/DDoS.htm , 2001.
[13] T. Lunt, “A survey in intrusion detection techniques”, Computer and Security, Vol. 12, no.4, June 1993, pp.405-418.
[14] Steve Maxwell, UNIX network management tools, McGraw-Hill, 1999.
[15] Vikram Pudi, Jayany R. Haritsa, “Quantifying the utility of the past in mining large databases”, Information Systems, Vol. 25, pp. 323-343, 2000.
[16] Mehhran Sahami, “A bayesian approach to filtering junk e-mail”, Proceedings of AAAI’98 Workshop on Learning for Text Categorization, 1998.
[17] Ashoka Savasere, Edward Omiecinski, and Shamkant B. Navathe, “An efficient algorithm for mining association rules in large databases”, Proceedings of the 21nd International Conference on Very Large Databases, pp. 432-444, Zurich, Swizerland, 1995.
[18] C. M. Su, S. S. Tseng, M. F. Jiang, and J. C. S. Chen, “A fast clustering process for outliers and remainder clusters,” Lecture Notes in Artificial Intelligence, Vol. 1574, pp. 360-364, 1999.
[19] E-Mail Spamming countermeasures,
http://ciac.llnl.gov/ciac/bulletins/i-005c.shtml,
CIAC INFORMATION BULLETIN, 1997.
[20] MRTG: The Multi Router Traffic Grapher,
http://people.ee.ethz.ch/~oetiker/webtools/mrtg/.
[21] MAILMGR: Sendmail log analyser, http://web.onda.com.br/orso/index.html.