臺灣博碩士論文加值系統

殭屍網路(Botnet)是現今一個網路犯罪中的主要威脅之一，經常被用來最為發動分散式阻斷服務攻擊(DDOS)、傳送垃圾郵件、竊取機密資料等惡意行為。檢測殭屍網絡是一個具有挑戰性的問題，因為這些殭屍網絡正在不斷改進以規避檢測。
在本篇論文中，我們提出一種基於機器學習方法的殭屍網絡偵測系統，這個系統可以有效的從網路流量中識別殭屍網絡。我們的方法從網路的流量中提取卷積版本的特徵屬性，並通過使用人工神經網絡來訓練分類模型。而實驗結果也證明了，使用卷積特徵屬性的檢測準確度優於使用傳統特徵屬性的檢測準確度。在已知的P2P殭屍網路數據集上可以達到94.7％的準確率(Accuracy)和2.2％的假陽性率(false positive rate)，此外，我們的系統為增加殭屍網路偵測的準確性使用了額外的信心測試。
信心測試進一步對神經網路信任度不足的網路流量進行再次分類，實驗結果也表明了，信心測試階段檢測精度可提高至98.6％，假陽性率有可以降低至0.5％

Botnet is one of the major threats on the Internet for committing cybercrimes, such as DDoS attacks, stealing sensitive information, spreading spams, etc. It is a challenging issue to detect modern botnets that are continuously improving for evading detection.
In this paper, we propose a machine learning based botnet detection system that is shown to be effective in identifying P2P botnets. Our approach extracts convolutional version of effective flow-based features, and trains a classification model by using a feed-forward artificial neural network. The experimental results show that the accuracy of detection using the convolutional features is better than the ones using the traditional features. It can achieve 94.7% of detection accuracy and 2.2% of false positive rate on the known P2P botnet datasets. Furthermore, our system provides an additional confidence testing for enhancing performance of botnet detection.
It further classifies the network traffic of insufficient confidence in the neural network. The experiment shows that this stage can increase the detection accuracy up to 98.6% and decrease the false positive rate up to 0.5%

目錄 I
圖目錄 III
表目錄 IV
1. 介紹 1
1.1. 背景介紹 1
1.1.1. 集中式殭屍網路架構 2
1.1.2. Peer to Peer殭屍網路架構 3
1.1.3. 混合式殭屍網路架構 4
1.2. 研究動機 5
1.3. 貢獻 5
1.4. 全文架構 6
2. 相關研究 7
2.1. Signature-based Detection 7
2.2. Anomaly-based Detection 8
2.3. DNS-based Detection 8
2.4. Machine Learning-based Detection 9
3. 研究方法 11
3.1. 類神經網路 11
3.2. 卷積神經網路 12
3.3. 決策樹 13
3.4. 支援向量機 15
4. 系統設計 16
4.1. 總覽 16
4.2. 資料前處理階段 17
4.3. 卷積神經網路階段 19
4.4. 信心測試階段 22
4.5. 主機偵測階段 22
5. 系統實作 25
5.1. 使用工具介紹 25
5.1.1. TShark 25
5.1.2. scikit-learn 26
5.2. 程式流程 26
5.2.1. Pcap檔轉換為文字檔 26
5.2.2. 封包轉換為網路流 28
5.2.3. 特徵屬性計算 28
5.2.4. 類神經網路與信心測試 30
5.2.5. 主機偵測 31
5.2.6. 訓練模型 33
5.3. 自動化 34
6. 實驗結果與討論 35
6.1. 資料集介紹 35
6.2. 實驗環境 35
6.3. 參數選擇 35
6.3.1. 選取範圍大小(m) 36
6.3.2. 隱藏層數量 37
6.3.3. 隱藏層節點數 38
6.4. 實驗結果比較 39
6.5. 信心測試實驗 41
6.6. 主機偵測實驗 44
7. 結論 46
參考文獻 47

[1] D. Acarali, M. Rajarajan, N. Komninos and . I. Herwono, "Survey of approaches and features for the identification of HTTP-based botnet traffic," In Journal of Network and Computer Applications, pp. 1-15,2016(76)
[2] T. Holz, M. Steiner, F. Dahl, E. Biersack and F. Freiling, "Measurements and mitigation of peer-to-peer-based botnets: a case study on storm worm," In First USENIX Workshop on Large-Scale Exploits and Emergent Threats, In Proceedings of the First USENIX Workshop on Large-Scale Exploits and Emergent Threats, pp. 1-15,2008
[3] P. Wang, S. Sparks and C. C. Zou, "An Advanced Hybrid Peer-to-Peer Botnet,"In IEEE Transactions on Dependable and Secure Computing, pp. 113-127, 2010(7)
[4] C. A. Schiller, J. Binkley, D. Harley, G. Evron, T. Bradley, C. Willems and M. Cross, "Botnets" Burlington: Syngress. pp. 29–75. ISBN 9781597491358.
[5] P. Wang, L. Wu, B. Aslam and C. C. Zou, "A Systematic Study on Peer-to-Peer Botnets" Proceedings of the 18th International Conference on Computer Communications and Networks, pp. 1-8, 2009
[6] P. Wang, S. Sparks and C. C. Zou, "An Advanced Hybrid Peer-to-Peer Botnet"In HotBots'07 Proceedings of the first conference on First Workshop on Hot Topics in Understanding Botnets, pp. 2-2, 2007
[7] S. S. Silva, M. R. Silva, . R. C. Pinto and . R. M. Salles, "Botnets: A survey," In Computer Networks, pp. 378-403,2013(57)
[8] "Snort," Available: https://www.snort.org/.
[9] J. Goebel and T. Holz, "Rishi: identify bot contaminated hosts by IRC nickname evaluation," In Understanding Botnets , 2007
[10] Y. Ji, Q. Li, Y. He and D. Guo, "BotCatch: leveraging signature and behavior for bot detection," In Security and Communication Networks, pp. 952-969, 2015(8)
[11] G. Gu, P. A. Porras, V. Yegneswaran and M. W. Fong, "BotHunter: Detecting Malware Infection Through IDS-Driven Dialog Correlation," In USENIX Security Symposium, 2007
[12] V. Paxson, "Bro: a system for detecting network intruders in real-time," In Computer Networks, pp. 2435-2463, 1999(31)
[13] P. Wurzinger, L. Bilge, T. Holz, J. Goebel, C. Kruegel and E. Kirda, "Automatically Generating Models for Botnet Detection," in European Symposium on Research in Computer Security, pp. 232-249, 2009
[14] G. Gu, J. Zhang and W. Lee, "BotSniffer: Detecting Botnet Command and Control Channels in Network Traffic," In Network and Distributed System Security Symposium, NDSS, 2008
[15] Karasaridis, B. Rexroad and D. A. Hoeflin, "Wide-scale botnet detection and characterization," In HotBots, 2007
[16] J. R. Binkley and S. Singh, "An Algorithm for Anomaly-based Botnet Detection," In Steps to Reducing Unwanted Traffic on the Internet, 2006
[17] M. N. Sakib and C.-T. Huang, "Using anomaly detection based techniques to detect HTTP-based botnet C&C traffic," In IEEE International Conference on Communications, pp. 1-6, 2016
[18] J. R. Binkley, "Anomaly-Based Botnet Server Detection," In FloCon, 2006
[19] S. Siboni and A. Cohen, "Botnet identification via universal anomaly detection," In IEEE International Workshop on Information Forensics and Security, pp. 101-106, 2014
[20] H. Choi, H. Lee, H. Lee and H. Kim, "Botnet Detection by Monitoring Group Activities in DNS Traffic," In Computer and Information Technology pp. 715-720, 2007
[21] J. Kwon, J. Lee, H. Lee and A. Perrig, "PsyBoG: A scalable botnet detection method for large-scale DNS traffic, " In Computer Networks, pp. 48-73, 2016(97)
[22] T.-D. Nguyen,T.-D. Cao and L.-G. Nguyen, "DGA Botnet detection using Collaborative Filtering and density-based Clustering, " In Symposium on Information and Communication Technology, pp. 203-209, 2015
[23] H. Choi and H. Lee, "Identifying botnets by capturing group activities in DNS traffic," In Computer Networks ,pp. 20-33, 2012(56)
[24] S. Schiavoni, F. Maggi, L. Cavallaro and S. Zanero, "Phoenix: DGA-Based Botnet Tracking and Intelligence," In Detection of Intrusions and Malware, and Vulnerability Assessment , pp. 191-211, 2014
[25] M. Manasrah, A. Hasan, O. A. Abouabdalla and S. Ramadass, "Detecting Botnet Activities Based on Abnormal DNS traffic," In International Journal of Computer Science and Information Security, pp. 97-104, 2009 (abs/0911.0487 )
[26] G. Gu, R. Perdisci, J. Zhang and W. Lee, "BotMiner: Clustering Analysis of Network Traffic for Protocol- and Structure Independent Botnet Detection" In USENIX Security Symposium, pp. 139-154, 2008
[27] S.-H. Li, . Y.-C. Kao, Z.-C. Zhang, Y.-P. Chuang and D. C. Yen, "A Network Behavior-Based Botnet Detection Mechanism Using PSO and K-means," In ACM Transactions on Management Information Systems, pp. 3:1-3:30,2015(6)
[28] E. B. Beigi, H. H. Jazi, N. Stakhanova and A. A. Ghorbani, "Towards effective feature selection in machine learning-based botnet detection approaches, " In IEEE Conference on Communications and Network Security, pp. 247-255, 2014
[29] G. Kirubavathi and R. Anitha, "Botnet detection via mining of traffic flow characteristics" In Computers & Electrical Engineering, pp. 91-101, 2016(50)
[30] W.-H. Liao and C.-C. Chang, "Peer to Peer Botnet Detection Using Data Mining Scheme," In IEEE Conference on Internet Technology and Applications, pp. 1-4, 2010
[31] S. Saad, I. Traore, A. Ghorbani, B. Sayed, D. Zhao, W. Lu, J. Felix and P. Hakimian, "Detecting P2P botnets through network behavior analysis and machine learning, " In Privacy, Security and Trust, pp. 174-180, 2011
[32] H. Hang, X. Wei, M. Faloutsos and T. Eliassi-Rad, "Entelecheia: Detecting P2P botnets in their waiting stage," In IFIP Networking Conference, pp. 1-9, 2013
[33] "Cisco IOS NetFlow," Available: http://www.cisco.com/c/en/us/products/ios-nx-os-software/ios-netflow/index.html
[34] P. Narang, C. Hota and V. N. Venkatakrishnan, "PeerShark: flow-clustering and conversation-generation for malicious peer-to-peer traffic identification," In EURASIP Journal on Information Security, pp. 15, 2014
[35] W. S. McCulloch and W. Pitts, “A logical calculus of the ideas immanent in nervous activity”In Neurocomputing: foundations of research, pp. 15-27, 1988
[36] T. Menzies and Y. Hu, “Data Mining For Very Busy People“ In IEEE Computer, pp. 18-26, 2003,
[37] C. Cortes and V. Vapnik,” Support-vector networks” In Machine Learning,pp. 273-297, 1995
[38] Y. Lecun, L. Bottou, Y. Bengio and P. Haffner, "Gradient-based learning applied to document recognition," In Proceedings of the IEEE, pp. 2278-2324, 1998(86)
[39] Krizhevsky, . I. Sutskever and G. E. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks," In Advances in Neural Information Processing Systems, pp. 1106-1114, 2012
[40] O.Abdel-Hamid, . A.-r. Mohamed , H. Jiang, L. Deng , G. Penn and D. Yu, "Convolutional Neural Networks for Speech Recognition, " In IEEE/ACM Transactions on Audio, Speech, and Language Processing, pp. 1533-1545, 2014(22)
[41] D. Zhao, I. Traoré, A. A. Ghorbani, B. Sayed, S. Saad and W. Lu, "Peer to Peer Botnet Detection Based on Flow Intervals, "In Information Security and Privacy Research, pp. 87-102, 2012
[42] "How To Standardize Data for Neural Networks," . Available: https://visualstudiomagazine.com/articles/2014/01/01/how-to-standardize-data-for-neural-networks.aspx
[43] I.-J. Kim, C. Choi, and S.-H. Lee, “Improving discrimination ability of convolutional neural networks by hybrid learning,” In Int’l Journal on Document Analysis and Recognition, pp. 1–9, 2016(19)
[44] “Tshark”: https://www.wireshark.org/docs/man-pages/tshark.html
[45] “Wireshark” available: https://www.wireshark.org/download.html
[46] “scikit-learn” available: http://scikit-learn.org/stable/
[47] “pyinotify” available: https://github.com/seb-m/pyinotify
[48] B. Rahbarinia, R. Perdisci, A. Lanzi and K. Li, "PeerRush: Mining for unwanted P2P traffic," In Information Security and Applications, pp. 194-208, 2014(19)
[49] S. García, . M. Grill, J. Stiborek and A. Zunino, "An empirical comparison of botnet detection methods," In Computers & Security, pp. 100-123,2014(45)