Global and local clustering soft assignment for intrusion detection system: a comparative study

Abstract

Intrusion Detection System (IDS) plays an important role in computer network defence mechanism against malicious objects. The ability of IDS to detect new sophisticated attacks compared to traditional method such as firewall is important to secure the network. Machine Learning algorithm such as unsupervised learning and supervised learning is capable to solve the problem of classification in IDS. To achieve that, KDD Cup 99 dataset is used in experiments. This dataset contains 5 million instances with 5 different categories which are Normal, DOS, U2R, R2L and Probe. With such a large dataset, the learning process consumes a lot of processing times and resources. Clustering is unsupervised learning method that can be used for organizing data by grouping similar features into same group. In literature, many researchers used global clustering approach whereby all input will be combined and clustered to construct a codebook. However, there is an alternative technique namely local clustering approach whereby the input will be split into 5 different categories and clustered independently to construct 5 different codebooks. The main objective of this research is to compare the classification performance between the global and local clustering approaches. For this purpose, the soft assignment approach is used for indexing on KDD input and SVM for classification. In the soft assignment approach, the smallest distance values are used for attack description and RBF kernel for SVM to classify attack. The results show that the global clustering approach outperforms the local clustering approach for binary classification. It gives 83.0% of the KDD Cup 99 dataset. However, the local clustering approach outperforms the global clustering approach on multi-class classification problem. It gives 60.6% of the KDD Cup 99 dataset.