A Literature Survey and Comprehensive Study of Intrusion Detection
---
paper_title: An Intrusion-Detection Model

paper_content:

A model of a real-time intrusion-detection expert system capable of detecting break-ins, penetrations, and other forms of computer abuse is described. The model is based on the hypothesis that security violations can be detected by monitoring a system's audit records for abnormal patterns of system usage. The model includes profiles for representing the behavior of subjects with respect to objects in terms of metrics and statistical models, and rules for acquiring knowledge about this behavior from audit records and for detecting anomalous behavior. The model is independent of any particular system, application environment, system vulnerability, or type of intrusion, thereby providing a framework for a general-purpose intrusion-detection expert system.
---
paper_title: An Application of Pattern Matching in Intrusion Detection

paper_content:

This report examines and classi es the characteristics of signatures used in misuse intrusion detection. E cient algorithms to match patterns in some of these classes are described. A generalized model for matching intrusion signatures based on Colored Petri Nets is presented, and some of its properties are derived.
---
paper_title: SVM approach with a genetic algorithm for network intrusion detection

paper_content:

Due to the increase in unauthorized access and stealing of internet resources, internet security has become a very significant issue. Network anomalies in particular can cause many potential problems, but it is difficult to discern these from normal traffic. In this paper, we focus on a Support Vector Machine (SVM) and a genetic algorithm to detect network anomalous attacks. We first use a genetic algorithm (GA) for choosing proper fields of traffic packets for analysis. Only the selected fields are used, and a time delay processing is applied to SVM for considering temporal relationships among packets. In order to verify our approach, we tested our proposal with the datasets of MIT Lincoln Lab, and then analyzed its performance. Our SVM approach with selected fields showed excellent performance.
---
paper_title: Fuzzy classification by evolutionary algorithms

paper_content:

Fuzzy sets and fuzzy logic can be used for efficient data classification by fuzzy rules and fuzzy classifiers. This paper presents an application of genetic programming to the evolution of fuzzy classifiers based on extended Boolean queries. Extended Boolean queries are well known concept in the area of fuzzy information retrieval. An extended Boolean query represents a complex soft search expression that defines a fuzzy set on the collection of searched documents. We interpret the data mining task as a fuzzy information retrieval problem and we apply a proven method for query induction from data to find useful fuzzy classifiers. The ability of the genetic programming to evolve useful fuzzy classifiers is demonstrated on two use cases in which we detect faulty products in a product processing plant and discover intrusions in a computer network.
---
paper_title: A data mining framework for building intrusion detection models

paper_content:

There is often the need to update an installed intrusion detection system (IDS) due to new attack methods or upgraded computing environments. Since many current IDSs are constructed by manual encoding of expert knowledge, changes to IDSs are expensive and slow. We describe a data mining framework for adaptively building Intrusion Detection (ID) models. The central idea is to utilize auditing programs to extract an extensive set of features that describe each network connection or host session, and apply data mining programs to learn rules that accurately capture the behavior of intrusions and normal activities. These rules can then be used for misuse detection and anomaly detection. New detection models are incorporated into an existing IDS through a meta-learning (or co-operative learning) process, which produces a meta detection model that combines evidence from multiple models. We discuss the strengths of our data mining programs, namely, classification, meta-learning, association rules, and frequent episodes. We report on the results of applying these programs to the extensively gathered network audit data for the 1998 DARPA Intrusion Detection Evaluation Program.
---
paper_title: The architecture of a network level intrusion detection system

paper_content:

This paper presents the preliminary architecture of a network level intrusion detection system. The proposed system will monitor base level information in network packets (source, destination, packet size, and time), learning the normal patterns and announcing anomalies as they occur. The goal of this research is to determine the applicability of current intrusion detection technology to the detection of network level intrusions. In particular, the authors are investigating the possibility of using this technology to detect and react to worm programs.
---
paper_title: Artificial Neural Networks for Misuse Detection

paper_content:

Misuse detection is the process of attempting to identify instances of network attacks by comparing current activity against the expected actions of an intruder. Most current approaches to misuse detection involve the use of rule-based expert systems to identify indications of known attacks. However, these techniques are less successful in identifying attacks which vary from expected patterns. Artificial neural networks provide the potential to identify and classify network activity based on limited, incomplete, and nonlinear data sources. We present an approach to the process of misuse detection that utilizes the analytical strengths of neural networks, and we provide the results from our preliminary analysis of this approach.
---
paper_title: Detecting attack signatures in the real network traffic with ANNIDA

paper_content:

In this paper, an improved version of ANNIDA for detecting attack signatures in the payload of network packets is presented. The Hamming Net artificial neural network methodology was used with good results. A review of the application's development is followed by a summary of the modifications made in the application in order to classify real data. Application improvements are reported, solving the problems of time delays in writing/reading data in the files and data collision effects when generating numeric keys used to model data for the neural network. Test results highlight the increased accuracy and efficiency of the new application when submitted to real data from HTTP network traffic containing actual traces of attacks and legitimate data. Finally, an evaluation of the application to detect signatures in real network traffic data is presented.
---
paper_title: An Improved Intrusion Detection Technique based on two Strategies Using Decision Tree and Neural Network

paper_content:

In this paper we enhance the notion of anomaly detection and use both neural network (NN) and decision tree (DT) for intrusion detection. While DTs are highly successful in detecting known attacks, NNs are more interesting to detect new attacks. In our method we proposed a new approach to design the system using both DT and combination of unsupervised and supervised NN for Intrusion Detection System (IDS). By applying DT known attacks would be recognized with a quick execution time. Unknown attacks would be detected by applying the unsupervised NN based on hybrid of Self Organizing Map (SOM) for clustering attacks into smaller categories and supervised NN based on Backpropagation for detailed clustering.
---
paper_title: Ensemble-based DDoS detection and mitigation model

paper_content:

This work-in-progress paper presents an ensemble-based model for detecting and mitigating Distributed Denial-of-Service (DDoS) attacks, and its partial implementation. The model utilises network traffic analysis and MIB (Management Information Base) server load analysis features for detecting a wide range of network and application layer DDoS attacks and distinguishing them from Flash Events. The proposed model will be evaluated against realistic synthetic network traffic generated using a software-based traffic generator that we have developed as part of this research. In this paper, we summarise our previous work, highlight the current work being undertaken along with preliminary results obtained and outline the future directions of our work.
---
paper_title: Application of bagging, boosting and stacking to intrusion detection

paper_content:

This paper investigates the possibility of using ensemble algorithms to improve the performance of network intrusion detection systems. We use an ensemble of three different methods, bagging, boosting and stacking, in order to improve the accuracy and reduce the false positive rate. We use four different data mining algorithms, naive bayes, J48 (decision tree), JRip (rule induction) and iBK( nearest neighbour), as base classifiers for those ensemble methods. Our experiment shows that the prototype which implements four base classifiers and three ensemble algorithms achieves an accuracy of more than 99% in detecting known intrusions, but failed to detect novel intrusions with the accuracy rates of around just 60%. The use of bagging, boosting and stacking is unable to significantly improve the accuracy. Stacking is the only method that was able to reduce the false positive rate by a significantly high amount (46.84%); unfortunately, this method has the longest execution time and so is inefficient to implement in the intrusion detection field.
---
paper_title: Unsupervised Clustering Approach for Network Anomaly Detection

paper_content:

This paper describes the advantages of using the anomaly detection approach over the misuse detection technique in detecting unknown network intrusions or attacks. It also investigates the performance of various clustering algorithms when applied to anomaly detection. Five different clustering algorithms: k-Means, improved k-Means, k-Medoids, EM clustering and distance-based outlier detection algorithms are used. Our experiment shows that misuse detection techniques, which implemented four different classifiers (naive Bayes, rule induction, decision tree and nearest neighbour) failed to detect network traffic, which contained a large number of unknown intrusions; where the highest accuracy was only 63.97% and the lowest false positive rate was 17.90%. On the other hand, the anomaly detection module showed promising results where the distance-based outlier detection algorithm outperformed other algorithms with an accuracy of 80.15%. The accuracy for EM clustering was 78.06%, for k-Medoids it was 76.71%, for improved k-Means it was 65.40% and for k-Means it was 57.81%. Unfortunately, our anomaly detection module produces high false positive rate (more than 20%) for all four clustering algorithms. Therefore, our future work will be more focus in reducing the false positive rate and improving the accuracy using more advance machine learning techniques.
---
paper_title: Packet and Flow Based Network Intrusion Dataset

paper_content:

With exponential growth in the number of computer applications and the size of networks, the potential damage that can be caused by attacks launched over the internet keeps increasing dramatically. A number of network intrusion detection methods have been developed with their respective strengths and weaknesses. The majority of research in the area of network intrusion detection is still based on the simulated datasets because of non-availability of real datasets. A simulated dataset cannot represent the real network intrusion scenario. It is important to generate real and timely datasets to ensure accurate and consistent evaluation of methods. We propose a new real dataset to ameliorate this crucial shortcoming. We have set up a testbed to launch network traffic of both attack as well as normal nature using attack tools. We capture the network traffic in packet and flow format. The captured traffic is filtered and preprocessed to generate a featured dataset. The dataset is made available for research purpose.
---
paper_title: Hybrid Intelligent Intrusion Detection Scheme

paper_content:

This paper introduces a hybrid scheme that combines the advantages of deep belief network and support vector machine. An application of intrusion detection imaging has been chosen and hybridization scheme have been applied to see their ability and accuracy to classify the intrusion into two outcomes: normal or attack, and the attacks fall into four classes; R2L, DoS, U2R, and Probing. First, we utilize deep belief network to reduct the dimensionality of the feature sets. This is followed by a support vector machine to classify the intrusion into five outcome; Normal, R2L, DoS, U2R, and Probing. To evaluate the performance of our approach, we present tests on NSL-KDD dataset and show that the overall accuracy offered by the employed approach is high.
---
paper_title: An autonomous labeling approach to support vector machines algorithms for network traffic anomaly detection

paper_content:

In the past years, several support vector machines (SVM) novelty detection approaches have been applied on the network intrusion detection field. The main advantage of these approaches is that they can characterize normal traffic even when trained with datasets containing not only normal traffic but also a number of attacks. Unfortunately, these algorithms seem to be accurate only when the normal traffic vastly outnumbers the number of attacks present in the dataset. A situation which can not be always hold. This work presents an approach for autonomous labeling of normal traffic as a way of dealing with situations where class distribution does not present the imbalance required for SVM algorithms. In this case, the autonomous labeling process is made by SNORT, a misuse-based intrusion detection system. Experiments conducted on the 1998 DARPA dataset show that the use of the proposed autonomous labeling approach not only outperforms existing SVM alternatives but also, under some attack distributions, obtains improvements over SNORT itself.
---
paper_title: An Intelligent Intrusion Detection System for Mobile Ad-Hoc Networks Using Classification Techniques

paper_content:

This paper proposes an intelligent multi level classification technique for effective intrusion detection in Mobile Ad-hoc Networks. The algorithm uses a combination of a tree classifier which uses a labeled training data and an Enhanced Multiclass SVM algorithm. Moreover, an effective preprocessing technique has been proposed and implemented in this work in order to improve the detection accuracy and to reduce the processing time. From the experiments carried out in this work, it has been observed that significant improvement has been achieved in this model from the view point of both high detection rates as well as low false alarm rates.
---
paper_title: A new intrusion detection system using support vector machines and hierarchical clustering

paper_content:

Whenever an intrusion occurs, the security and value of a computer system is compromised. Network-based attacks make it difficult for legitimate users to access various network services by purposely occupying or sabotaging network resources and services. This can be done by sending large amounts of network traffic, exploiting well-known faults in networking services, and by overloading network hosts. Intrusion Detection attempts to detect computer attacks by examining various data records observed in processes on the network and it is split into two groups, anomaly detection systems and misuse detection systems. Anomaly detection is an attempt to search for malicious behavior that deviates from established normal patterns. Misuse detection is used to identify intrusions that match known attack scenarios. Our interest here is in anomaly detection and our proposed method is a scalable solution for detecting network-based anomalies. We use Support Vector Machines (SVM) for classification. The SVM is one of the most successful classification algorithms in the data mining area, but its long training time limits its use. This paper presents a study for enhancing the training time of SVM, specifically when dealing with large data sets, using hierarchical clustering analysis. We use the Dynamically Growing Self-Organizing Tree (DGSOT) algorithm for clustering because it has proved to overcome the drawbacks of traditional hierarchical clustering algorithms (e.g., hierarchical agglomerative clustering). Clustering analysis helps find the boundary points, which are the most qualified data points to train SVM, between two classes. We present a new approach of combination of SVM and DGSOT, which starts with an initial training set and expands it gradually using the clustering structure produced by the DGSOT algorithm. We compare our approach with the Rocchio Bundling technique and random selection in terms of accuracy loss and training time gain using a single benchmark real data set. We show that our proposed variations contribute significantly in improving the training process of SVM with high generalization accuracy and outperform the Rocchio Bundling technique.
---
paper_title: Optimized intrusion detection mechanism using soft computing techniques

paper_content:

Intrusion detection is an important technique in computer and network security. A variety of intrusion detection approaches be present to resolve this severe issue but the main problem is performance. It is important to increase the detection rates and reduce false alarm rates in the area of intrusion detection. Therefore, in this research, an optimized intrusion detection mechanism using soft computing techniques is proposed to overcome performance issues. The KDD-cup dataset is used that is a benchmark for evaluating the security detection mechanisms. The Principal Component Analysis (PCA) is applied to transform the input samples into a new feature space. The selecting of an appropriate number of principal components is a critical problem. So, Genetic Algorithm (GA) is used in the optimum selection of principal components instead of using traditional method. The Support Vector Machine (SVM) is used for classification purpose. The performance of this approach is addresses. Further, a comparative analysis is made with existing approaches. Consequently, this method provides optimal intrusion detection mechanism which is capable to minimize amount of features and maximize the detection rates.
---
paper_title: Neural networks learning improvement using the K-means clustering algorithm to detect network intrusions

paper_content:

In the present work, we propose a new technique to enhance the learning capabilities and reduce the computation intensity of a competitive learning multi-layered neural network using the K-means clustering algorithm. The proposed model use multi-layered network architecture with a backpropagation learning mechanism. The K-means algorithm is first applied to the training dataset to reduce the amount of samples to be presented to the neural network, by automatically selecting an optimal set of samples. The obtained results demonstrate that the proposed technique performs exceptionally in terms of both accuracy and computation time when applied to the KDD99 dataset compared to a standard learning schema that use the full dataset..
---
paper_title: Y-means: a clustering method for intrusion detection

paper_content:

As the Internet spreads to each comer of the world, computers are exposed to miscellaneous intrusions from the World Wide Web. We need effective intrusion detection systems to protect our computers from these unauthorized or malicious actions. Traditional instance-based learning methods for intrusion detection can only detect known intrusions since these methods classify instances based on what they have learned. They rarely detect the intrusions that they have not learned before. In this paper, we present a clustering heuristic for intrusion detection, called Y-means. This proposed heuristic is based on the K-means algorithm and other related clustering algorithms. It overcomes two shortcomings of K-means: number of clusters dependency and degeneracy. The result of simulations run on the KDD-99 data set shows that Y-means is an effective method for partitioning large data space. A detection rate of 89.89% and a false alarm rate of 1.00% are achieved with Y-means.
---
paper_title: K-Means+ID3: A Novel Method for Supervised Anomaly Detection by Cascading K-Means Clustering and ID3 Decision Tree Learning Methods

paper_content:

In this paper, we present "k-means+ID3", a method to cascade k-means clustering and the ID3 decision tree learning methods for classifying anomalous and normal activities in a computer network, an active electronic circuit, and a mechanical mass-beam system. The k-means clustering method first partitions the training instances into k clusters using Euclidean distance similarity. On each cluster, representing a density region of normal or anomaly instances, we build an ID3 decision tree. The decision tree on each cluster refines the decision boundaries by learning the subgroups within the cluster. To obtain a final decision on classification, the decisions of the k-means and ID3 methods are combined using two rules: 1) the nearest-neighbor rule and 2) the nearest-consensus rule. We perform experiments on three data sets: 1) network anomaly data (NAD), 2) Duffing equation data (DED), and 3) mechanical system data (MSD), which contain measurements from three distinct application domains of computer networks, an electronic circuit implementing a forced Duffing equation, and a mechanical system, respectively. Results show that the detection accuracy of the k-means+ID3 method is as high as 96.24 percent at a false-positive-rate of 0.03 percent on NAD; the total accuracy is as high as 80.01 percent on MSD and 79.9 percent on DED
---
paper_title: Fusion of multiple classifiers for intrusion detection in computer networks

paper_content:

The security of computer networks plays a strategic role in modern computer systems. In order to enforce high protection levels against threats, a number of software tools have been currently developed. Intrusion Detection Systems aim at detecting intruders who elude "first line" protection. In this paper, a pattern recognition approach to network intrusion detection based on the fusion of multiple classifiers is proposed. Five decision fusion methods are assessed by experiments and their performances compared. The potentialities of classifier fusion for the development of effective intrusion detection systems are evaluated and discussed.
---
paper_title: Evaluation of an adaptive genetic-based signature extraction system for network intrusion detection

paper_content:

Machine learning techniques are frequently applied to intrusion detection problems in various ways such as to classify normal and intrusive activities or to mine interesting intrusion patterns. Self-learning rule-based systems can relieve domain experts from the difficult task of hand crafting signatures, in addition to providing intrusion classification capabilities. To this end, a genetic-based signature learning system has been developed that can adaptively and dynamically learn signatures of both normal and intrusive activities from the network traffic. In this paper, we extend the evaluation of our systems to real time network traffic which is captured from a university departmental server. A methodology is developed to build fully labelled intrusion detection data set by mixing real background traffic with attacks simulated in a controlled environment. Tools are developed to pre-process the raw network data into feature vector format suitable for a supervised learning classifier system and other related machine learning systems. The signature extraction system is then applied to this data set and the results are discussed. We show that even simple feature sets can help detecting payload-based attacks.
---
paper_title: Hierarchical Kohonenen net for anomaly detection in network security

paper_content:

A novel multilevel hierarchical Kohonen Net (K-Map) for an intrusion detection system is presented. Each level of the hierarchical map is modeled as a simple winner-take-all K-Map. One significant advantage of this multilevel hierarchical K-Map is its computational efficiency. Unlike other statistical anomaly detection methods such as nearest neighbor approach, K-means clustering or probabilistic analysis that employ distance computation in the feature space to identify the outliers, our approach does not involve costly point-to-point computation in organizing the data into clusters. Another advantage is the reduced network size. We use the classification capability of the K-Map on selected dimensions of data set in detecting anomalies. Randomly selected subsets that contain both attacks and normal records from the KDD Cup 1999 benchmark data are used to train the hierarchical net. We use a confidence measure to label the clusters. Then we use the test set from the same KDD Cup 1999 benchmark to test the hierarchical net. We show that a hierarchical K-Map in which each layer operates on a small subset of the feature space is superior to a single-layer K-Map operating on the whole feature space in detecting a variety of attacks in terms of detection rate as well as false positive rate.
---
paper_title: Anomaly Network Intrusion Detection Based on Improved Self Adaptive Bayesian Algorithm

paper_content:

Recently, research on intrusion detection in computer systems has received much attention to the computational intelligence society. Many intelligence learning algorithms applied to the huge volume of complex and dynamic dataset for the construction of efficient intrusion detection systems (IDSs). Despite of many advances that have been achieved in existing IDSs, there are still some difficulties, such as correct classification of large intrusion detection dataset, unbalanced detection accuracy in the high speed network traffic, and reduce false positives. This paper presents a new approach to the alert classification to reduce false positives in intrusion detection using improved self adaptive Bayesian algorithm (ISABA). The proposed approach applied to the security domain of anomaly based network intrusion detection, which correctly classifies different types of attacks of KDD99 benchmark dataset with high classification rates in short response time and reduce false positives using limited computational resources.
---
paper_title: Anomaly Intrusion Detection System using Hamming Network Approach

paper_content:

Intrusion detection is an interesting approach that could be used to improve the security of network system. IDS detects suspected patterns of network traffic on the remaining open parts through monitoring user activities. The major problems of existing models is recognition of new attacks, low accuracy, detection time and system adaptability. In this paper, evolving anomaly intrusion detection system is constructed using hamming and MAXNET Neural Network for recognize attack class in the network traffic. The result is encouraging, the detection rate is 95% which is relatively high. We describe another approach based on Multilayer Perceptrons (MLP) network and compare the results of the two approaches to evaluate the system. The experimental results demonstrate that the designed models are promising in terms of accuracy and computational time of real word intrusion detection. Training and testing data obtains from the Defense Advanced Research Projects Agency(DARPA) intrusion detection evaluation datasets.
---
paper_title: Anomaly Detection in Ethernet Networks Using Self Organizing Maps

paper_content:

Anomaly detection attempts to recognize abnormal behavior to detect intrusions. We have concentrated to design a prototype UNIX Anomaly Detection System. Neural Networks are tolerant of imprecise data and uncertain information. A tool has been devised for detecting such intrusions into the network. The tool uses the machine learning approaches ad clustering techniques like Self Organizing Map and compares it with the K-means approach. Our system is described for applying hierarchical unsupervised neural network to intrusion detection system.
---
paper_title: Layered Approach Using Conditional Random Fields for Intrusion Detection

paper_content:

Intrusion detection faces a number of challenges; an intrusion detection system must reliably detect malicious activities in a network and must perform efficiently to cope with the large amount of network traffic. In this paper, we address these two issues of Accuracy and Efficiency using Conditional Random Fields and Layered Approach. We demonstrate that high attack detection accuracy can be achieved by using Conditional Random Fields and high efficiency by implementing the Layered Approach. Experimental results on the benchmark KDD '99 intrusion data set show that our proposed system based on Layered Conditional Random Fields outperforms other well-known methods such as the decision trees and the naive Bayes. The improvement in attack detection accuracy is very high, particularly, for the U2R attacks (34.8 percent improvement) and the R2L attacks (34.5 percent improvement). Statistical Tests also demonstrate higher confidence in detection accuracy for our method. Finally, we show that our system is robust and is able to handle noisy data without compromising performance.
---
paper_title: Bayesian based intrusion detection system

paper_content:

In this paper intrusion detection using Bayesian probability is discussed. The systems designed are trained a priori using a subset of the KDD dataset. The trained classifier is then tested using a larger subset of KDD dataset. Initially, a system was developed using a naive Bayesian classifier that is used to identify possible intrusions. This classifier was able to detect intrusion with an acceptable detection rate. The classier was then extended to a multi-layer Bayesian based intrusion detection. Finally, we introduce the concept that the best possible intrusion detection system is a layered approach using different techniques in each layer.
---
paper_title: Autonomous rule creation for intrusion detection

paper_content:

Many computational intelligence techniques for anomaly based network intrusion detection can be found in literature. Translating a newly discovered intrusion recognition criteria into a distributable rule can be a human intensive effort. This paper explores a multi-modal genetic algorithm solution for autonomous rule creation. This algorithm focuses on the process of creating rules once an intrusion has been identified, rather than the evolution of rules to provide a solution for intrusion detection. The algorithm was demonstrated on anomalous ICMP network packets (input) and Snort rules (output of the algorithm). Output rules were sorted according to a fitness value and any duplicates were removed. The experimental results on ten test cases demonstrated a 100 percent rule alert rate. Out of 33,804 test packets 3 produced false positives. Each test case produced a minimum of three rule variations that could be used as candidates for a production system.
---
paper_title: An active learning based TCM-KNN algorithm for supervised network intrusion detection

paper_content:

As network attacks have increased in number and severity over the past few years, intrusion detection is increasingly becoming a critical component of secure information systems and supervised network intrusion detection has been an active and difficult research topic in the field of intrusion detection for many years. However, it hasn't been widely applied in practice due to some inherent issues. The most important reason is the difficulties in obtaining adequate attack data for the supervised classifiers to model the attack patterns, and the data acquisition task is always time-consuming and greatly relies on the domain experts. In this paper, we propose a novel supervised network intrusion detection method based on TCM-KNN (Transductive Confidence Machines for K-Nearest Neighbors) machine learning algorithm and active learning based training data selection method. It can effectively detect anomalies with high detection rate, low false positives under the circumstance of using much fewer selected data as well as selected features for training in comparison with the traditional supervised intrusion detection methods. A series of experimental results on the well-known KDD Cup 1999 data set demonstrate that the proposed method is more robust and effective than the state-of-the-art intrusion detection methods, as well as can be further optimized as discussed in this paper for real applications.
---
paper_title: Hierarchical Kohonenen net for anomaly detection in network security

paper_content:

A novel multilevel hierarchical Kohonen Net (K-Map) for an intrusion detection system is presented. Each level of the hierarchical map is modeled as a simple winner-take-all K-Map. One significant advantage of this multilevel hierarchical K-Map is its computational efficiency. Unlike other statistical anomaly detection methods such as nearest neighbor approach, K-means clustering or probabilistic analysis that employ distance computation in the feature space to identify the outliers, our approach does not involve costly point-to-point computation in organizing the data into clusters. Another advantage is the reduced network size. We use the classification capability of the K-Map on selected dimensions of data set in detecting anomalies. Randomly selected subsets that contain both attacks and normal records from the KDD Cup 1999 benchmark data are used to train the hierarchical net. We use a confidence measure to label the clusters. Then we use the test set from the same KDD Cup 1999 benchmark to test the hierarchical net. We show that a hierarchical K-Map in which each layer operates on a small subset of the feature space is superior to a single-layer K-Map operating on the whole feature space in detecting a variety of attacks in terms of detection rate as well as false positive rate.
---
paper_title: Unsupervised Clustering Approach for Network Anomaly Detection

paper_content:

This paper describes the advantages of using the anomaly detection approach over the misuse detection technique in detecting unknown network intrusions or attacks. It also investigates the performance of various clustering algorithms when applied to anomaly detection. Five different clustering algorithms: k-Means, improved k-Means, k-Medoids, EM clustering and distance-based outlier detection algorithms are used. Our experiment shows that misuse detection techniques, which implemented four different classifiers (naive Bayes, rule induction, decision tree and nearest neighbour) failed to detect network traffic, which contained a large number of unknown intrusions; where the highest accuracy was only 63.97% and the lowest false positive rate was 17.90%. On the other hand, the anomaly detection module showed promising results where the distance-based outlier detection algorithm outperformed other algorithms with an accuracy of 80.15%. The accuracy for EM clustering was 78.06%, for k-Medoids it was 76.71%, for improved k-Means it was 65.40% and for k-Means it was 57.81%. Unfortunately, our anomaly detection module produces high false positive rate (more than 20%) for all four clustering algorithms. Therefore, our future work will be more focus in reducing the false positive rate and improving the accuracy using more advance machine learning techniques.
---

Title: A Literature Survey and Comprehensive Study of Intrusion Detection

Section 1: INTRODUCTION
Description 1: This section provides an overview of intrusion detection, its importance in network security, and various techniques used to detect suspicious activities in computer networks.

Section 2: SURVEY OF INTRUSION DETECTION BASED ON DIFFERENT TECHNIQUES
Description 2: This section presents an extensive study of various intrusion detection classifier techniques and other methods categorized into neural network, support vector machine, k-means classifier, hybrid technique, and other detection techniques.

Section 3: Neural network based intrusion detection
Description 3: This section provides a brief review of neural network-based intrusion detection techniques and their applications.

Section 4: Support vector machine based intrusion detection
Description 4: This section discusses various papers related to support vector machine-based intrusion detection techniques.

Section 5: K-means algorithm based intrusion detection
Description 5: This section explores different papers utilizing the k-means algorithm for intrusion detection and their outcomes.

Section 6: Hybrid technique based intrusion detection
Description 6: This section reviews the hybrid approaches combining multiple techniques for effective intrusion detection.

Section 7: Other classifier based intrusion detection
Description 7: This section discusses various other classifier-based techniques for intrusion detection presented in different research papers.

Section 8: COMPREHENSIVE ANALYSIS AND DISCUSSIONS
Description 8: This section presents a comprehensive analysis of the various intrusion detection methods with respect to detection rate, time, and false alarm rate.

Section 9: CONCLUSION
Description 9: This section concludes the survey by summarizing the various techniques available for intrusion detection and highlights the importance of Network Intrusion Detection Systems in network security.