面向网络入侵检测的GAN-SDAE-RF模型研究

针对传统机器学习方法在处理不平衡的海量高维数据时罕见攻击类检测率低的问题,提出了一种基于深度学习的随机森林算法的入侵检测模型,为了避免传统的随机森林面对高维数据和不平衡数据时分类精度低、稳定性差和对罕见攻击类检测率低的问题,引入生成式对抗网络（GAN）和栈式降噪自编码器（SDAE）对随机森林算法（RF）进行改进。将罕见攻击类数据集输入GAN神经网络中,生成新的攻击类样本,改善网络入侵数据在样本集中不均衡分布的情况,通过堆叠深层的SDAE逐层抽取网络数据的分布规则,并结合各个编码层的系数惩罚和重构误差,来确定高维数据中与入侵行为相关的特征,基于降维后的特征数据构建森林决策树。采用UNSW-NB15数据集的实验结果表明,与SVM、KNN、CNN、LSTM、DBN方法相比,GAN-SDAE-RF整体检测准确率平均提高了9.39%、误报率和漏报率平均降低了9%和15.24%以及在少数类Analysis、Shellcode、Backdoor、Worms上检测率分别提高了26.8%、27.98%、27.85%、39.97%。     

Practical real-time intrusion detection using machine learning approaches

The growing prevalence of network attacks is a well-known problem which can impact the availability, confidentiality, and integrity of critical information for both individuals and enterprises. In this paper, we propose a real-time intrusion detection approach using a supervised machine learning technique. Our approach is simple and efficient, and can be used with many machine learning techniques. We applied different well-known machine learning techniques to evaluate the performance of our IDS approach. Our experimental results show that the Decision Tree technique can outperform the other techniques. Therefore, we further developed a real-time intrusion detection system (RT-IDS) using the Decision Tree technique to classify on-line network data as normal or attack data. We also identified 12 essential features of network data which are relevant to detecting network attacks using the information gain as our feature selection criterions. Our RT-IDS can distinguish normal network activities from main attack types (Probe and Denial of Service (DoS)) with a detection rate higher than 98% within 2 s. We also developed a new post-processing procedure to reduce the false-alarm rate as well as increase the reliability and detection accuracy of the intrusion detection system.     

A network intrusion detection system based on a Hidden Naïve Bayes multiclass classifier

With increasing Internet connectivity and traffic volume, recent intrusion incidents have reemphasized the importance of network intrusion detection systems for combating increasingly sophisticated network attacks. Techniques such as pattern recognition and the data mining of network events are often used by intrusion detection systems to classify the network events as either normal events or attack events. Our research study claims that the Hidden Naïve Bayes (HNB) model can be applied to intrusion detection problems that suffer from dimensionality, highly correlated features and high network data stream volumes. HNB is a data mining model that relaxes the Naïve Bayes method’s conditional independence assumption. Our experimental results show that the HNB model exhibits a superior overall performance in terms of accuracy, error rate and misclassification cost compared with the traditional Naïve Bayes model, leading extended Naïve Bayes models and the Knowledge Discovery and Data Mining (KDD) Cup 1999 winner. Our model performed better than other leading state-of-the art models, such as SVM, in predictive accuracy. The results also indicate that our model significantly improves the accuracy of detecting denial-of-services (DoS) attacks.     

基于音位属性和边界信息的音素识别

在检测出音位属性的基础上,提出了一种基于音位属性后验概率的音素边界检测算法,并将音位属性与边界信息应用于基于条件随机场的音素识别.该方法首先计算得出相邻帧音位属性后验概率向量间的夹角,然后将夹角的极大值点所在的帧选为侯选边界,最后通过约束条件去除极值点中的错误边界.本文将音素边界与音位属性信息进行组合,作为基于条件随机场模型的识别系统的观测特征,实验结果表明,增加边界信息后,音素正确识别率有了显著提升.     

基于Chi-square检验的分布式网络入侵检测系统

针对网络攻击的新特点,本文提出了一种基于Chi-square检验的分布式网络入侵检测系统模型CTDIDS。设计并实现了一个基于异常检测的入侵分析引擎。通过对网络数据包的分析,运用Chi-square值比较对系统的行为进行检测。与现有的入侵检测方法相比,本文提出的方法具有更好的环境适应性和数据协同分析能力。实验证明,分布式入侵检测系统CTDIDS具有更高的准确性和扩展性。     

Language models for detection of unknown attacks in network traffic

In this paper, we propose a method for network intrusion detection based on language models. Our method proceeds by extracting language features such as n-grams and words from connection payloads and applying unsupervised anomaly detection—without prior learning phase or presence of labeled data. The essential part of this procedure is linear-time computation of similarity measures between language models of connection payloads. Particular patterns in these models decisive for differentiation of attacks and normal data can be traced back to attack semantics and utilized for automatic generation of attack signatures. Results of experiments conducted on two datasets of network traffic demonstrate the importance of high-order n-grams and variable-length language models for detection of unknown network attacks. An implementation of our system achieved detection accuracy of over 80% with no false positives on instances of recent remote-to-local attacks in HTTP, FTP and SMTP traffic.     

Association Rule Mining Frequent-Pattern-Based Intrusion Detection in Network

In the network security system, intrusion detection plays a significant role. The network security system detects the malicious actions in the network and also conforms the availability, integrity and confidentiality of data information resources. Intrusion identification system can easily detect the false positive alerts. If large number of false positive alerts are created then it makes intrusion detection system as difficult to differentiate the false positive alerts from genuine attacks. Many research works have been done. The issues in the existing algorithms are more memory space and need more time to execute the transactions of records. This paper proposes a novel framework of network security Intrusion Detection System (IDS) using Modified Frequent Pattern (MFP-Tree) via K-means algorithm. The accuracy rate of Modified Frequent Pattern Tree (MFPT)-K means method in finding the various attacks are Normal 94.89%, for DoS based attack 98.34%, for User to Root (U2R) attacks got 96.73%, Remote to Local (R2L) got 95.89% and Probe attack got 92.67% and is optimal when it is compared with other existing algorithms of K-Means and APRIORI.     

网络入侵报警信息实时融合处理模型

针对分布式入侵检测和网络安全预警所需要解决的问题,对多传感器数据融合技术进行了研究。在分析IDS警报信息之间的各种复杂关系的基础上,提出了一个警报信息实时融合处理模型,并根据该模型建立警报信息融合处理系统。实时融合来自多异构IDS传感器的警报信息,形成关于入侵事件的攻击序列图,在此基础上进行威胁评估及攻击预测。该模型拓展了漏报推断功能,以减少漏报警带来的影响,使得到的攻击场景更为完整。实验结果表明,根据该模型建立的融合处理系统应用效果好,具有很高的准确率和警报缩减率。     

A new dependency and correlation analysis for features

The quality of the data being analyzed is a critical factor that affects the accuracy of data mining algorithms. There are two important aspects of the data quality, one is relevance and the other is data redundancy. The inclusion of irrelevant and redundant features in the data mining model results in poor predictions and high computational overhead. This paper presents an efficient method concerning both the relevance of the features and the pairwise features correlation in order to improve the prediction and accuracy of our data mining algorithm. We introduce a new feature correlation metric Q/sub Y/(X/sub i/,X/sub j/) and feature subset merit measure e(S) to quantify the relevance and the correlation among features with respect to a desired data mining task (e.g., detection of an abnormal behavior in a network service due to network attacks). Our approach takes into consideration not only the dependency among the features, but also their dependency with respect to a given data mining task. Our analysis shows that the correlation relationship among features depends on the decision task and, thus, they display different behaviors as we change the decision task. We applied our data mining approach to network security and validated it using the DARPA KDD99 benchmark data set. Our results show that, using the new decision dependent correlation metric, we can efficiently detect rare network attacks such as User to Root (U2R) and Remote to Local (R2L) attacks. The best reported detection rates for U2R and R2L on the KDD99 data sets were 13.2 percent and 8.4 percent with 0.5 percent false alarm, respectively. For U2R attacks, our approach can achieve a 92.5 percent detection rate with a false alarm of 0.7587 percent. For R2L attacks, our approach can achieve a 92.47 percent detection rate with a false alarm of 8.35 percent.     

一个新的SYN Flood攻击防御模型的研究

针对现有的SYN Flood攻击防御方法的不足,本文提出了一个基于TCP连接三次握手的新的防御模型。当系统检测到SYN Flood攻击后,立即把那些占用系统资源的带有典型攻击特征的第一次握手请求永久抛弃,以保证新的正常请求能够被接受;而把其他带有疑似攻击特征的第一次握手请求暂时抛弃,尔后启动自适应学习模块来修正现有的入侵模式,最后再启动SYN Flood攻击检测模块来进一步精确判定。在此基础上设计实现了一套新的SYN Flood攻击防御系统。实验测试结果表明,本入侵防御系统能有效地帮助整个系统提高对抗SYN Flood攻击的能力。     

数据挖掘技术在入侵检测系统中的应用研究

随着以太网的快速发展,基于网络的攻击方式越来越多,传统的入侵检测系统越来越难以应付;将数据挖掘技术引入到入侵检测系统中来,分析网络中各种行为记录中潜在的攻击信息,自动辨别出网络入侵的模式,从而提高系统的检测效率;将K- MEANS算法及DBSCAN算法相综合,应用到入侵检测系统,并针对K- MEANS算法的一些不足进行了改进,提出了通过信息嫡理论的使用解决K- MEANS算法选择初始簇中心问题,然后利用其分类结果完善DBSCAN算法两个关键参数(Eps,Minpts)的设置,通过DB-SCAN算法,进一步地分析可疑的异常聚类,提高聚类的准确度.     

基于深度循环神经网络和改进SMOTE算法的组合式入侵检测模型

已有入侵检测模型普遍只针对网络入侵行为的静态特征进行分析检测,造成检测率低及误报率高等缺陷,且无法有效应用低频攻击。为此提出一种新的基于深度循环神经网络（DRNN）和区域自适应合成过采样算法（RA-SMOTE）的组合式入侵检测模型（DRRS）。首先,RA-SMOTE 对数据集中低频攻击样本进行自适应区域划分,实现差别样本增量,从数据层面提升低频攻击样本数量;其次,利用 DRNN 特有的层间反馈单元,完成多阶段分类特征的时序积累学习,同时多隐层网络结构实现对原始数据分布的最优非线性拟合;最后,使用训练好的DRRS模型完成入侵检测。实验结果表明,相比已有入侵检测模型,DRRS在改善整体检测效果的同时显著提高了低频攻击检测率,且对未知新型攻击具有一定检出率,适用于实际网络环境。     

基于智能蜂群算法的DDoS攻击检测系统

随着大数据应用的普及,DDoS攻击日益严重并已成为主要的网络安全问题。针对大数据环境下的DDoS攻击检测问题,设计了一种融合聚类和智能蜂群算法(DFSABC_elite)的DDoS攻击检测系统。该系统将聚类算法与智能蜂群算法相结合来进行数据流分类,用流量特征分布熵与广义似然比较判别因子来检测DDoS攻击数据流的特征,从而实现了DDoS攻击数据流的高效检测。实验结果显示,该系统在类内紧密度、类间分离度、聚类准确率、算法耗时和DDoS检测准确率方面明显优于基于并行化K-means的普通蜂群算法和基于并行化K-means算法的DDoS检测方法。     

SDN环境下基于DBN的DDoS攻击检测

软件定义网络(SDN)作为新型网络架构模式,其安全威胁主要来自DDoS攻击,建立高效的DDoS攻击检测系统是网络安全管理的重要内容.在SDN环境下,针对DDoS的入侵检测算法具有支持协议少、实用性差等缺陷,为此,提出一种基于深度信念网络(DBN)的DDoS攻击检测算法.分析SDN环境下DDoS攻击的机制,通过Mininet模拟SDN的网络拓扑结构,并使用Wireshark完成DDoS流量数据包的收集和检测.实验结果表明,与XGBoost、随机森林、支持向量机算法相比,该算法具有攻击检测准确性高、误报率低、检测速率快和易于扩展等优势,综合性能较好.     

Trust Management and Admission Control for Host-Based Collaborative Intrusion Detection

The accuracy of detecting an intrusion within a network of intrusion detection systems (IDSes) depends on the efficiency of collaboration between member IDSes. The security itself within this network is an additional concern that needs to be addressed. In this paper, we present a trust-based framework for secure and effective collaboration within an intrusion detection network (IDN). In particular, we design a trust model that allows each IDS to evaluate the trustworthiness of other IDSes based on its personal experience. We also propose an admission control algorithm for the IDS to manage the acquaintances it approaches for advice about intrusions. We discuss the effectiveness of our approach in protecting the IDN against common attacks. Additionally, experimental results demonstrate that our system yields significant improvement in detecting intrusions. The trust model further improves the robustness of the collaborative system against malicious attacks. The experimental results also support that our admission control algorithm is effective and fair, and creates incentives for collaboration.     

基于CanpoySMOTE和自适应学习的入侵检测方法研究

提出了一种基于Canopy与人工合成少数类别过采样技术（CSMOTE）和自适应增强学习（AdaBoostM1）的入侵检测分类方法,以有效减少入侵检测模型因训练数据集攻击类型不均衡而导致的分类误差,提高分类准确率。通过Canopy聚类消除训练集中的孤立点或噪音点,减少训练集噪声;并在预处理时通过SMOTE增加少数类别的样本数量,构造类间平衡的平衡数据集,然后在平衡数据集上用AdaBoosM1训练得到分类器。与在原始训练集上训练的分类器相比,该方法在保持整体准确率高的情况下,少数类别U2R攻击的准确率提升20%,R2L攻击的准确率提升5%,同时平均漏报率降低9%,实验结果表明该方法可以有效提升少数类别准确率,降低平均漏报率,能有效地解决网络入侵检测少数类误分类问题。     

DoS Attack Detection Based on Deep Factorization Machine in SDN

Software-Defined Network (SDN) decouples the control plane of network devices from the data plane. While alleviating the problems presented in traditional network architectures, it also brings potential security risks, particularly network Denial-of-Service (DoS) attacks. While many research efforts have been devoted to identifying new features for DoS attack detection, detection methods are less accurate in detecting DoS attacks against client hosts due to the high stealth of such attacks. To solve this problem, a new method of DoS attack detection based on Deep Factorization Machine (DeepFM) is proposed in SDN. Firstly, we select the Growth Rate of Max Matched Packets (GRMMP) in SDN as detection feature. Then, the DeepFM algorithm is used to extract features from flow rules and classify them into dense and discrete features to detect DoS attacks. After training, the model can be used to infer whether SDN is under DoS attacks, and a DeepFM-based detection method for DoS attacks against client host is implemented. Simulation results show that our method can effectively detect DoS attacks in SDN. Compared with the K-Nearest Neighbor (K-NN), Artificial Neural Network (ANN) models, Support Vector Machine (SVM) and Random Forest models, our proposed method outperforms in accuracy, precision and F1 values.     

基于DBN-XGBDT的入侵检测模型研究

在分布均匀的海量数据情况下,现有的入侵检测模型均具备良好的检测性能。但网络中产生的海量入侵数据的分布通常具有不均衡特点,而大多数检测模型针对罕见攻击类型的检测率低。针对上述问题,提出了一种深度信念网络（Deep Belief Networks,DBN）融合极限梯度提升（eXtreme Gradient Boosting,XGBoost）基于决策树算法（Decision Tree,DT）的入侵检测模型（DBN-XGBDT）。该模型将预处理后的数据集输入深度信念网络中,实现对入侵检测数据的降维处理,将得到的特征数据根据攻击类别任两类为一组,通过XGBoost算法逐一构建梯度提升树并细化为二分类;最后运用控制变量法和XGBoost内置的交叉验证进行调参,择优调整模型参数,对未知网络攻击实现有效检测。基于NSL-KDD数据集对DBN-XGBDT模型与XGBoost、DBN-BP、DBN-MSVM等优越模型进行了检测实验。实验结果表明,DBN-XGBDT模型较上述3个单一、混合分类模型的正确率分别提升2.07个百分点、1.14个百分点,对U2R的检测率提升至75.37%,平均误报率降至56.23%,为入侵检测处理不均衡数据且提高对罕见攻击的检测性能提供了新方法。     

一种粗糙集-决策树结合的入侵检测方法

针对入侵检测系统收集数据海量、高维、检测模型复杂和检测准确率低等问题,采用粗糙集属性约简的优势寻找与判断入侵与否相关的属性,利用决策树分类算法生成模型并对网络连接进行入侵预测分类检测,从而提出了一种粗糙集属性约简和决策树预测分类相结合的网络入侵检测方法.实验结果表明,该方法在入侵检测准确率上有很大的提高,对DoS攻击、Probe攻击和R2L攻击的检测效果均有所提高,同时大大降低了检测的误报率.     

IDGraphs: intrusion detection and analysis using stream compositing

IDGraphs is an interactive visualization system, supporting intrusion detection over massive network traffic streams. It features a novel time-versus-failed-connections mapping that aids in discovery of attack patterns. The number of failed connections (SYN-SYN/ACK) is a strong indicator of suspicious network flows. IDGraphs offers several flow aggregation methods that help reveal different attack patterns. The system also offers high visual scalability through the use of Histographs. The IDGraphs intrusion detection system detects and analyzes a variety of attacks and anomalies, including port scanning, worm outbreaks, stealthy TCP SYN flooding, and some distributed attacks. In this article, we demonstrate IDGraphs using a single day of NetFlow network traffic traces collected at edge routers at Northwestern University which has several OC-3 links.