改进SVM在入侵检测中的应用研究

软间隔支持向量机（SVM,support vector machine）分类算法是目前入侵检测中最好的分类异常行为的机器学习算法之一,但是它是有监督学习方法,并不能适用于检测新的入侵行为;而1类SVM方法是一种可用于检测异常的无监督学习方法,但误警率比较高。根据以上两种方法,提出了一种改进的SVM方法,仿真实验证明这种方法是一种具有低误警率的无监督学习方法,具有和软间隔SVM相似的检测能力。     

基于双层注意力神经网络的入侵检测方法

基于网络的入侵检测技术作为一种重要的安全防护手段,对及时发现网络攻击行为起着重要的作用。目前,采用特征工程的机器学习算法是检测分析网络入侵的常用方法,但是人工设计的特征往往会丢失有效载荷的重要信息;另外,网络攻击流量中的不同数据包信息在入侵检测中所起的作用是不同的,而现有算法大都对重要信息的捕捉能力不足。针对上述问题,提出了一种新的深度学习模型L2-AMNN,无需复杂的特征工程,直接提取原始网络流量的有效载荷数据作为样本,在双向长短时记忆神经网络基础上,引入双层注意力机制,捕获关键字节信息和数据包信息,生成更加准确的入侵检测特征向量。实验结果表明,与SVM、DNN、LSTM等模型相比,L2-AMNN对网络入侵检测的准确率、检出率平均提升了4.05%和2.48%,同时误报率、漏报率平均降低了4.41%和2.61%,总体检测性能优于其他同类模型。     

A hybrid machine learning approach for detecting unprecedented DDoS attacks

Service availability plays a vital role on computer networks, against which Distributed Denial of Service (DDoS) attacks are an increasingly growing threat each year. Machine learning (ML) is a promising approach widely used for DDoS detection, which obtains satisfactory results for pre-known attacks. However, they are almost incapable of detecting unknown malicious traffic. This paper proposes a novel method combining both supervised and unsupervised algorithms. First, a clustering algorithm separates the anomalous traffic from the normal data using several flow-based features. Then, using certain statistical measures, a classification algorithm is used to label the clusters. Employing a big data processing framework, we evaluate the proposed method by training on the CICIDS2017 dataset and testing on a different set of attacks provided in the more up-to-date CICDDoS2019. The results demonstrate that the Positive Likelihood Ratio (LR+) of our method is approximately 198% higher than the ML classification algorithms.

     

基于SDN的DDoS攻击防御系统

软件定义网络（SDN）是一种新兴网络架构,通过将转发层和控制层分离,实现网络的集中管控。控制器作为SDN网络的核心,容易成为被攻击的目标,分布式拒绝服务（DDoS）攻击是SDN网络面临的最具威胁的攻击之一。针对这一问题,本文提出一种基于机器学习的DDoS攻击检测模型。首先基于信息熵监控交换机端口流量来判断是否存在异常流量,检测到异常后提取流量特征,使用SVM+K-Means的复合算法检测DDoS攻击,最后控制器下发丢弃流表处理攻击流量。实验结果表明,本文算法在误报率、检测率和准确率指标上均优于SVM算法和K-Means算法。     

一种基于L2-SVM 的多视角核心向量机

核化一类硬划分SVDD、一/二类L2-SVM、L2 支持向量回归和Ranking SVM均已被证明是中心约束最小包含球. 这里将多视角学习引入核化L2-SVM, 提出核化两类多视角L2-SVM (Multi-view L2-SVM), 并证明该核化两类Multi-view L2-SVM 亦为中心约束最小包含球, 进而提出一种多视角核心向量机MvCVM. 所提出的Multi-view L2-SVM 和MvCVM既考虑了视角之间的差异性, 又考虑了视角之间的关联性, 使得分类器在各个视角上的学习结果趋于一致. 人造多视角数据集和真实多视角数据集的实验均表明了Multi-view L2-SVM 和MvCVM方法的有效性.

     

Practical real-time intrusion detection using machine learning approaches

The growing prevalence of network attacks is a well-known problem which can impact the availability, confidentiality, and integrity of critical information for both individuals and enterprises. In this paper, we propose a real-time intrusion detection approach using a supervised machine learning technique. Our approach is simple and efficient, and can be used with many machine learning techniques. We applied different well-known machine learning techniques to evaluate the performance of our IDS approach. Our experimental results show that the Decision Tree technique can outperform the other techniques. Therefore, we further developed a real-time intrusion detection system (RT-IDS) using the Decision Tree technique to classify on-line network data as normal or attack data. We also identified 12 essential features of network data which are relevant to detecting network attacks using the information gain as our feature selection criterions. Our RT-IDS can distinguish normal network activities from main attack types (Probe and Denial of Service (DoS)) with a detection rate higher than 98% within 2 s. We also developed a new post-processing procedure to reduce the false-alarm rate as well as increase the reliability and detection accuracy of the intrusion detection system.     

Bayesian Neural Networks for Internet Traffic Classification

Internet traffic identification is an important tool for network management. It allows operators to better predict future traffic matrices and demands, security personnel to detect anomalous behavior, and researchers to develop more realistic traffic models. We present here a traffic classifier that can achieve a high accuracy across a range of application types without any source or destination host-address or port information. We use supervised machine learning based on a Bayesian trained neural network. Though our technique uses training data with categories derived from packet content, training and testing were done using features derived from packet streams consisting of one or more packet headers. By providing classification without access to the contents of packets, our technique offers wider application than methods that require full packet/payloads for classification. This is a powerful advantage, using samples of classified traffic to permit the categorization of traffic based only upon commonly available information     

A study of SVM using a combination of the online learning method and the midpoint-validation method

The support vector machine (SVM) is known as one of the most influential and powerful tools for solving classification and regression problems, but the original SVM does not have an online learning technique. Therefore, many researchers have introduced online learning techniques to the SVM. In a previous article, we proposed an unsupervised online learning method using the technique of the self-organized map for the SVM. In another article, we proposed the midpoint validation method for an improved SVM. We test the performance of the SVM using a combination of the two techniques in this article. In addition, we compare its performance with the original hard-margin SVM, the soft-margin SVM, and the k-NN method, and also experiment with our proposed method on surface electromyogram recognition problems with changes in the position of the electrode. These experiments showed that our proposed method gave a better performance than the other SVMs and corresponded to the changing data.     

基于改进AAR模型的DIDoS攻击早期检测方法

分布式增速拒绝服务(DIDoS)攻击采用逐步提升发包速率的方式来造成受害者资源的慢消耗,较之传统的分布式拒绝服务(DDoS)攻击更具隐蔽性,如何尽可能早地将其捕获是一个亟待研究的问题。本文针对DIDoS攻击的特点,提出了一种基于改进AAR模型的DIDoS攻击早期检测方法。为此,首先提出了一组基于条件熵的检测特征:流特征条件熵(TFCE),用以反映DIDoS攻击流速的增长变化;然后根据改进的AAR模型对TFCE值进行多步预测;最后采用经过训练的SVM分类器对预测值进行分类,以识别攻击企图。实验结果表明,在保证检测精度相当的前提下,该方法比部分现有方法能够更快检测到攻击。     

Evaluation of an adaptive genetic-based signature extraction system for network intrusion detection

Machine learning techniques are frequently applied to intrusion detection problems in various ways such as to classify normal and intrusive activities or to mine interesting intrusion patterns. Self-learning rule-based systems can relieve domain experts from the difficult task of hand crafting signatures, in addition to providing intrusion classification capabilities. To this end, a genetic-based signature learning system has been developed that can adaptively and dynamically learn signatures of both normal and intrusive activities from the network traffic. In this paper, we extend the evaluation of our systems to real time network traffic which is captured from a university departmental server. A methodology is developed to build fully labelled intrusion detection data set by mixing real background traffic with attacks simulated in a controlled environment. Tools are developed to pre-process the raw network data into feature vector format suitable for a supervised learning classifier system and other related machine learning systems. The signature extraction system is then applied to this data set and the results are discussed. We show that even simple feature sets can help detecting payload-based attacks.     

Detection of DDoS attacks using optimized traffic matrix

Distributed Denial of Service (DDoS) attacks have been increasing with the growth of computer and network infrastructures in Ubiquitous computing. DDoS attacks generating mass traffic deplete network bandwidth and/or system resources. It is therefore significant to detect DDoS attacks in their early stage. Our previous approach used a traffic matrix to detect DDoS attacks quickly and accurately. However, it could not find out to tune up parameters of the traffic matrix including (i) size of traffic matrix, (ii) time based window size, and (iii) a threshold value of variance from packets information with respect to various monitored environments and DDoS attacks. Moreover, the time based window size led to computational overheads when DDoS attacks did not occur. To cope with it, we propose an enhanced DDoS attacks detection approach by optimizing the parameters of the traffic matrix using a Genetic Algorithm (GA) to maximize the detection rates. Furthermore, we improve the traffic matrix building operation by (i) reforming the hash function to decrease hash collisions and (ii) replacing the time based window size with a packet based window size to reduce the computational overheads. We perform experiments with DARPA 2000 LLDOS 1.0, LBL-PKT-4 of Lawrence Berkeley Laboratory and generated attack datasets. The experimental results show the feasibility of our approach in terms of detection accuracy and speed.     

An evaluation of dimension reduction techniques for one-class classification

Dimension reduction (DR) is important in the processing of data in domains such as multimedia or bioinformatics because such data can be of very high dimension. Dimension reduction in a supervised learning context is a well posed problem in that there is a clear objective of discovering a reduced representation of the data where the classes are well separated. By contrast DR in an unsupervised context is ill posed in that the overall objective is less clear. Nevertheless successful unsupervised DR techniques such as principal component analysis (PCA) exist—PCA has the pragmatic objective of transforming the data into a reduced number of dimensions that still captures most of the variation in the data. While one-class classification falls somewhere between the supervised and unsupervised learning categories, supervised DR techniques appear not to be applicable at all for one-class classification because of the absence of a second class label in the training data. In this paper we evaluate the use of a number of up-to-date unsupervised DR techniques for one-class classification and we show that techniques based on cluster coherence and locality preservation are effective.     

One-class remote sensing classification: one-class vs. binary classifiers

Many applications of remote sensing only require the classification of a single land type. This is known as the one-class classification problem and it can be performed using either binary classifiers, by treating all other classes as the negative class, or one-class classifiers which only consider the class of interest. The key difference between these two approaches is in their training data and the amount of effort needed to produce it. Binary classifiers require an exhaustively labelled training data set while one-class classifiers are trained using samples of just the class of interest. Given ample and complete training data, binary classifiers generally outperform one-class classifiers. However, what is not clear is which approach is more accurate when given the same amount of labelled training data. That is, for a fixed labelling effort, is it better to use a binary or one-class classifier. This is the question we consider in this article. We compare several binary classifiers, including backpropagation neural networks, support vector machines, and maximum likelihood classifiers, with two one-class classifiers, one-class SVM, and presence and background learning (PBL), on the problem of one-class classification in high-resolution remote sensing imagery. We show that, given a fixed labelling budget, PBL consistently outperforms the other methods. This advantage stems from the fact that PBL is a positive-unlabelled method in which large amounts of readily available unlabelled data is incorporated into the training phase, allowing the classifier to model the negative class more effectively.     

Bloccess: Enabling Fine-Grained Access Control Based on Blockchain

Botnets pose significant threats to cybersecurity. The infected Internet of Things (IoT) devices are used to launch unsupported malicious activities on target entities to disrupt their operations and services. To address this danger, we propose a machine learning-based method, for detecting botnets by analyzing network traffic data flow including various types of botnet attacks. Our method uses a hybrid model where a Variational AutoEncoder (VAE) is trained in an unsupervised manner to learn latent representations that describe the benign traffic data, and one-class classifier (OCC) for detecting anomaly (also called novelty detection). The main aim of this research is to learn the discriminating representations of the normal data in low dimensional latent space generated by VAE, and thus improve the predictive power of the OCC to detect malicious traffic. We have evaluated the performance of our model, and compared it against baseline models using a real network based dataset, containing popular IoT devices, and presenting a wide variety of attacks from two recent botnet families Mirai and Bashlite. Tests showed that our model can detect botnets with a satisfactory performance.

     

Network intrusion detection in covariance feature space

Detecting multiple and various network intrusions is essential to maintain the reliability of network services. The problem of network intrusion detection can be regarded as a pattern recognition problem. Traditional detection approaches neglect the correlation information contained in groups of network traffic samples which leads to their failure to improve the detection effectiveness. This paper directly utilizes the covariance matrices of sequential samples to detect multiple network attacks. It constructs a covariance feature space where the correlation differences among sequential samples are evaluated. Two statistical supervised learning approaches are compared: a proposed threshold based detection approach and a traditional decision tree approach. Experimental results show that both achieve high performance in distinguishing multiple known attacks while the threshold based detection approach offers an advantage of identifying unknown attacks. It is also pointed out that utilizing statistical information in groups of samples, especially utilizing the covariance information, will benefit the detection effectiveness.     

A More Practical Approach for Single-Packet IP Traceback using Packet Logging and Marking

Tracing IP packets to their origins is an important step in defending Internet against denial-of-service attacks. Two kinds of IP traceback techniques have been proposed as packet marking and packet logging. In packet marking, routers probabilistically write their identification information into forwarded packets. This approach incurs little overhead but requires large flow of packets to collect the complete path information. In packet logging, routers record digests of the forwarded packets. This approach makes it possible to trace a single packet and is considered more powerful. At routers forwarding large volume of traffic, the high storage overhead and access time requirement for recording packet digests introduce practicality problems. In this paper, we present a novel scheme to improve the practicality of log-based IP traceback by reducing its overhead on routers. Our approach makes an intelligent use of packet marking to improve scalability of log-based IP traceback. We use mathematical analysis and simulations to evaluate our approach. Our evaluation results show that, compared to the state-of-the-art log-based approach called hash-based IP traceback, our approach maintains the ability to trace single IP packet while reducing the storage overhead by half and the access time overhead by a factor of the number of neighboring routers.     

Rule-based OneClass-DS learning algorithm

One-class learning algorithms are used in situations when training data are available only for one class, called target class. Data for other class(es), called outliers, are not available. One-class learning algorithms are used for detecting outliers, or novelty, in the data. The common approach in one-class learning is to use density estimation techniques or adapt standard classification algorithms to define a decision boundary that encompasses only the target data. In this paper, we introduce OneClass-DS learning algorithm that combines rule-based classification with greedy search algorithm based on density of features. Its performance is tested on 25 data sets and compared with eight other one-class algorithms; the results show that it performs on par with those algorithms.     

Detecting unseen falls from wearable devices using channel-wise ensemble of autoencoders

A fall is an abnormal activity that occurs rarely, so it is hard to collect real data for falls. It is, therefore, difficult to use supervised learning methods to automatically detect falls. Another challenge in automatically detecting falls is the choice of engineered features. In this paper, we formulate fall detection as an anomaly detection problem and propose to use an ensemble of autoencoders to learn features from different channels of wearable sensor data trained only on normal activities. We show that the traditional approach of choosing a threshold as the maximum of the reconstruction error on the training normal data is not the right way to identify unseen falls. We propose two methods for automatic tightening of reconstruction error from only the normal activities for better identification of unseen falls. We present our results on two activity recognition datasets and show the efficacy of our proposed method against traditional autoencoder models and two standard one-class classification methods.     

面向信息内容安全的文本过滤和分类系统研究与实现

本文设计并实现了一个面向信息内容安全应用的文本过滤与自动分类系统。系统采取探测器和分类器两个步骤实现高速网络环境下数据截取、还原、分类的功能。探测器采取简单规则匹配和高速字符串匹配算法来提高数据截取和自身的过滤性能,分类器采取基于简单向量空间模型设计,采取自动学习和人工干预相结合的方法来提高系统的查准率和查全率。本文给出了探测器和分类器在实际应用时的技术性能。     

Local outlier factor and stronger one class classifier based hierarchical model for detection of attacks in network intrusion detection dataset

Identification of attacks by a network intrusion detection system (NIDS) is an important task. In signature or rule based detection, the previously encountered attacks are modeled, and signatures/rules are extracted. These rules are used to detect such attacks in future, but in anomaly or outlier detection system, the normal network traffic is modeled. Any deviation from the normal model is deemed to be an outlier/ attack. Data mining and machine learning techniques are widely used in offline NIDS. Unsupervised and supervised learning techniques differ the way NIDS dataset is treated. The characteristic features of unsupervised and supervised learning are finding patterns in data, detecting outliers, and determining a learned function for input features, generalizing the data instances respectively. The intuition is that if these two techniques are combined, better performance may be obtained. Hence, in this paper the advantages of unsupervised and supervised techniques are inherited in the proposed hierarchical model and devised into three stages to detect attacks in NIDS dataset. NIDS dataset is clustered using Dirichlet process (DP) clustering based on the underlying data distribution. Iteratively on each cluster, local denser areas are identified using local outlier factor (LOF) which in turn is discretized into four bins of separation based on LOF score. Further, in each bin the normal data instances are modeled using one class classifier (OCC). A combination of Density Estimation method, Reconstruction method, and Boundary methods are used for OCC model. A product rule combination of the threemethods takes into consideration the strengths of each method in building a stronger OCC model. Any deviation from this model is considered as an attack. Experiments are conducted on KDD CUP’99 and SSENet-2011 datasets. The results show that the proposed model is able to identify attacks with higher detection rate and low false alarms.