Detection and defense of application-layer DDoS attacks in backbone web traffic

Web servers are usually located in a well-organized data center where these servers connect with the outside Internet directly through backbones. Meanwhile, the application-layer distributed denials of service (AL-DDoS) attacks are critical threats to the Internet, particularly to those business web servers. Currently, there are some methods designed to handle the AL-DDoS attacks, but most of them cannot be used in heavy backbones. In this paper, we propose a new method to detect AL-DDoS attacks. Our work distinguishes itself from previous methods by considering AL-DDoS attack detection in heavy backbone traffic. Besides, the detection of AL-DDoS attacks is easily misled by flash crowd traffic. In order to overcome this problem, our proposed method constructs a Real-time Frequency Vector (RFV) and real-timely characterizes the traffic as a set of models. By examining the entropy of AL-DDoS attacks and flash crowds, these models can be used to recognize the real AL-DDoS attacks. We integrate the above detection principles into a modularized defense architecture, which consists of a head-end sensor, a detection module and a traffic filter. With a swift AL-DDoS detection speed, the filter is capable of letting the legitimate requests through but the attack traffic is stopped. In the experiment, we adopt certain episodes of real traffic from Sina and Taobao to evaluate our AL-DDoS detection method and architecture. Compared with previous methods, the results show that our approach is very effective in defending AL-DDoS attacks at backbones.     

PAPMSC: Power-Aware Performance Management Approach for Virtualized Web Servers via Stochastic Control

As green computing is becoming a popular computing paradigm, the performance of energy-efficient data center becomes increasingly important. This paper proposes power-aware performance management via stochastic control method (PAPMSC), a novel stochastic control approach for virtualized web servers. It addresses the instability and inefficiency issues due to dynamic web workloads. It features a coordinated control architecture that optimizes the resource allocation and minimizes the overall power consumption while guaranteeing the service level agreements (SLAs). More specifically, due to the interference effect among the co-located virtualized web servers and time-varying workloads, the relationship between the hardware resource assignment to different virtual servers and the web applications’ performance is considered as a coupled Multi-Input-Multi-Output (MIMO) system and formulated as a robust optimization problem. We propose a constrained stochastic linear-quadratic controller (cSLQC) to solve the problem by minimizing the quadratic cost function subject to constraints on resource allocation and applications’ performance. Furthermore, a proportional controller is integrated to enhance system stability. In the second layer, we dynamically manipulate the physical frequency for power efficiency using an adaptive linear quadratic regulator (ALQR). Experiments on our testbed server with a variety of workload patterns demonstrate that the proposed control solution significantly outperforms existing solutions in terms of effectiveness and robustness.     

Classifying and clustering malicious advertisement uniform resource locators using deep learning

Malicious online advertisement detection has attracted increasing attention in recent years in both academia and industry. The existing advertising blocking systems are vulnerable to the evolution of new attacks and can cause time latency issues by analyzing web content or querying remote servers. This article proposes a lightweight detection system for advertisement Uniform resource locators (URLs) detection, depending only on lexical‐based features. Deep learning algorithms are used for online advertising classification. After optimizing the deep neural network architecture, our proposed approach can achieve satisfactory results with false negative rate as low as 1.31%. We also design a novel unsupervised method for data clustering. With the implementation of AutoEncoder for feature preprocessing and t‐distributed stochastic neighbor embedding for clustering and visualization, our model outperforms other dimensionality reduction algorithms by generating clear clusterings for different URL families.     

An adaptable distributed query processing architecture

Traditionally, distributed query optimization techniques generate static query plans at compile time. However, the optimality of these plans depends on many parameters (such as the selectivities of operations, the transmission speeds and workloads of servers) that are not only difficult to estimate but are also often unpredictable and fluctuant at runtime. As the query processor cannot dynamically adjust the plans at runtime, the system performance is often less than satisfactory. In this paper, we introduce a new highly adaptive distributed query processing architecture. Our architecture can quickly detect fluctuations in selectivities of operations, as well as transmission speeds and workloads of servers, and accordingly change the operation order of a distributed query plan during execution. We have implemented a prototype based on the Telegraph system [Telegragraph project. Available from >]. Our experimental study shows that our mechanism can adapt itself to the changes in the environment and hence approach to an optimal plan during execution.     

QoS-Broker for transactional workloads

Typical request processing systems, such as web servers and database servers, try to accommodate all requests as fast as possible, which can be described as a Best-Effort approach. However, different application items may have different quality-of-service (QoS) requirements, and this can be viewed as an orthogonal concern to the basic system functionality. In this paper we propose the QoS-Broker, a middleware for delivering QoS over servers and applications. We show its architecture to support contracts over varied targets including queries, transactions, services or sessions, also allowing expressions on variables to be specified in those targets. We also discuss how the QoS-Broker implements basic strategies for QoS over workloads. Our experimental results illustrate the middleware by applying priority and weighted- fair-queuing based differentiation over clients and over transactions, and also admission control, using a benchmark as a case-study.     

基于生成式对抗网络的拟态蜜罐特征生成方法

拟态蜜罐借鉴生物拟态博弈思想,是一种综合运用“蜜罐模拟服务特征”的保护色机制和“服务模拟蜜罐特征”的警戒色机制进行诱骗博弈的动态蜜罐技术,其核心策略是特征生成与演化。生成式对抗网络(GAN)则是一种特征生成方法,它通过生成器与判别器之间的对抗博弈,使生成器生成的数据达到“以假乱真”的效果,其对抗博弈的思想与拟态蜜罐思想极为相近。本文提出一种基于生成式对抗网络的拟态蜜罐特征生成方法MMHP-GAN(Mimicry honeypot-GAN),通过对MMHP-GAN的结构及参数优化训练,产生真假难辨的蜜罐或服务新特征。实验表明,通过该方法生成的特征数据进行演化,服务可以有效抵抗攻击,并且通过对比,本文的方案要优于当前已有的特征生成方案。     

Learning DFA representations of HTTP for protecting web applications

Intrusion detection is a key technology for self-healing systems designed to prevent or manage damage caused by security threats. Protecting web server-based applications using intrusion detection is challenging, especially when autonomy is required (i.e., without signature updates or extensive administrative overhead). Web applications are difficult to protect because they are large, complex, highly customized, and often created by programmers with little security background. Anomaly-based intrusion detection has been proposed as a strategy to meet these requirements.This paper describes how DFA (Deterministic Finite Automata) induction can be used to detect malicious web requests. The method is used in combination with rules for reducing variability among requests and heuristics for filtering and grouping anomalies. With this setup a wide variety of attacks is detectable with few false-positives, even when the system is trained on data containing benign attacks (e.g., attacks that fail against properly patched servers).     

基于流量分析的App-DDoS攻击检测

针对当前应用层分布式拒绝服务攻击(App-DDoS)检测方法高度依赖于系统日志,且检测攻击类型单一的问题,提出了基于卡尔曼滤波和信息熵的联合检测模型DFM-FA(detection and filtering model against App-DDoSattacks based on flow analysis),将应用层的行为异常检测映射为网络层的流量异常检测,最大限度地保证了合法用户的优先正常访问.实验证明,DFM-FA既不依赖于系统日志,同时又能检测到FTP、DNS等多种App-DDoS攻击.     

Hybrid unsupervised web-attack detection and classification – A deep learning approach

Web requests made by users of web applications are manipulated by hackers to gain control of web servers. Moreover, detecting web attacks has been increasingly important in the distribution of information over the last few decades. Also, several existing techniques had been performed on detecting vulnerable web attacks using machine learning and deep learning techniques. However, there is a lack in achieving attack detection ratio owing to the utilization of supervised and semi-supervised learning approaches. Thus to overcome the aforementioned issues, this research proposes a hybrid unsupervised detection model a deep learning-based anomaly-based web attack detection. Whereas, the encoded outputs of De-Noising Autoencoder (DAE), as well as Stacked Autoencoder (SAE), are integrated and given to the Generative adversarial network (GAN) as input to improve the feature representation ability to detect the web attacks. Consequently, for classifying the type of attacks, a novel DBM-Bi LSTM-based classification model has been introduced. Which incorporates DBM for binary classification and Bi-LSTM for multi-class classification to classify the various attacks. Finally, the performance of the classifier in terms of recall, precision, F1-Score, and accuracy are evaluated and compared. The proposed method achieved high accuracy of 98%.     

Intrusion-Tolerant Server Architecture for Survivable Services

Survivable systems are increasingly needed in a wide range of applications. As a step toward realizing survivable systems, this paper presents architecture of intrusion-tolerant servers. It is to deliver intended services transparently to the clients even when a computing node fails due to failures, intrusions, and other threats. In order to deliver only secure results to the client, we need an algorithm to decide agreement on results from replicated servers. For this purpose, a secure and practical decentralized voting algorithm for the architecture is proposed in the paper. Through the experiments on a test-bed, especially, for web services, the approach turned out very effective in terms of extra cost and considered to be able to cope with both confidentiality and integrity attacks.     

Intrusion-tolerant server architecture for survivable services

Survivable systems are increasingly needed in a wide range of applications. As a step toward realizing survivable systems, this paper presents architecture of intrusion-tolerant servers. It is to deliver intended services transparently to the clients even when a computing node fails due to failures, intrusions, and other threats. In order to deliver only secure results to the client, we need an algorithm to decide agreement on results from replicated servers. For this purpose, a secure and practical decentralized voting algorithm for the architecture is proposed in the paper. Through the experiments on a test-bed, especially, for web services, the approach turned out very effective in terms of extra cost and considered to be able to cope with both confidentiality and integrity attacks.     

Web服务器性能评测

Web服务器性能评测是一种理解Web服务器对不同负载反应能力的方法，它对Web服务器的容量规划和性能增强有很大的帮助。讨论了Web服务器性能评测的原理、方法、难点及解决方案，介绍了基于Web负载的特点、ON／OFF源模型及浏览器／服务器体系结构，开发了一个Web服务器性能评测工具－WSBench。WSBench产生渐近自相似的HTTP请求序列，从静态文档、动态文档（没有数据库存取）、动态文档（有数据库存取）及前三者根据Zipf法则的组合4个层次来评测Web服务器的性能。性能测试结果表现为每秒请求数、每秒字节数和往返时间3个指标。最后讨论了Web服务器性能问题及使用WSBench测得的指标来建议Web服务器性能增强可以采用的方法。     

IoTGuardEye:一种面向物联网服务的Web攻击检测方法

在包括物联网(Internet of Things,IoT)设备的绝大部分边缘计算应用中,基于互联网应用技术(通常被称为Web技术)开发的应用程序接口(Application Programming Interface,API)是设备与远程服务器进行信息交互的核心。相比传统的Web应用,大部分用户无法直接接触到边缘设备使用的API,使得其遭受的攻击相对较少。但随着物联网设备的普及,针对API的攻击逐渐成为热点。因此,文中提出了一种面向物联网服务的Web攻击向量检测方法,用于对物联网服务收到的Web流量进行检测,并挖掘出其中的恶意流量,从而为安全运营中心(Security Operation Center,SOC)提供安全情报。该方法在对超文本传输协议(Hypertext Transfer Protocol,HTTP)请求的文本序列进行特征抽取的基础上,针对API请求的报文格式相对固定的特点,结合双向长短期记忆网络(Bidirectional Long Short-Term Memory,BLSTM)实现对Web流量的攻击向量检测。实验结果表明,相比基于规则的Web应用防火墙(Web Application Firewall,WAF)和传统的机器学习方法,所提方法针对面向物联网服务API的攻击具有更好的识别能力。     

A novel approach to visualize web anomaly attacks in pervasive computing environment

These days, a pervasive computing environment is a rapidly changing trend towards increasingly always-on connected computing devices in the convergence environment. In a pervasive computing environment, there are various multimedia web services and communications for various devices in order to provide interesting and invaluable information to users. Meanwhile, providing a wide variety of the web-based multimedia services and communications may cause various security threats and abnormal behaviors. In this paper, a multimedia visualization approach for pervasive computing environment is proposed which analyzes HTTP request and response header information to detect and visualize multimedia web attacks based on the Bayesian method. We conducted a few cases’ experiment for the verification of the proposed approach in a real environment. The experimental results such as web attack detection visualization, scanning and password attack visualization, and attacker’s position tracking visualization verify the usability of the proposed approach.     

An approximation-based load-balancing algorithm with admission control for cluster web servers with dynamic workloads

The growth of web-based applications in business and e-commerce is building up demands for high performance web servers for better throughputs and lower user-perceived latency. These demands are leading to a widespread substitution of powerful single servers by robust newcomers, cluster web servers, in many enterprise companies. In this respect the load-balancing algorithms play an important role in boosting the performance of cluster servers. The previous load-balancing algorithms which were designed for the handling of static contents in web services suffer from significant performance degradation under dynamic and database-driven workloads. Regarding this, we propose an approximation-based load-balancing algorithm with admission control for cluster-based web servers in this study. Since it is difficult to accurately determine the loads of web servers through feedbacks from distributed agents in web servers, we propose an analytical model of a web server to estimate the web servers’ loads. To achieve this, the algorithm classifies requests based on their service times and track numbers of outstanding requests for each class of each web server node and also based on their resource demands to dynamically estimate the loads of each node. For the error handling of the model a proportional integral (PI) controller from control theory is used. Then the estimated available capacity of each web server is used for load balancing and admission control decisions. The implementation results with a standard benchmark confirm the effectiveness of the proposed scheme, which improves both the mean response time and the throughput of the cluster compared to rival load-balancing algorithms, and also avoids situations in which the cluster is overloaded, even when the request rates are beyond the cluster capacity.     

Securing Cloud Computing from Flash Crowd Attack Using Ensemble Intrusion Detection System

Flash Crowd attacks are a form of Distributed Denial of Service (DDoS) attack that is becoming increasingly difficult to detect due to its ability to imitate normal user behavior in Cloud Computing (CC). Botnets are often used by attackers to perform a wide range of DDoS attacks. With advancements in technology, bots are now able to simulate DDoS attacks as flash crowd events, making them difficult to detect. When it comes to application layer DDoS attacks, the Flash Crowd attack that occurs during a Flash Event is viewed as the most intricate issue. This is mainly because it can imitate typical user behavior, leading to a substantial influx of requests that can overwhelm the server by consuming either its network bandwidth or resources. Therefore, identifying these types of attacks on web servers has become crucial, particularly in the CC. In this article, an efficient intrusion detection method is proposed based on White Shark Optimizer and ensemble classifier (Convolutional Neural Network (CNN) and LighGBM). Experiments were conducted using a CICIDS 2017 dataset to evaluate the performance of the proposed method in real-life situations. The proposed IDS achieved superior results, with 95.84% accuracy, 96.15% precision, 95.54% recall, and 95.84% F1 measure. Flash crowd attacks are challenging to detect, but the proposed IDS has proven its effectiveness in identifying such attacks in CC and holds potential for future improvement.     

ACER: detecting Shadowsocks server based on active probe technology

Anonymous server is created for hiding the information of hosts when they are surfing the Internet, such as Tor, Shadowsocks, etc. It is quite difficult to identify these servers, which provides potential criminals with opportunities to commit crime. Also, hackers can make use of these servers to threaten public network security, such as DDoS and Phishing attacks. Hence, the study of identifying these servers is pretty crucial. Current works on detecting Shadowsocks servers are mostly based on the features of servers’ data stream combined with machine learning. However, they are passive methods because they can only be established when the servers are in connection state. Therefore, we propose a new system named ACER, which AC means active and ER means expert, to detect these servers. Besides, we introduce XGBoost algorithm to process the data stream to optimize the detection. The method can recognize more Shadowsocks servers actively instead of monitoring the communication tunnel passively to identify the servers. The experiment result has achieved an accuracy of 94.63% by taking proposed framework and 1.20% more accurate than other existing solutions. We hope to provide a novel solution for those who are conducting research in this area, and provide a detection scheme for network censors to block illegal servers at the same time.

     

Utilizing bloom filters for detecting flooding attacks against SIP based services

Any application or service utilizing the Internet is exposed to both general Internet attacks and other specific ones. Most of the times the latter are exploiting a vulnerability or misconfiguration in the provided service and/or in the utilized protocol itself. Consequently, the employment of critical services, like Voice over IP (VoIP) services, over the Internet is vulnerable to such attacks and, on top of that, they offer a field for new attacks or variations of existing ones. Among the various threats–attacks that a service provider should consider are the flooding attacks, at the signaling level, which are very similar to those against TCP servers but have emerged at the application level of the Internet architecture. This paper examines flooding attacks against VoIP architectures that employ the Session Initiation Protocol (SIP) as their signaling protocol. The focus is on the design and implementation of the appropriate detection method. Specifically, a bloom filter based monitor is presented and a new metric, named session distance, is introduced in order to provide an effective protection scheme against flooding attacks. The proposed scheme is evaluated through experimental test bed architecture under different scenarios. The results of the evaluation demonstrate that the required time to detect such an attack is negligible and also that the number of false alarms is close to zero.     

一种面向DDoS攻击的网络安全态势评估方法

从网络拥塞对应用服务器及网络结构造成的影响出发,引入图论的相关算法,提出一种面向DDoS攻击的网络安全态势评估方法,根据拥塞链路与服务器的距离以及拥塞链路是否处于网络映射图的最小边割集内,计算攻击行为对网络安全态势的影响值,以此进行态势的量化分析。最后使用网络仿真工具验证了该方法的适用性。     

Decentralized multi-dimensional alert correlation for collaborative intrusion detection

The growth in coordinated network attacks such as scans, worms and distributed denial-of-service (DDoS) attacks is a profound threat to the security of the Internet. Collaborative intrusion detection systems (CIDSs) have the potential to detect these attacks, by enabling all the participating intrusion detection systems (IDSs) to share suspicious intelligence with each other to form a global view of the current security threats. Current correlation algorithms in CIDSs are either too simple to capture the important characteristics of attacks, or too computationally expensive to detect attacks in a timely manner. We propose a decentralized, multi-dimensional alert correlation algorithm for CIDSs to address these challenges. A multi-dimensional alert clustering algorithm is used to extract the significant intrusion patterns from raw intrusion alerts. A two-stage correlation algorithm is used, which first clusters alerts locally at each IDS, before reporting significant alert patterns to a global correlation stage. We introduce a probabilistic approach to decide when a pattern at the local stage is sufficiently significant to warrant correlation at the global stage. We then implement the proposed two-stage correlation algorithm in a fully distributed CIDS. Our experiments on a large real-world intrusion data set show that our approach can achieve a significant reduction in the number of alert messages generated by the local correlation stage with negligible false negatives compared to a centralized scheme. The proposed probabilistic threshold approach gains a significant improvement in detection accuracy in a stealthy attack scenario, compared to a naive scheme that uses the same threshold at the local and global stages. A large scale experiment on PlanetLab shows that our decentralized architecture is significantly more efficient than a centralized approach in terms of the time required to correlate alerts.