基于机器学习和深度学习的网络流量分类研究

随着互联网技术的不断发展以及网络规模的不断扩大,应用的类别纷繁复杂,新型应用层出不穷。为了保障用户服务质量(QoS)并确保网络安全,准确快速的流量分类是运营商及网络管理者亟须解决的问题。首先给出网络流量分类的问题定义和性能指标;然后分别介绍基于机器学习和基于深度学习的流量分类方法,分析了这些方法的优缺点,并对现存问题进行阐述;接着围绕流量分类线上部署时会遇到的3个问题:数据集问题、新应用识别问题、部署开销问题对相关工作进行阐述与分析,并进一步探讨目前网络流量分类研究面临的挑战;最后对网络流量分类下一步的研究方向进行展望。     

A topology and application‐aware relay path allocation scheme in multipath transport system based on application‐level relay

The ever‐increasing transmission requirements of quality of service (QoS)‐sensitive applications, especially real‐time multimedia applications, can hardly be met by the single path routing protocols. Multipath transmission mechanism is a feasible approach to provide QoS for various applications. On the basis of the general framework of multipath transport system based on application‐level relay, we present a relay path allocation scheme, whose goal is to select suitable relay paths, while balancing the overlay traffic among the different domains and relayers. With the application‐layer traffic optimization service under the standardization within the Internet Engineering Task Force (IETF), the controller has the topology‐aware ability to allocate relay paths with excellent routing performance. To further develop the universality of our method, the controller perceives transmission performance of relay overlay network through relayers' performance detection processes and, thus, has the application‐aware ability to allocate relay paths with excellent transmission performance for different applications by consulting application‐specific transmission metrics. Simulation results demonstrate that the proposed relay path allocation algorithm performs well in allocating superior relay paths and can balance the distribution of overlay traffic across domains in different network situations.     

Adaptive traffic sampling for P2P botnet detection

Peer‐to‐peer (P2P) botnets have become one of the major threats to network security. Most existing botnet detection systems detect bots by examining network traffic. Unfortunately, the traffic volumes typical of current high‐speed Internet Service Provider and enterprise networks are challenging for these network‐based systems, which perform computationally complex analyses. In this paper, we propose an adaptive traffic sampling system that aims to effectively reduce the volume of traffic that P2P botnet detectors need to process while not degrading their detection accuracy. Our system first identifies a small number of potential P2P bots in high‐speed networks as soon as possible, and then samples as many botnet‐related packets as possible with a predefined target sampling rate. The sampled traffic then can be delivered to fine‐grained detectors for further in‐depth analysis. We evaluate our system using traffic datasets of real‐world and popular P2P botnets. The experiments demonstrate that our system can identify potential P2P bots quickly and accurately with few false positives and greatly increase the proportion of botnet‐related packets in the sampled packets while maintain the high detection accuracy of the fine‐grained detectors.     

A compressive sensing‐based network tomography approach to estimating origin–destination flow traffic in large‐scale backbone networks

A traffic matrix can exhibit the volume of network traffic from origin nodes to destination nodes. It is a critical input parameter to network management and traffic engineering, and thus it is necessary to obtain accurate traffic matrix estimates. Network tomography method is widely used to reconstruct end‐to‐end network traffic from link loads and routing matrix in a large‐scale Internet protocol backbone networks. However, it is a significant challenge because solving network tomography model is an ill‐posed and under‐constrained inverse problem. Compressive sensing reconstruction algorithms have been well known as efficient and precise approaches to deal with the under‐constrained inference problem. Hence, in this paper, we propose a compressive sensing‐based network traffic reconstruction algorithm. Taking into account the constraints in compressive sensing theory, we propose an approach for constructing a novel network tomography model that obeys the constraints of compressive sensing. In the proposed network tomography model, a framework of measurement matrix according to routing matrix is proposed. To obtain optimal traffic matrix estimates, we propose an iteration algorithm to solve the proposed model. Numerical results demonstrate that our method is able to pursuit the trace of each origin–destination flow faithfully. Copyright © 2014 John Wiley & Sons, Ltd.     

基于多级分类的网络流量在线识别方法(英文)

Internet traffic classification plays an important role in network management. Many approaches have been proposed to classify different categories of Internet traffic. However, these approaches have specific usage contexts that restrict their ability when they are applied in the current network environment. For example, the port based approach cannot identify network applications with dynamic ports; the deep packet inspection approach is invalid for encrypted network applications; and the statistical based approach is time-consuming. In this paper, a novel technique is proposed to classify different categories of network applications. The port based, deep packet inspection based and statistical based approaches are integrated as a multistage classifier. The experimental results demonstrate that this approach has high recognition rate which is up to 98% and good performance of real-time for traffic identification.     

Fine‐grained traffic classification based on functional separation

Current efforts to classify Internet traffic highlight accuracy. Previous studies have focused on the detection of major applications such as P2P and streaming applications. However, these applications can generate various types of traffic which are often considered as minor and ignorant traffic portions. As network applications become more complex, the price paid for not concentrating on minor traffic classes is in reduction of accuracy and completeness. In this context, we propose a fine‐grained traffic classification scheme and its detailed method, called functional separation. Our proposal can detect, according to functionalities, different types of traffic generated by a single application and should increase completeness by reducing the amount of undetected traffic. We verify our method with real‐world traffic. Our performance comparison against existing DPI‐based classification frameworks shows that the fine‐grained classification scheme achieves consistently higher accuracyand completeness. Copyright © 2013 John Wiley & Sons, Ltd.     

Game Traffic Classification Using Statistical Characteristics at the Transport Layer

The pervasive game environments have activated explosive growth of the Internet over recent decades. Thus, understanding Internet traffic characteristics and precise classification have become important issues in network management, resource provisioning, and game application development. Naturally, much attention has been given to analyzing and modeling game traffic. Little research, however, has been undertaken on the classification of game traffic. In this paper, we perform an interpretive traffic analysis of popular game applications at the transport layer and propose a new classification method based on a simple decision tree, called an alternative decision tree (ADT), which utilizes the statistical traffic characteristics of game applications. Experimental results show that ADT precisely classifies game traffic from other application traffic types with limited traffic features and a small number of packets, while maintaining low complexity by utilizing a simple decision tree.     

互联网骨干网流量的建模和特征

With enormous growth of the number of Internet users and appearance of new applications, characterization of Internet traffic has attracted more and more attention and has become one of the major challenging issues in telecommunication network over the past few years. In this paper, we study the network traffic pattern of the aggregate traffic and of specific application traffic, especially the popular applications such as P2P, VoIP that contribute most network traffic. Our study verified that majority Internet backbone traffic is contributed by a small portion of users and a power function can be used to approximate the contribution of each user to the overall traffic. We show that P2P applications are the dominant traffic contributor in current Internet Backbone of China. In addition, we selectively present the traffic pattern of different applications in detail.     

UARA in edge routers: an effective approach to user fairness and traffic shaping

The ever‐increasing share of the peer‐to‐peer (P2P) traffic flowing in the Internet has unleashed new challenges to the quality of service provisioning. Striving to accommodate the rise of P2P traffic or to curb its growth has led to many schemes being proposed: P2P caches, P2P filters, ALTO mechanisms and re‐ECN. In this paper, we propose a scheme named ‘UARA:textbfUser/ A pplication‐aware R ED‐based A QM’ which has a better perspective on the problem: UARA is proposed to be implemented at the edge routers providing real‐time near‐end‐user traffic shaping and congestion avoidance. UARA closes the loopholes exploited by the P2P traffic by bringing under control the P2P users who open and use numerous simultaneous connections. In congestion times, UARA monitors the flows of each user and caps the bandwidth used by ‘power users’ which leads to the fair usage of network resources. While doing so, UARA also prioritizes the real‐time traffic of each user, further enhancing the average user quality of experience (QoE). UARA hence centralizes three important functionalities at the edge routers: (1) congestion avoidance; (2) providing user fairness; (3) prioritizing real‐time traffic. The simulation results indicate that average user QoE is significantly improved in congestion times with UARA at the edge routers. Copyright © 2011 John Wiley & Sons, Ltd.     

A reconstructing approach to end‐to‐end network traffic based on multifractal wavelet model

This paper studies the reconstructing method of end‐to‐end network traffic. Due to the development of current communication networks, our networks become more complex and heterogeneous. Meanwhile, because of time‐varying nature and spatio‐temporal correlations of the end‐to‐end network traffic, to obtain it accurately is a great challenge. We propose to exploit discrete wavelet transforms and multifractal analysis to reconstruct the end‐to‐end network traffic from time–frequency domain. First, its time–frequency properties can be characterized in detail by discrete wavelet transforms. And then, we combine discrete wavelet transforms and multifractal analysis to reconstruct end‐to‐end network traffic from link loads. Furthermore, our method needs to measure end‐to‐end network traffic to build the statistical model named multifractal wavelet model. Finally, simulation results from the real backbone networks suggest that our method can reconstruct the end‐to‐end network traffic more accurately than previous methods. Copyright © 2013 John Wiley & Sons, Ltd.     

Header signature maintenance for Internet traffic identification

Various traffic identification methods have been proposed with the focus on application‐level traffic analysis. Header signature–based identification using the 3‐tuple (Internet Protocol address, port number, and L4 protocol) within a packet header has garnered a lot of attention because it overcomes the limitations faced by the payload‐based method, such as encryption, privacy concerns, and computational overhead. However, header signature–based identification does have a significant flaw in that the volume of header signatures increases rapidly over time as a number of applications emerge, evolve, and vanish. In this article, we propose an efficient method for header signature maintenance. Our approach automatically constructs header signatures for traffic identification and only retains the most significant signatures in the signature repository to save memory space and to improve matching speed. For the signature maintenance, we define a new metric, the so‐called signature weight, that reflects its potential ability to identify traffic. Signature weight is periodically calculated and updated to adapt to the changes of network environment. We prove the feasibility of the proposed method by developing a prototype system and deploying it in a real operational network. Finally, we prove the superiority of our signature maintenance method through comparison analysis against other existing methods on the basis of various evaluation metrics.     

Exploiting scheduling and free‐riding for offline downloading in BitTorrent networks

Recently, the peer‐to‐peer (P2P) architecture has become a popular scheme for Internet users to rapidly exchange files. As reported in previous studies, P2P traffic accounts for a significant portion of overall Internet traffic. Many computing resources, for example, network bandwidth and disk space, may be consumed by P2P clients. Accordingly, in this paper, we devise a novel scheme that integrates advantageous features of both the conventional client–server and the P2P architectures to create an offline downloading service. Specifically, the proposed service acts as an agent that downloads required files from the BitTorrent network without consuming local computing resources. In other words, users can stay offline during the download process. Because the proposed scheme aims to provide service to numerous users at the same time, a proper scheduling technique is adopted to achieve better download performance. Moreover, a free‐riding mechanism is seamlessly incorporated with the proposed priority queuing to facilitate more effective bandwidth utilization. Empirical studies show that our scheme is promising in practical applications. Copyright © 2012 John Wiley & Sons, Ltd.     

An efficient traffic control system using dynamic thresholding techniques in wireless mesh networks

Wireless mesh networks (WMNs) depend on a resilient and high‐performance infrastructure to provide users pervasive Internet access. In WMNs, all Internet traffic will be forwarded to the Internet gateways. Hence, these gateways are generally bottleneck nodes. This work proposes a traffic control technique to reduce the bottleneck problem and increase the utilization of network resources. Our approach provides a traffic control strategy that exploits dynamic techniques to adjust the threshold according to the traffic load of each gateway. The base threshold is defined in order to effectively control the traffic. When the current load exceeds the threshold of a gateway, the traffic redirection strategy is implemented by switching border nodes. The service regions can be adjusted for each gateway based on the traffic load. Furthermore, the proposed dynamic thresholding approaches can distribute the workloads of gateways and maintain the thresholds of any two gateways within a level range, making an in‐band balance of load. Thus, our proposed scheme can handle the unnecessary traffic redirection and reduce the traffic control overhead for various distributions of traffic. Experimental results demonstrate that our scheme outperforms other schemes in terms of packet delivery ratio and efficiency, especially in bursty traffic environments. Copyright © 2010 John Wiley & Sons, Ltd.     

基于深度报文检测和深度流检测的骨干网流量特征分析

Based on the massive data collected with a passive network monitoring equipment placed in China's backbone, we present a deep insight into the network backbone traffic and evaluate various ways for improving traffic classifying efficiency in this paper. In particular, the study has scrutinized the network traffic in terms of protocol types and signatures, flow length, and port distribution, from which meaningful and interesting insights on the current Internet of China from the perspective of both the packet and flow levels are derived. We show that the classification efficiency can be greatly improved by using the information of preferred ports of the network applications. Quantitatively, we find two traffic duration thresholds, with which 40% of TCP flows and 70% of UDP flows can be excluded from classification processing while the impact on classification accuracy is trivial, i.e., the classification accuracy can still reach a high level by saving 85% of the resources.     

An efficient method of P2P traffic identification based on wavelet packet decomposition and kernel principal component analysis

Peer‐to‐peer (P2P) traffic identification is currently an important challenge to network management and measurement. Many approaches based on statistics have been proposed to identify P2P traffic. However, flow features extracted by traditional methods are rough and one‐sided, which might lead to inaccuracy identification of network traffic. Besides, P2P traffic has too many statistical features, which is a challenge to the time complexity and space complexity of the classifier. This work focuses on the study of flow features. First, micro features of flow signals are extracted based on wavelet packet decomposition, and we combine them with the traditional features into combination features. The experimental results show that combination features have better performance than traditional features for P2P traffic identification, and 16 kinds of wavelet functions were tested to find the best one. Second, a feature reduction algorithm based on improved kernel principal component analysis is provided. The results show that the feature reduction algorithm proposed in this paper plays good performance to P2P traffic identification, because it could greatly reduced the number of features while having no affection on identification accuracy. Copyright © 2012 John Wiley & Sons, Ltd.     

互联网流量识别中的基于标签传播的重叠社团发现算法

The increased capacity and availability of the Internet has led to a wide variety of applications. Internet traffic characterization and application i-dentification is important for network management. In this paper, based on detailed flow data collected from the public networks of Internet Service Pro-viders, we construct a flow graph to model the in-teractions among users. Considering traffic from different applications, we analyze the community structure of the flow graph in terms of community size, degree distribution of the community, commu-nity overlap, and overlap modularity. The near line-ar time community detection algorithm in complex networks, the Label Propagation Algorithm (LPA), is extended to the flow graph for application identi-fication. We propose a new initialization and label propagation and update scheme. Experimental re-sults show that the proposed algorithm has high ac-curacy and efficiency.     

NetCube: a comprehensive network traffic analysis model based on multidimensional OLAP data cube

Network traffic monitoring and analysis are essential for effective network operation and resource management. In particular, multidimensional analysis for long‐term traffic data is necessary for comprehensive understanding of the traffic trend and effective quality‐of‐service provision considering the extremely dynamic behavior of the current Internet, where various types of traffic occur from high‐speed network links and greatly increasing number of applications. However, only limited analysis results are provided, as the existing network traffic analysis tools and systems are developed and deployed focusing on their own specialized analysis purposes. Consequently, it is difficult to understand the network comprehensively and deeply, which increases the necessity for multilateral analysis of long‐term traffic data. In this paper, we propose a novel traffic analysis model for large volumes of Internet traffic accumulated over a long period of time. The NetCube, the proposed network traffic analysis model using online analytical processing (OLAP) on a multidimensional data cube, provides an easy and fast way to construct a multidimensional traffic analysis system for comprehensive and detailed analysis of long‐term traffic data by utilizing simple OLAP operations and powerful data‐mining techniques on various abstraction levels of traffic data to complete the analysis purpose. We validate the feasibility and applicability of the proposed NetCube traffic analysis model by implementing a traffic analysis system and applying it to our campus network. Copyright © 2012 John Wiley & Sons, Ltd.     

基于MPLS的流量工程——分布实时网络承载能力估计与分配模型

本文的主要工作是建立基于MPLS的流量工程模型.在商业运行的网络中,通过对已投资设施的充分利用获取竞争优势和商业回报的要求使得流量工程日益成为网络运营中不可缺少的手段.通过借鉴TCP和ATM的优点,结合面向连接和面向非连接两种处理方法的长处,建立一种基于MPLS的流量管理模型,该模型的特点是:管理功能模块边缘化,运输功能模块平面化,在高速,简单的运载核心上运行一个可管理的流量承载平台.模型的有效性通过两个方法得到评估.一是从形式上证明了模型运行状态的稳定性,完备性和对于时间轴的收敛性.二是使用Network Simulator建立模拟的网络环境验证对网络运行状态的优化结果.本文的工作基于以下重要概念:基于聚集的流量、流聚集、流量聚集点和流量分解点、扩充的链路耗费参数定义(承载能力占用率)、分布式公平队列等.     

Quality‐of‐Service Mechanisms for Flow‐Based Routers

In this paper, we propose quality of service mechanisms for flow‐based routers which have to handle several million flows at wire speed in high‐speed networks. Traffic management mechanisms are proposed for guaranteed traffic and non‐guaranteed traffic separately, and then the effective harmonization of the two mechanisms is introduced for real networks in which both traffic types are mixed together. A simple non‐work‐conserving fair queuing algorithm is proposed for guaranteed traffic, and an adaptive flow‐based random early drop algorithm is proposed for non‐guaranteed traffic. Based on that basic architecture, we propose a dynamic traffic identification method to dynamically prioritize traffic according to the traffic characteristics of applications. In a high‐speed router system, the dynamic traffic identification method could be a good alternative to deep packet inspection, which requires handling of the IP packet header and payload. Through numerical analysis, simulation, and a real system experiment, we demonstrate the performance of the proposed mechanisms.