Application of reinforcement learning for security enhancement in cognitive radio networks

Cognitive radio network (CRN) enables unlicensed users (or secondary users, SUs) to sense for and opportunistically operate in underutilized licensed channels, which are owned by the licensed users (or primary users, PUs). Cognitive radio network (CRN) has been regarded as the next-generation wireless network centered on the application of artificial intelligence, which helps the SUs to learn about, as well as to adaptively and dynamically reconfigure its operating parameters, including the sensing and transmission channels, for network performance enhancement. This motivates the use of artificial intelligence to enhance security schemes for CRNs. Provisioning security in CRNs is challenging since existing techniques, such as entity authentication, are not feasible in the dynamic environment that CRN presents since they require pre-registration. In addition these techniques cannot prevent an authenticated node from acting maliciously. In this article, we advocate the use of reinforcement learning (RL) to achieve optimal or near-optimal solutions for security enhancement through the detection of various malicious nodes and their attacks in CRNs. RL, which is an artificial intelligence technique, has the ability to learn new attacks and to detect previously learned ones. RL has been perceived as a promising approach to enhance the overall security aspect of CRNs. RL, which has been applied to address the dynamic aspect of security schemes in other wireless networks, such as wireless sensor networks and wireless mesh networks can be leveraged to design security schemes in CRNs. We believe that these RL solutions will complement and enhance existing security solutions applied to CRN To the best of our knowledge, this is the first survey article that focuses on the use of RL-based techniques for security enhancement in CRNs.     

Reinforcement learning models for scheduling in wireless networks

The dynamicity of available resources and network conditions, such as channel capacity and traffic characteristics, have posed major challenges to scheduling in wireless networks. Reinforcement learning (RL) enables wireless nodes to observe their respective operating environment, learn, and make optimal or near-optimal scheduling decisions. Learning, which is the main intrinsic characteristic of RL, enables wireless nodes to adapt to most forms of dynamicity in the operating environment as time goes by. This paper presents an extensive review on the application of the traditional and enhanced RL approaches to various types of scheduling schemes, namely packet, sleep-wake and task schedulers, in wireless networks, as well as the advantages and performance enhancements brought about by RL. Additionally, it presents how various challenges associated with scheduling schemes have been approached using RL. Finally, we discuss various open issues related to RL-based scheduling schemes in wireless networks in order to explore new research directions in this area. Discussions in this paper are presented in a tutorial manner in order to establish a foundation for further research in this field.     

Scalable continuous object detection and tracking in sensor networks

With the advancement of MEMS technologies, sensor networks have opened up broad application prospects. An important issue in wireless sensor networks is object detection and tracking, which typically involves two basic components, collaborative data processing and object location reporting. The former aims to have sensors collaborating in determining a concise digest of object location information, while the latter aims to transport a concise digest to sink in a timely manner. This issue has been intensively studied in individual objects, such as intruders. However, the characteristic of continuous objects has posed new challenges to this issue. Continuous objects can diffuse, increase in size, or split into multiple continuous objects, such as a noxious gas. In this paper, a scalable, topology-control-based approach for continuous object detection and tracking is proposed. Extensive simulations are conducted, which show a significant improvement over existing solutions.     

Efficient corona training protocols for sensor networks

Phenomenal advances in nano-technology and packaging have made it possible to develop miniaturized low-power devices that integrate sensing, special-purpose computing, and wireless communications capabilities. It is expected that these small devices, referred to as sensors, will be mass-produced and deployed, making their production cost negligible. Due to their small form factor and modest non-renewable energy budget, individual sensors are not expected to be GPS-enabled. Moreover, in most applications, exact geographic location is not necessary, and all that the individual sensors need is a coarse-grain location awareness. The task of acquiring such a coarse-grain location awareness is referred to as training. In this paper, two scalable energy-efficient training protocols are proposed for massively-deployed sensor networks, where sensors are initially anonymous and unaware of their location. The training protocols are lightweight and simple to implement; they are based on an intuitive coordinate system imposed onto the deployment area which partitions the anonymous sensors into clusters where data can be gathered from the environment and synthesized under local control.     

An appraisal and design of a multi-agent system based cooperative wireless intrusion detection computational intelligence technique

The deployment of wireless sensor networks and mobile ad-hoc networks in applications such as emergency services, warfare and health monitoring poses the threat of various cyber hazards, intrusions and attacks as a consequence of these networks’ openness. Among the most significant research difficulties in such networks safety is intrusion detection, whose target is to distinguish between misuse and abnormal behavior so as to ensure secure, reliable network operations and services. Intrusion detection is best delivered by multi-agent system technologies and advanced computing techniques. To date, diverse soft computing and machine learning techniques in terms of computational intelligence have been utilized to create Intrusion Detection and Prevention Systems (IDPS), yet the literature does not report any state-of-the-art reviews investigating the performance and consequences of such techniques solving wireless environment intrusion recognition issues as they gain entry into cloud computing. The principal contribution of this paper is a review and categorization of existing IDPS schemes in terms of traditional artificial computational intelligence with a multi-agent support. The significance of the techniques and methodologies and their performance and limitations are additionally analyzed in this study, and the limitations are addressed as challenges to obtain a set of requirements for IDPS in establishing a collaborative-based wireless IDPS (Co-WIDPS) architectural design. It amalgamates a fuzzy reinforcement learning knowledge management by creating a far superior technological platform that is far more accurate in detecting attacks. In conclusion, we elaborate on several key future research topics with the potential to accelerate the progress and deployment of computational intelligence based Co-WIDPSs.     

SeCA: A framework for Secure Channel Assignment in wireless mesh networks

To maximize the available throughput in multi-channel multi-radio wireless mesh networks (WMNs), it is a critical issue to design a channel assignment scheme efficiently utilizing orthogonal channels. However, most channel assignment schemes are vulnerable to the misbehaviors of nodes participating in channel assignment, and existing secure channel assignment schemes do not address all of the vulnerabilities. In this paper, we address the threats to channel assignment in WMNs resulting from node misbehaviors and present a generic verification framework to detect such misbehaviors. We develop a concrete verification scheme based on this framework and an existing distributed channel assignment scheme. We validate our approach by implementing the verification scheme and evaluating it through simulation. The results show that our approach improves misbehavior detection with minimum performance overhead.     

Managing cohort movement of mobile sensors via GPS-free and compass-free node localization

A critical problem in mobile ad hoc wireless sensor networks is each node’s awareness of its position relative to the network. This problem is known as localization. In this paper, we introduce a variant of this problem, directional localization, where each node must be aware of both its position and orientation relative to its neighbors. Directional localization is relevant for applications that require uniform area coverage and coherent movement. Using global positioning systems for localization in large scale sensor networks may be impractical in enclosed spaces, and might not be cost effective. In addition, a set of pre-existing anchors with globally known positions may not always be available. In this context, we propose two distributed algorithms based on directional localization that facilitate the collaborative movement of nodes in a sensor network without the need for global positioning systems, seed nodes or a pre-existing infrastructure such as anchors with known positions. Our first algorithm, GPS-free Directed Localization (GDL) assumes the availability of a simple digital compass on each sensor node. We relax this requirement in our second algorithm termed GPS- and Compass-free Directed Localization (GCDL). Through experimentation, we demonstrate that our algorithms scale well for large numbers of nodes and provide convergent localization over time, despite errors introduced by motion actuators and distance measurements. In addition, we introduce mechanisms to preserve swarm formation during directed sensor network mobility. Our simulations confirm that, in a number of realistic scenarios, our algorithms provide for a mobile sensor network that preserves its formation over time, irrespective of speed and distance traveled. We also present our method to organize the sensor nodes in a polygonal geometric shape of our choice even in noisy environments, and investigate the possible uses of this approach in search-and-rescue type of missions.     

Swarm intelligence based routing protocol for wireless sensor networks: Survey and future directions

Swarm intelligence is a relatively novel field. It addresses the study of the collective behaviors of systems made by many components that coordinate using decentralized controls and self-organization. A large part of the research in swarm intelligence has focused on the reverse engineering and the adaptation of collective behaviors observed in natural systems with the aim of designing effective algorithms for distributed optimization. These algorithms, like their natural systems of inspiration, show the desirable properties of being adaptive, scalable, and robust. These are key properties in the context of network routing, and in particular of routing in wireless sensor networks. Therefore, in the last decade, a number of routing protocols for wireless sensor networks have been developed according to the principles of swarm intelligence, and, in particular, taking inspiration from the foraging behaviors of ant and bee colonies. In this paper, we provide an extensive survey of these protocols. We discuss the general principles of swarm intelligence and of its application to routing. We also introduce a novel taxonomy for routing protocols in wireless sensor networks and use it to classify the surveyed protocols. We conclude the paper with a critical analysis of the status of the field, pointing out a number of fundamental issues related to the (mis) use of scientific methodology and evaluation procedures, and we identify some future research directions.     

Survey of network coding-aware routing protocols in wireless networks

In recent times, there have been many advances in the field of information theory and wireless ad hoc network technologies. Regarding information theory progression and its connection with wireless ad hoc networks, this study presents fundamental concepts related to the application of the state-of-the-art Network Coding (NC) within wireless ad hoc networks in the context of routing. To begin with, this paper briefly describes opportunistic routing and identifies differentiation between NC-aware and NC-based routing mechanisms in wireless ad hoc networks. However, our main focus is to provide a survey of available NC-aware routing protocols that make forwarding decisions based on the information of available coding opportunities across several routes within wireless ad hoc networks. The taxonomy and characteristics of various representative NC-aware routing protocols will also be discussed. In summary, we provide a comparison of available NC-aware routing schemes and conclude that NC-aware routing techniques have several advantages over traditional routing in terms of high throughput, high reliability, and lower delay in a wireless scenario. To the best of our knowledge, this work is the first that provides comprehensive discussion about NC-aware routing protocols.     

一种高吞吐量的IEEE 802．11Mesh网AP选择算法

现有的IEEE802．11mesh网访问点（AP）选择算法仅仅基于对mesh用户周围链路质量的测量，无法使用户获得高性能的Internet访问。本文提出了一种基于期望传输吞吐量（Expected Transmission Throughput，EXT）的AP选择算法，该算法同时考虑了mesh路由器访问网关的能力以及用户与路由器的连接时间，目的是使用户选择能够获得最大吞吐量的路由器进行连接。同时，本文针对mesh网络特点和用户的移动方式给出了实际可行的算法实现过程。本文使用NS2对算法进行了仿真，结果证明与传统的基于接收信号强度指示（RSSI）的算法相比，基于EXT的AP选择算法可以使用户的访问吞吐量得到较大提高。     

组合网--第三代网络之上的一个台阶

由于当今技术的发展 ,形形色色基于数据通信的服务也相应而出。例如 ,基于多媒体的远程教育或影视节目、可视电话以及基于网络的交互式电子游戏。然而目前所有的无线网络都是针对某种特定的服务而设计及优化的 ,各个无线网络有着不同的特性。例如 ,不同的无线网支持的带宽不同 ,对应用软件提供的服务质量不同 ,对终端移动性支持的程度不同 ,以及为传输每比特所花费的代价不同。每一个无线网可有效地支持一种或几种数据服务。然而 ,没有一个无线网络可有效地支持所有基于数据通信的服务。不仅如此 ,在欧洲、美洲及亚洲即将推出的第三代无线网络也不能满足所有数据通信的要求。因此 ,如今科技发展的方向是 :将各种用于不同服务的无线网络融合成一体来支持形形色色的数据服务[1～ 3 ] 。将阐述组合网的概念 ,描述组合网的结构 ,并且提出一个基于第三代无线网络之上的组合网。     

CrimeSPOT: A language and runtime for developing active wireless sensor network applications

Advances in wireless sensing and actuation technology allow embedding significant amounts of application logic inside wireless sensor networks. Such active WSN applications are more autonomous, but are significantly more complex to implement. Event-based middleware lends itself to implementing these applications. It offers developers fine-grained control over how an individual node interacts with the other nodes of the network. However, this control comes at the cost of event handlers which lack composability and violate software engineering principles such as separation of concerns. In this paper, we present CrimeSPOT as a domain-specific language for programming WSN applications on top of event-driven middleware. Its node-centric features enable programming a node’s interactions through declarative rules rather than event handlers. Its network-centric features support reusing code within and among WSN applications. Unique to CrimeSPOT is its support for associating application-specific semantics with events that carry sensor readings. These preclude transposing existing approaches that address the shortcomings of event-based middleware to the domain of wireless sensor networks. We provide a comprehensive overview of the language and the implementation of its accompanying runtime. The latter comprises several extensions to the Rete forward chaining algorithm. We evaluate the expressiveness of the language and the overhead of its runtime using small, but representative active WSN applications.     

A dynamic programming approach: Improving the performance of wireless networks

Traditional wireless networks focus on transparent data transmission where the data are processed at either the source or destination nodes. In contrast, the proposed approach aims at distributing data processing among the nodes in the network thus providing a higher processing capability than a single device. Moreover, energy consumption is balanced in the proposed scheme since the energy intensive processing will be distributed among the nodes. The performance of a wireless network is dependent on a number of factors including the available energy, energy–efficiency, data processing delay, transmission delay, routing decisions, security architecture etc. Typical existing distributed processing schemes have a fixed node or node type assigned to the processing at the design phase, for example a cluster head in wireless sensor networks aggregating the data. In contrast, the proposed approach aims to virtualize the processing, energy, and communication resources of the entire heterogeneous network and dynamically distribute processing steps along the communication path while optimizing performance. Moreover, the security of the communication is considered an important factor in the decision to either process or forward the data. Overall, the proposed scheme creates a wireless “computing cloud” where the processing tasks are dynamically assigned to the nodes using the Dynamic Programming (DP) methodology. The processing and transmission decisions are analytically derived from network models in order to optimize the utilization of the network resources including: available energy, processing capacity, security overhead, bandwidth etc. The proposed DP-based scheme is mathematically derived thus guaranteeing performance. Moreover, the scheme is verified through network simulations.     

无线传感器网络移动性支持问题的研究

在传统的无线传感器网络研究中,用户、汇聚节点及传感器节点都被认为是静态的,这与目前应用中节点需要较强的移动特性并不相符。分析了传感器网络中移动性应用的场景,从网络体系结构的观点出发,总结了当前在无线传感器网络移动性支持方面所取得的研究进展,并进一步分析了关键算法和策略。     

Efficient and adaptive resource scheduling in IEEE 802.16j transparent relay networks

Relaying is regarded as a promising technique for enhancing the system throughput and coverage of emerging broadband wireless access networks. While next generation broadband wireless standardization bodies (e.g., LTE-advanced and IEEE 802.16j) have specified the support of relays in general terms, the problem of allocating network resources in an efficient and effective manner remains a challenging issue. In this study, the utility-based resource scheduling optimization problem for IEEE 802.16j transparent relay networks is formulated mathematically and proven to be NP-hard. Two efficient heuristic algorithms, namely Relay Resource Scheduling (RRS) and Adaptive Relay Resource Scheduling (ARRS), are proposed to resolve the resource scheduling problem. RRS maximizes the system utility in networks containing direct and relay mobile stations by exploiting the benefits of optional transmissions. ARRS enhances upon the performance of RRS by adaptively determining the split ratio between the access zone and the transparent zone in the downlink subframe. The computational complexities of both algorithms are shown to be polynomial and proportional to the number of links and subchannels in the network. The simulation results show that the proposed algorithms achieve significantly better throughput and fairness performance than existing schemes such as Partial Proportional Fairness (PPF) and GenArgMAX.     

Discrete logarithm based additively homomorphic encryption and secure data aggregation

At PKC 2006, Chevallier-Mames, Paillier, and Pointcheval proposed discrete logarithm based encryption schemes that are partially homomorphic, either additively or multiplicatively and announced an open problem: finding a discrete logarithm based cryptosystem that would help realize fully additive or multiplicative homomorphism. In this study, we achieve this goal by enclosing two opposite settings on the discrete logarithm problems (DLP) simultaneously: the first setting is that DLP over Z_p0 (where p₀ − 1 is smooth) is used to encode messages, while the second setting is that DLP over Z_p (where p − 1 is non-smooth, i.e., containing large prime factors) is used to encrypt plaintexts. Then, based on the proposed scheme, novel protocols for secure data aggregation in wireless sensor networks are presented. Finally, taking Paillier’s factoring-based additively homomorphic encryption schemes as the reference framework, we present detailed performance comparisons and further enhancement.     

REUSE: A combined routing and link scheduling mechanism for wireless mesh networks

Increasing the capacity of wireless mesh networks has motivated numerous studies. In this context, the cross-layer optimization techniques involving joint use of routing and link scheduling are able to provide better capacity improvements. Most works in the literature propose linear programming models to combine both mechanisms. However, this approach has high computational complexity and cannot be extended to large-scale networks. Alternatively, algorithmic solutions are less complex and can obtain capacity values close to the optimal. Thus, we propose the REUSE algorithm, which combines routing and link scheduling and aims to increase throughput capacity in wireless mesh networks. Through simulations, the performance of the proposal is compared to a developed linear programming model, which provides optimal results, and to other proposed mechanisms in the literature that also deal with the problem algorithmically. We observed higher values of capacity in favor of our proposal when compared to the benchmark algorithms.     

Mesh-Mon: A multi-radio mesh monitoring and management system

Mesh networks are a potential solution for providing communication infrastructure in an emergency. They can be rapidly deployed by first responders in the wake of a major disaster to augment an existing wireless or wired network. We imagine a mesh node with multiple radios embedded in each emergency vehicle arriving at the site to form the backbone of a mobile wireless mesh. The ability of such a mesh network to monitor itself, diagnose faults and anticipate problems are essential features for its sustainable operation. Typical SNMP-based centralized solutions introduce a single point of failure and are unsuitable for managing such a network. Mesh-Mon is a decentralized monitoring and management system designed for such a mobile, rapidly deployed, unplanned mesh network and works independently of the underlying mesh routing protocol. Mesh-Mon nodes are designed to actively cooperate and use localized algorithms to predict, detect, diagnose and resolve network problems in a scalable manner. Mesh-Mon is independent of the underlying routing protocol and can operate even if the mesh routing protocol completely fails. One novel aspect of our approach is that we employ mobile users of the mesh, running software called Mesh-Mon-Ami, to ferry management packets between physically-disconnected partitions in a delay-tolerant-network manner. The main contributions of this paper are the design, implementation and evaluation of a comprehensive monitoring and management architecture that helps a network administrator proactively identify, diagnose and resolve a range of issues that can occur in a dynamic mesh network. In experiments on Dart-Mesh, our 16-node indoor mesh testbed, we found Mesh-Mon to be effective in quickly diagnosing and resolving a variety of problems with high accuracy, without adding significant management overhead.     

Routing in wireless sensor networks for wind turbine monitoring

Smart wireless sensor devices are rapidly emerging as key enablers of the next evolution in wind turbine monitoring. The potential for in-situ monitoring of turbine elements, employing methodologies that are not possible with existing wired technology, make it possible to attain new levels of granularity and autonomy in the monitoring of these structures. Wireless sensor devices are limited in terms of communication by the range of their radio modules and, thus, need to form networks in order to transfer data from distant points. Routing protocols are primary enablers of such ad hoc wireless sensor networks and these require the implementation of reliable and energy-efficient mechanisms to maximize network reliability and availability. Existing routing protocols cannot be directly applied to the monitoring of wind turbines without addressing the unique context and operational characteristics of these structures in multi-hop wireless communication. This work identifies the potential effects associated with the operation, environment and structure of wind turbines in wireless sensor network multi-hop communication, and proposes and evaluates a reliable routing protocol for wireless sensor networks employed in these domains.     

无线传感器网络能量高效综述

综述了无线传感器网络中能量高效的相关工作。结合其自身特点,首先对传感器网络能耗和能量高效途径进行了分析,然后结合已有研究,从无线传感器节点和无线传感器网络两方面阐述了能量高效策略。