Optimization of relay selection and ergodic capacity in cognitive radio sensor networks with wireless energy harvesting

This paper addresses relay selection and resource allocation issues in cognitive radio sensor networks with wireless energy harvesting. We first consider a three-phase energy harvesting and information transmission protocol based on cooperative decode-and-forward relaying for a secondary system in coexistence with the primary system. In the first phase, the energy-constrained relay harvests energy through radio-frequency signals from the primary source. In the second phase, the destinations decode the primary signal. The relay uses the harvested energy to forward the primary signal and the secondary signal in the third phase. We derive the close-form upper bound of the ergodic capacity of the primary system and propose a relay selection algorithm. In particular, we calculate the critical region to ensure that the ergodic capacity of the primary system is equal or larger than that of the direct system. Finally, numerical results show that the proposed schemes achieve a satisfying performance.     

Application of reinforcement learning for security enhancement in cognitive radio networks

Cognitive radio network (CRN) enables unlicensed users (or secondary users, SUs) to sense for and opportunistically operate in underutilized licensed channels, which are owned by the licensed users (or primary users, PUs). Cognitive radio network (CRN) has been regarded as the next-generation wireless network centered on the application of artificial intelligence, which helps the SUs to learn about, as well as to adaptively and dynamically reconfigure its operating parameters, including the sensing and transmission channels, for network performance enhancement. This motivates the use of artificial intelligence to enhance security schemes for CRNs. Provisioning security in CRNs is challenging since existing techniques, such as entity authentication, are not feasible in the dynamic environment that CRN presents since they require pre-registration. In addition these techniques cannot prevent an authenticated node from acting maliciously. In this article, we advocate the use of reinforcement learning (RL) to achieve optimal or near-optimal solutions for security enhancement through the detection of various malicious nodes and their attacks in CRNs. RL, which is an artificial intelligence technique, has the ability to learn new attacks and to detect previously learned ones. RL has been perceived as a promising approach to enhance the overall security aspect of CRNs. RL, which has been applied to address the dynamic aspect of security schemes in other wireless networks, such as wireless sensor networks and wireless mesh networks can be leveraged to design security schemes in CRNs. We believe that these RL solutions will complement and enhance existing security solutions applied to CRN To the best of our knowledge, this is the first survey article that focuses on the use of RL-based techniques for security enhancement in CRNs.     

Distributed cooperative spectrum sensing in cognitive radio for ad hoc networks

Cognitive radio refers to an intelligent radio with the capability of sensing the radio environment and dynamically reconfiguring the operating parameters. Recent research has focused on using cognitive radios in ad hoc environments. Spectrum sensing is the most important aspect of successful cognitive radio ad hoc network deployment to overcome spectrum scarcity. Multiple cognitive radio users can cooperate to sense the primary user and improve sensing performance. Cognitive radio ad hoc networks are dynamic in nature and have no central point for data fusion. In this paper, gradient-based fully distributed cooperative spectrum sensing in cognitive radio is proposed for ad hoc networks. The licensed band used for TV transmission is considered the primary user. The gradient field changes with the energy sensed by cognitive radios, and the gradient is calculated based on the components, which include energy sensed by secondary users and received from neighbors. The proposed scheme was evaluated from the perspective of reliable sensing, convergence time, and energy consumption. Simulation results demonstrated the effectiveness of the proposed scheme.     

Proportional and double imitation rules for spectrum access in cognitive radio networks

In this paper, we tackle the problem of opportunistic spectrum access in large-scale cognitive radio networks, where the unlicensed Secondary Users (SUs) access the frequency channels partially occupied by the licensed Primary Users (PUs). Each channel is characterized by an availability probability unknown to the SUs. We apply population game theory to model the spectrum access problem and develop distributed spectrum access policies based on imitation, a behavior rule widely applied in human societies consisting of imitating successful behaviors. We develop two imitation-based spectrum access policies based on the basic Proportional Imitation (PI) rule and the more advanced Double Imitation (DI) rule given that a SU can only imitate the other SUs operating on the same channel. A systematic theoretical analysis is presented for both policies on the induced imitation dynamics and the convergence properties of the proposed policies to the Nash equilibrium. Simple and natural, the proposed imitation-based spectrum access policies can be implemented distributedly based on solely local interactions and thus is especially suited in decentralized adaptive learning environments as cognitive radio networks.     

A stackelberg game for spectrum leasing in cooperative cognitive radio networks

According to the property-rights model of cognitive radio, primary users (PUs) who own the spectrum resource have the right to lease part of spectrum to secondary users (SUs) in exchange for appropriate profit. In this paper, we propose a pricing-based spectrum leasing framework between one PU and multiple SUs. In this scenario, the PU attempts to maximize its utility by setting the price of spectrum. Then, the selected SUs have the right to decide their power levels to help PU’s transmission, aiming to obtain corresponding access time. The spectrum leasing problem can be cast into a stackelberg game, where the PU plays the seller-level game and the selected SUs play the buyer-level game. Through analysis based on the backward induction, we prove that there exists a unique equilibrium in the stackelberg game with certain constraints. Numerical results show that the proposed pricing-based spectrum leasing framework is effective, and the performance of both PU and SUs is improved, compared to the traditional mechanism without cooperation.     

Analytical modeling for spectrum handoff decision in cognitive radio networks

Cognitive Radio (CR) is an emerging technology used to significantly improve the efficiency of spectrum utilization. Although some spectrum bands in the primary user’s licensed spectrum are intensively used, most of the spectrum bands remain underutilized. The introduction of open spectrum and dynamic spectrum access lets the secondary (unlicensed) users, supported by cognitive radios; opportunistically utilize the unused spectrum bands. However, if a primary user returns to a band occupied by a secondary user, the occupied spectrum band is vacated immediately by handing off the secondary user’s call to another idle spectrum band. Multiple spectrum handoffs can severely degrade quality of service (QoS) for the interrupted users. To avoid multiple handoffs, when a licensed primary user appears at the engaged licensed band utilized by a secondary user, an effective spectrum handoff procedure should be initiated to maintain a required level of QoS for secondary users. In other words, it enables the channel clearing while searching for target vacant channel(s) for completing unfinished transmission. This paper proposes prioritized proactive spectrum handoff decision schemes to reduce the handoff delay and the total service time. The proposed schemes have been modeled using a preemptive resume priority (PRP) M/G/1 queue to give a high priority to interrupted users to resume their transmission ahead of any other uninterrupted secondary user. The performance of proposed handoff schemes has been evaluated and compared against the existing spectrum handoff schemes. Experimental results show that the schemes developed here outperform the existing schemes in terms of average handoff delay and total service time under various traffic arrival rates as well as service rates.     

A semi-Markov decision process-based joint call admission control for inter-RAT cell re-selection in next generation wireless networks

In the next generation wireless networks (NGWNs), where different radio access technologies (RAT) will coexist and work in collaboration to provide ubiquitous access, a mechanism called Joint Call Admission Control (JCAC) will play an important role by deciding whether or not an incoming service request will be accepted according to an admission constraint as well as determining in which RAT (among the available) it will be connected. In this paper, we propose an optimal JCAC for inter-RAT cell re-selection problem also referred to as initial RAT selection in co-located wireless networks, which supports both real-time services and non-real-time services. To properly meet the JCAC goals, we propose a cost function that weigh two criteria: the blocking cost function, which takes into account the priority of each service class in each RAT, and the alternative acceptance cost, which reflects the multiplicity of RATs working in a collaborative fashion, mandatory in NGWN. We use the framework of Semi-Markov Decision Process (SMDP) to formulate the optimization problem and the value iteration algorithm to compute the optimal policy. Our model still takes into consideration the ratio between the radius of the co-located RATs and shows how it may impact on optimal initial RAT selection. Numerical results, supported by an analysis of the structure of the optimal policy, show that the proposed optimal JCAC selects for real-time service class the biggest RAT and for non-real-time service class the smallest one. This optimal JCAC policy is ratified by the current trend in the design of NGWN and also follows the 3rd Generation Partnership Project (3GPP) expectations.     

Reinforcement learning for context awareness and intelligence in wireless networks: Review, new features and open issues

In wireless networks, context awareness and intelligence are capabilities that enable each host to observe, learn, and respond to its complex and dynamic operating environment in an efficient manner. These capabilities contrast with traditional approaches where each host adheres to a predefined set of rules, and responds accordingly. In recent years, context awareness and intelligence have gained tremendous popularity due to the substantial network-wide performance enhancement they have to offer. In this article, we advocate the use of reinforcement learning (RL) to achieve context awareness and intelligence. The RL approach has been applied in a variety of schemes such as routing, resource management and dynamic channel selection in wireless networks. Examples of wireless networks are mobile ad hoc networks, wireless sensor networks, cellular networks and cognitive radio networks. This article presents an overview of classical RL and three extensions, including events, rules and agent interaction and coordination, to wireless networks. We discuss how several wireless network schemes have been approached using RL to provide network performance enhancement, and also open issues associated with this approach. Throughout the paper, discussions are presented in a tutorial manner, and are related to existing work in order to establish a foundation for further research in this field, specifically, for the improvement of the RL approach in the context of wireless networking, for the improvement of the RL approach through the use of the extensions in existing schemes, as well as for the design and implementation of RL in new schemes.     

Elastic threshold-based admission control for QoS satisfaction with reward optimization for servicing multiple priority classes in wireless networks

We analyze performance characteristics of a class of call admission control (CAC) algorithms designed for servicing multiple priority classes in wireless networks with the goal of quality of service (QoS) satisfaction and reward optimization. By reward, we mean some sort of “value” obtained by the system as a result of servicing multiple priority classes. In this paper we design and evaluate the performance of a new algorithm, elastic threshold-based CAC, in terms of the maximum reward obtainable with QoS satisfaction from servicing multiple priority classes with distinct QoS requirements, and compare it with existing partitioning, threshold, and spillover CAC algorithms. We also develop a heuristic-based search method to determine the best threshold-value sets used for multiple service classes by sequentially adjusting these thresholds based on the reward and rejection rate obtainable vs. QoS constraints of each service class. We demonstrate through test cases and simulation that elastic threshold-based CAC outperforms existing CAC algorithms for QoS satisfaction and reward optimization in solution optimality for serving multiple QoS service classes in wireless networks.