An online power allocation algorithm based on deep reinforcement learning in multibeam satellite systems

Dynamic power allocation (DPA) is the key technique to improve the system throughput by matching the offered capacity with that required among distributed beams in multibeam satellite systems. Existing power allocation studies tend to adopt the metaheuristic optimization algorithms such as the genetic algorithm. The achieved DPA cannot adapt to the dynamic environments due to the varying traffic demands and the channel conditions. To solve this problem, an online algorithm named deep reinforcement learning‐based dynamic power allocation (DRL‐DPA) algorithm is proposed in this paper. The key idea of the proposed DRL‐DPA lies in the online power allocation decision making other than the offline way of the traditional metaheuristic methods. Simulation results show that the proposed DRL‐DPA algorithm can improve the system performance in terms of system throughput and power consumption in multibeam satellite systems.     

面向密集多波束组网的卫星通信系统资源调度算法

研究了高动态、资源受限条件下的卫星通信系统资源调度问题。以时间窗口、卫星功耗、信道数量、用户优先级以及任务突发性为约束,建立了卫星资源调度模型。考虑到传统的蚁群优化算法存在初期搜索速度过慢、局部搜索能力较弱以及易陷入局部最优等缺点,提出了以初始解集构造、额外信息素沉积为核心的改进蚁群优化算法,来求解资源调度问题。仿真实验评估了所提资源调度算法在完成任务的数量、优先级和调度完成时间方面的性能。实验结果表明,所提算法具有较快的收敛速度,且与同类型优化算法相比具有更高的调度效率,适用于调度面向密集多波束组网需求的卫星通信系统资源。     

基于深度强化学习的智能决策方法

针对传统深度强化学习算法难以快速解决长时序复杂任务的问题,提出了一种引入历史信息和人类知识的深度强化学习方法,对经典近端策略优化(Proximal Policy Optimization, PPO)强化学习算法进行改进,在状态空间引入历史状态以反映环境的时序变化特征,在策略模型中基于人类认知增加无效动作掩膜,禁止智能体进行无效探索,提高探索效率,从而提升模型的训练性能。仿真结果表明,所提方法能够有效解决长时序复杂任务的智能决策问题,相比传统的深度强化学习算法可显著提高模型收敛效果。     

基于深度强化学习的异构云无线接入网自适应无线资源分配算法

为了满足无线数据流量大幅增长的需求,异构云无线接入网(H-CRAN)的资源优化仍然是亟待解决的重要问题。该文在H-CRAN下行链路场景下,提出一种基于深度强化学习(DRL)的无线资源分配算法。首先,该算法以队列稳定为约束,联合优化拥塞控制、用户关联、子载波分配和功率分配,并建立网络总吞吐量最大化的随机优化模型。其次,考虑到调度问题的复杂性,DRL算法利用神经网络作为非线性近似函数,高效地解决维度灾问题。最后,针对无线网络环境的复杂性和动态多变性,引入迁移学习(TL)算法,利用TL的小样本学习特性,使得DRL算法在少量样本的情况下也能获得最优的资源分配策略。此外,TL通过迁移DRL模型的权重参数,进一步地加快了DRL算法的收敛速度。仿真结果表明,该文所提算法可以有效地增加网络吞吐量,提高网络的稳定性。     

无蜂窝大规模MIMO中基于深度强化学习的无人机辅助通信与资源调度

无蜂窝大规模多入多出(MIMO)网络中分布式接入点(AP)同时服务多个用户,可以实现较大区域内虚拟MIMO的大容量传输;而无人机辅助通信能够为该目标区域热点或边缘用户提供覆盖增强.为了降低反馈链路负载,并有效提升无人机辅助通信的频谱利用率,该文研究了基于AP功率分配、无人机服务区选择和接入用户选择的联合调度;首先将AP...     

基于能量效率的分布式星群下行链路功率分配方法研究

针对多波束卫星通信系统星上资源稀缺和能量利用效率不高的问题,本文提出了分布式星群网络下行链路中兼顾系统功耗和数据速率的功率分配方法,通过合理的资源分配来优化系统的能量效率.首先建立分布式星群功率分配模型,将复杂的分式问题转化为易于求解的减法形式问题,然后基于凸优化理论,提出功耗-数据速率权衡功率分配算法,并讨论了能量效率（energy efficiency）与频谱效率（spectral efficiency）之间的权衡关系.仿真结果验证了提出算法的有效性和EE-SE权衡关系,并分析了电路功耗对系统性能的影响.     

基于深度强化学习的反向散射网络资源分配机制

为了提升反向散射网络中物联网设备的平均吞吐量,提出了一种资源分配机制,构建了用户配对和时隙分配联合优化资源分配模型。由于该模型直接利用深度强化学习(Deep Reinforcement Learning,DRL )算法求解导致动作空间维度较高且神经网络复杂,故将其分解为两层子问题以降低动作空间维度：首先,基于深度强化学习算法,利用历史信道信息推断当前的信道信息以进行最优的用户配对;然后,在用户固定配对的情况下,基于凸优化算法,以最大化物联网设备总吞吐量为目标进行最优的时隙分配。仿真结果表明,与其他资源分配方法相比,所提资源分配方法能有效提升系统吞吐量,且有较好的信道适应性和收敛性。     

异构云无线接入网下基于功率域NOMA的能效优化算法

针对异构云无线接入网络的频谱效率和能效问题,该文提出一种基于功率域-非正交多址接入(PD-NOMA)的能效优化算法。首先,该算法以队列稳定和前传链路容量为约束,联合优化用户关联、功率分配和资源块分配,并建立网络能效和用户公平的联合优化模型;其次,由于系统的状态空间和动作空间都是高维且具有连续性,研究问题为连续域的NP-hard问题,进而引入置信域策略优化(TRPO)算法,高效地解决连续域问题;最后,针对TRPO算法的标准解法产生的计算量较为庞大,采用近端策略优化(PPO)算法进行优化求解,PPO算法既保证了TRPO算法的可靠性,又有效地降低TRPO的计算复杂度。仿真结果表明,该文所提算法在保证用户公平性约束下,进一步提高了网络能效性能。     

一种基于接入与回传一体化小基站的用户满意度最大化算法

针对接入与回传一体化小基站场景下用户个性化视频流业务需求问题,该文提出一种基于接入与回传一体化小基站的用户满意度最大化算法。该算法首先根据系统实际可达速率和用户满意度需求速率间的不匹配程度,动态调整下一周期队列传输所需频谱资源,并建立用户质量满意度最大化模型,其次运用Lyapunov优化方法把初始问题转化为Lyapunov偏移加罚项的优化,将溢出概率约束转化为关于自变量的不等式,最后基于拉格朗日对偶分解的用户接入带宽分配算法和基于内点法的回传和接入带宽分配算法进行求解。仿真结果表明,该算法提高了系统用户质量满意度,同时保证了系统稳定性。     

Tasking planning based on task splitting and merging in relay satellite network

To handle the low planning efficiency of the tasks with too long or too short service time,a task planning scheme was proposed based on task splitting and merging for relay satellite systems.First,a task splitting and merging was developed to transfer the task requirements of user to task units which could be planned with high efficiency.Secondly,based on the parallel machine scheduling model,the optimization problem of the task unit planning to maximize the number of completed tasks in the network was built.Further,a heuristic polynomial time scheduling algorithm was proposed.Simulation results show that compared to the traditional scheme,the task planning scheme perform better in terms of completed task number,resource utilization and fairness.     

高低轨频率共存卫星通信系统鲁棒波束成形算法研究

针对静止轨道(Geostationary Earth Orbit, GEO)卫星系统和低轨道(Low Earth Orbit, LEO)卫星系统频率共享时存在的干扰问题,本文基于低轨分布式卫星编队提出了一种鲁棒自适应波束成形算法,从空间域功率隔离角度解决了高低轨卫星通信系统上行链路共用频谱时GEO用户对于LEO卫星共线干扰问题。算法中综合考虑卫星通信系统长传播时延导致的阵列导向矢量存在一定误差,在基于线性约束最小方差(LCMV)准则的自适应波束成形器中设计了考虑系统最恶劣误差情况的鲁棒性约束,并采用泰勒级数逼近法求解波束成形器加权矢量。仿真结果表明本文的鲁棒波束成形算法在卫星通信环境下适应度较高,能够有效地缓解高低轨卫星通信系统频率共享带来的同频干扰问题。      

Energy-Spectral-Efficiency Tradeoff in Interference-Limited Wireless Networks

Spectral efficiency (SE) is an important metric in traditional wireless network design. However, as the development of high-data rate services and rapid increase of energy consumption, energy efficiency (EE) has received more and more attention. In this paper, we investigate the EE–SE tradeoff problem in interference-limited wireless networks. Different from previous researches, we try to optimize EE and SE simultaneously. Firstly, the problem is formulated as a multi-objective optimization problem (MOP), with the constraint of transmit power limit. Then, we convert the MOP to a single-objective optimization problem by the weighted linear sum method. We present an algorithm utilizing difference between two convex functions programming (DCP) to handle with SE optimization problem (SD). EE optimization problem can be solved by an algorithm (EFD) consists of fractional programming embedded with DCP. While for EE–SE tradeoff problem, a particle swarm optimization algorithm is proposed (ESTP) to deal with it. Simulation results validate that the proposed algorithm can efficiently balance EE and SE by adjusting the value of weighted coefficient, which could be used to design a flexible energy efficient network in the future.

     

基于DDPG的综合化航电系统多分区任务分配优化方法

综合化航电系统（Integrated Modular Avionics,IMA）通过时空分区机制实现共享资源平台下的多航电功能集成,分区间的任务分配方法的优劣决定着航电系统的整体效能。针对航电任务集合在多分区内的分配调度问题,提出了一种基于深度强化学习的优化方法。构建了航电系统模型与任务模型,以系统资源限制与任务实时性需求为约束,以提高系统资源利用率为优化目标,将任务分配过程描述为序贯决策问题。引入马尔科夫决策模型,建立基于深度确定性策略梯度（Deep Deterministic Policy Gradient,DDPG）法的IMA任务分配模型并提出通用分配架构;引入状态归一化、行为噪声等策略训练技巧,提高DDPG算法的学习性能和训练能力。仿真结果表明,提出的优化算法迭代次数达到500次时开始收敛,分析800次之后多分区内驻留任务方案在能满足约束要求的同时,最低处理效率提升20.55%。相较于传统分配方案和AC(Actor-Critic)算法,提出的DDPG算法在收敛能力、优化性能以及稳定性上均有显著优势。     

Joint energy efficiency and spectral efficiency optimization algorithm for UDN under the restriction of interference threshold and backhaul capacity

Aiming at the scenarios which consider the constraint of backhaul capacity restriction and interference threshold in ultra-dense networks (UDN),an integer linear programming (ILP) and Lagrangian dual decomposition (LDD) based joint optimization algorithm of energy efficiency and spectrum efficiency was proposed.In the proposed algorithms,the user association problem with the constraint of limited backhaul capacity was modelled as an ILP problem and then finished the connection between the user and the base station of microcell by solving this problem with dynamic programming method.Therefor,Lagrangian dual decomposition (LDD) was applied in an iteration algorithm for spectrum resource allocation and power allocation.The simulation results show that compared with traditional schemes,the proposed algorithm can significantly improve the energy efficiency and spectrum efficiency of system and use the microcell’s load capacity more efficiently.     

Resource allocation for sum-rate maximization in NOMA-based generalized spatial modulation

The Multiple-Input Multiple-Output (MIMO) Non-Orthogonal Multiple Access (NOMA) based on Spatial Modulation (SM-MIMO-NOMA) system has been proposed to achieve better spectral efficiency with reduced radio frequency chains comparing to the traditional MIMO-NOMA system. To improve the performance of SM-MIMO-NOMA systems, we extend them to generalized spatial modulation scenarios while maintaining moderate complexity and fairness. In this paper, system spectral efficiency and transmission quality improvements are proposed by investigating a sum-rate maximization resource allocation problem that is subject to the total transmitted power, user grouping, and resource block constraints. To solve this non-convex and difficult problem, a graph-based user grouping strategy is proposed initially to maximize the mutual gains of intragroup users. An auxiliary-variable approach is then adopted to transform the power allocation subproblem into a convex one. Simulation results demonstrate that the proposed algorithm has better performance in terms of bit error rate and sum rates.     

AG‐DPA: Assignment game–based dynamic power allocation in multibeam satellite systems

Dynamic power allocation is the key technology to maintain the link quality and improve the system throughput in multibeam satellite systems. Many numerical optimization algorithms have been proposed to optimize the power allocation schemes among beams. However, current metaheuristic algorithms, most of which are off‐line iterative methods, are not appropriate in nonuniform traffic demands and time‐varying channels due to the high computational complexity. To solve this problem, an assignment game–based dynamic power allocation (AG‐DPA) is proposed to achieve the suboptimality with low complexity in multibeam satellite systems. The key idea of the proposed AG‐DPA is to model the DPA problem into an assignment game model where the competitive equilibrium is achieved. Further, an adaptive price factor is introduced to make a trade‐off between algorithm performance and complexity. Simulation results show the effectiveness of the proposed AG‐DPA algorithm.     

基于无人机中继的星地认知网络波束成形算法

针对基于无人机中继的星地认知网络,提出了两种波束成形（beamforming, BF）算法,通过对各种干扰进行抑制,实现系统间的频谱共享。具体而言,在基于无人机中继的卫星网络作为次级网络、地面网络作为主网络的情况下,以无人机最大发射功率和主用户所受干扰为约束条件,建立次级用户信干噪比最大化准则的优化问题;接下来在已知次级用户统计信道状态信息的条件下,提出一种基于迭代的BF算法对优化问题进行求解;更进一步,为了降低迭代算法的实现复杂度,提出了一种基于迫零的BF算法。最后,计算机仿真验证了所提两种波束成形方案的正确性与有效性。     

Robust power and subcarrier allocation algorithm for cognitive network based on interference efficiency maximization

For the cognitive OFDMA uplink communication system,a robust power and subcarrier allocation algorithm based on maximum interference efficiency was proposed.Firstly,considering primary user interference constraint,secondary user transmit power constraint,subcarrier allocation constraint and secondary user minimum rate constraint,a robust resource optimization model based on outage probability was established.Then,by using Bernstein approximation and Dinkelbach’s method,the original non-convex problem based on outage probability was transformed into an equivalent convex optimization one,and the analytical solution was obtained by Lagrangian dual function method.Meanwhile,the computational complexity and robust sensitivity of the algorithm were analyzed.The simulation results show that the proposed algorithm has better interference efficiency and robustness.     

多用户大规模MIMO系统能效资源分配算法

该文针对多用户大规模多输入多输出(MIMO)移动通信上行系统,提出一种基于能效优化的资源分配算法。所提方法在采用最大比合并(MRC)接收情况下,满足用户数据速率和可容忍的干扰水平约束条件下,以最大化系统能效下界为准则建立优化模型。根据分数规划的性质,把原始的分数最优化问题转换成减式的形式,进而采用凸优化的方法,通过联合调整基站端的发射天线数和用户的发射功率来优化能效函数。仿真结果表明,所提算法与穷举算法在能效上的差距不足9%,并且有较好的系统频谱效率性能,同时算法复杂度得到了显著降低。     

Adaptive resource allocation framework for user satisfaction maximization in multi-service wireless networks

In order to capture and maintain a representative share of the wireless communication market, effective ways to manage the scarce physical resources of cellular networks are fundamental for cellular network operators. In this context, this paper proposes an adaptive Radio Resource Allocation algorithm that targets the user satisfaction maximization in cellular networks with multiple services. The proposed algorithm is mathematically derived from a utility-based cross-layer optimization framework and employs user weights as well as an innovative service weight that is adapted to meet the satisfaction target of the most prioritized service. Furthermore, the proposed algorithm is scalable to several services classes and can be employed in the current and future generations of wireless systems that guarantee orthogonality among the allocable resources. The performance evaluation is conducted in realistic scenarios of the downlink of an Orthogonal Frequency Division Multiple Access based cellular network serving video and Constant Bit Rate flows, where we assume imperfect Channel State Information at the transmitter. Significant gains in the joint system capacity were obtained, demonstrating that the adaptability and service prioritization allow the accomplishment of simultaneously maximizing the user satisfaction for distinct services.