基于深度强化学习的无人机小区间干扰抑制技术研究

无人机(UAV)辅助蜂窝网络的空中基站工作在频谱高度拥挤的场景中,会造成严重空中小区间干扰(Inter-Cell Interference, ICI)而大大降低网络性能。为解决该问题,研究了基于深度强化学习的无人机辅助蜂窝网络小区间干扰抑制技术。首先建立了无人机辅助蜂窝网络中基于联合波束成形与功率控制(Joint Beamforming and Power Control, JBPC)的抗干扰优化模型。然后提出了基于置信区间上界(Upper Confidence Bound, UCB) Dueling深度Q网络(Deep Q Network, DQN)的深度强化学习求解算法,并利用该算法设计了联合波束成形和功率控制的ICI抑制技术。仿真结果表明,基于UCB Dueling DQN学习的JBPC干扰抑制技术的收敛性优于DQN和Dueling DQN算法,且能达到穷举法的最优容量,有利于提高无人机辅助蜂窝网络性能。     

基于线性滤波器的四旋翼无人机强化学习控制策略

针对四旋翼无人机(UAVs)系统,该文提出一种基于线性降阶滤波器的深度强化学习(RL)策略,进而设计了一种新型的智能控制方法,有效地提高了旋翼无人机对外界干扰和未建模动态的鲁棒性。首先,基于线性降阶滤波技术,设计了维数更少的滤波器变量作为深度网络的输入,减小了策略的探索空间,提高了策略的探索效率。在此基础上,为了增强策略对稳态误差的感知,该文结合滤波器变量和积分项,设计集总误差作为策略的新输入,提高了旋翼无人机的定位精度。该文的新颖之处在于,首次提出一种基于线性滤波器的深度强化学习策略,有效地消除了未知干扰和未建模动态对四旋翼无人机控制系统的影响,提高了系统的定位精度。对比实验结果表明,该方法能显著地提升旋翼无人机的定位精度和对干扰的鲁棒性。     

基于强化学习的智能超表面辅助无人机通信系统物理层安全算法

该文从物理层安全的角度出发研究了智能超表面(RIS)辅助的无人机(UAV) 3D轨迹优化。具体地说,当RIS辅助的UAV向地面用户进行无线传输时,通过联合优化RIS相移和UAV的3D轨迹来最大化物理层安全速率。然而,由于目标函数是非凸的,传统的优化技术很难直接求解。深度强化学习能够处理无线通信中动态复杂的优化问题,该文基于强化学习双深度Q网络(DDQN)设计一种联合优化RIS相移和无人机3D轨迹算法,最大化可实现的平均安全速率。仿真结果表明,所设计的RIS辅助UAV通信优化算法可以获得比固定飞行高度的连续凸逼近算法(SCA)、随机相移下的RIS算法和没有RIS的算法有更高的安全速率。     

基于深度强化学习的无人机编队控制

针对无人机编队控制中无人机智能化程度不足、缺乏自主学习能力等问题,基于深度强化学习中DDQN算法设计无人机编队控制器,该控制器可同时控制速度与航向通道,使僚机能够自学习跟踪长机并保持编队,提高无人机智能化程度。为验证设计控制器的有效性,通过仿真将设计的DDQN控制器与传统PID控制器进行对比。对比结果表明,该控制器可有效形成无人机编队并满足编队要求,对无人机编队智能化控制进行了有效探索。     

基于强化学习的通信受限环境多无人机协同策略

文章对基于强化学习的通信受限环境中的多无人机协同架构、通信需求和协同策略进行了分析,旨在提高多无人机协同的合理性,解决多无人机在通信受限环境中出现的问题,发挥强化学习的作用。     

软件定义物联网中基于深度强化学习的路由优化算法

随着物联网的飞速发展,高速、海量的数据通信向服务质量保障机制提出了挑战。为了很好地满足用户对高效、低延迟路由的需求,文章结合软件定义网络（SDN）技术,提出一种软件定义物联网中基于深度强化学习的路由优化算法（RDIS）。RDIS利用控制器收集网络信息,基于改进的深度确定性策略梯度算法,在经验回放池中根据重要性对样本采样,最终获得一条性能近乎最优的路径。仿真实验表明,相较于传统的路由算法,RDIS在吞吐量和延迟方面具有更好的网络性能。     

基于强化学习算法的路径规划技术分析

近年来,强化学习路径规划技术在各个领域逐渐凸显出技术优势,在与信息技术、GPS技术等高新技术的结合下,成为林业领域、工业领域甚至是军事领域的重要技术组合。为进一步了解强化学习路径规划技术的应用优势,文章将其与无人机技术结合,分析无人机在复杂工作环境下受到诸多外界因素干扰,在无法抓取准确定位信息时学习路径规划状态。通过具体应用案例研究方式,探究无人机飞行过程中强化学习算法对无人机路径规划的影响。在获取具体应用案例实验数据情况下,了解到改进强化学习算法可以调整无人机路径规划方案,帮助无人机在复杂环境下快速完成任务。     

基于深度强化学习的智能干扰方法研究

随着人工智能技术的发展,智能终端已经可以通过频谱感知、通信效果检测分析等方式智能地改变通信手段,进而实现高效率抗干扰,这给传统干扰带来了巨大挑战。而深度强化学习在复杂场景中的探索效率高,面对高难度任务的能力强大,在军事干扰领域应用前景广阔。基于此,介绍深度强化学习、智能干扰方法这两个基本问题的研究现状和存在的难点问题,总结并提出未来基于深度强化学习的智能干扰方法的研究前景和技术展望。     

基于强化学习的无人机中继网络节点轨迹优化

无人机在远程采集和传输数据时可能遇到频谱不足的问题,此时需要借助地面网络共享频谱,即部分无人机获得地面网络提供的额外频谱作为回报,另一部分无人机为地面网络提供中继服务。针对无线网络频谱共享系统中多个无人机的飞行调度问题展开了研究,提出了一种基于Q-Learning的无人机飞行调度算法。在多个无人机进行数据中继传输时,该算法结合了放大转发和解码转发的特点,采用自适应转发模式,以最大化系统吞吐量。仿真结果表明,所提的调度算法可以对两个网络的无人机数目进行合理的分配,使每个无人机能够找到各自的最优或者次优位置,从而实现较高的系统吞吐量。     

Associative Tasks Computing Offloading Scheme in Internet of Medical Things with Deep Reinforcement Learning

The Internet of Medical Things(Io MT) is regarded as a critical technology for intelligent healthcare in the foreseeable 6G era. Nevertheless, due to the limited computing power capability of edge devices and task-related coupling relationships, Io MT faces unprecedented challenges. Considering the associative connections among tasks, this paper proposes a computing offloading policy for multiple-user devices(UDs) considering device-to-device(D2D) communication and a multi-access edge computing(...     

Blockchain-Enabled Deep Reinforcement Learning Approach for Performance Optimization on the Internet of Things

Internet of Things (IoT) networks are rapidly expanding, which requires adequate and reliable infrastructure and a large amount of data. The IoT device security and technical confidentiality may benefit from using Blockchain, a decentralised and trustworthy ledger. Increasing transaction throughput and coping with big data transfer situations is critical when dealing with significant volumes of IoT data on the Blockchain. Consequently, this research investigates the Deep Reinforcement Learning (DRL) crucial functioning of the blockchain-enabled IoT structure, wherever transactions are instantaneously expanded and public divisibility is confirmed. It is important to note that DRL and Blockchain are two separate advancements devoted to the reliability and usefulness of system operation. These are both transactional systems. Technology integration into information exchange and research solutions is becoming increasingly critical. As a result of Blockchain, information may be exchanged securely and decentralised. When used in tandem with DRL, it can significantly improve communication efficiency. By combining DRL and Blockchain throughout the IoT, the author first presents a decentralised and efficient communication structure that allows for scalable and trustworthy information allocation and better performance than earlier options. The DRL approach assesses whether to offload and which service to dump to improve performance up to 87.5%. Furthermore, this method focuses on constructing an effective offloading mechanism for Blockchain-based communication systems to boost performance.

     

Toward intelligent resource management in dynamic Fog Computing-based Internet of Things environment with Deep Reinforcement Learning: A survey

Fog computing has already started to gain a lot of momentum in the industry for its ability to turn scattered computing resources into a large-scale, virtualized, and elastic computing environment. Resource management (RM) is one of the key challenges in fog computing which is also related to the success of fog computing. Deep learning has been applied to the fog computing field for some time, and it is widely used in large-scale network RM. Reinforcement learning (RL) is a type of machine learning algorithms, and it can be used to learn and make decisions based on reward signals that are obtained from interactions with the environment. We examine current research in this area, comparing RL and deep reinforcement learning (DRL) approaches with traditional algorithmic methods such as graph theory, heuristics, and greedy for managing resources in fog computing environments (published between 2013 and 2022) illustrating how RL and DRL algorithms can be more effective than conventional techniques. Various algorithms based on DRL has been shown to be applicable to RM problem and proved that it has a lot of potential in fog computing. A new microservice model based on the DRL framework is proposed to achieve the goal of efficient fog computing RM. The positive impact of this work is that it can successfully provide a resource manager to efficiently schedule resources and maximize the overall performance.     

异构物联网下资源高效的分层协同联邦学习方法

物联网(IoT)设备资源存在高度异构性,严重影响联邦学习(FL)的训练时间和精度。已有研究未充分考虑物联网设备资源的异构性,且缺乏异构设备间协同训练机制的设计,导致训练效果有限且设备的资源利用率较低。为此,该文提出资源高效的分层协同联邦学习方法(HCFL),设计了端边云分层混合聚合机制,考虑边缘服务器的差异化参数聚合频率,提出自适应异步加权聚合方法,提高模型参数聚合效率。提出资源重均衡的客户端选择算法,考虑模型精度与数据分布特征动态选取客户端,缓解资源异构性对联邦学习性能的影响。设计自组织联邦协同训练算法,充分利用空闲物联网设备资源加速联邦学习训练进程。仿真结果表明,在不同资源异构状态下,与基线方法相比,模型训练时间平均降低15％,模型精度平均提高6％,设备平均资源利用率提高52％。     

物联网中带有隐私保护的鲁棒联邦学习研究

联邦学习允许数据不出本地的情况下实现数据价值的有效流动,被认为是物联网(IoT)场景下兼顾数据共享与隐私保护的有效方法。然而,联邦学习系统易受拜占庭攻击和推理攻击的影响,导致系统的鲁棒性和数据的隐私性受损。物联网设备的数据异构性和资源瓶颈,也为带有隐私保护的鲁棒聚合算法设计带来巨大挑战。该文提出面向异构物联网的带有数据重采样的鲁棒聚合方法Re-Sim,通过测量方向相似性和标准化更新幅度实现模型的鲁棒聚合,并采用数据重采样技术增强数据异构环境下模型的鲁棒性。同时构建轻量安全聚合协议(LSA),在保证数据隐私性的同时兼顾模型鲁棒性、准确性和计算开销,并从理论上对协议的隐私性进行了分析。仿真结果表明,该方案能在数据异构情况下有效抵抗拜占庭攻击和推理攻击,与基线方法相比,该文所提方案精度提高1%～3%,同时减轻客户端侧计算开销79%。     

基于强化学习的无人机电磁干扰感知与抗干扰传输方法

无人机对于无线信道的依赖性和无线传播环境的开放性,导致其通信易受到恶意的电磁干扰。针对其中恶意的信道跟随干扰,在感知干扰信道信息的基础上,将无人机的发射功率和信道选择策略建模为马尔科夫决策过程(Markov Decision Process, MDP),利用强化学习算法对该通信系统的抗干扰方法进行智能优化,提出了基于赢或快学习策略爬山算法(Win or Learn Fast Policy Hill-climbing, WoLF-PHC)的抗干扰算法。仿真结果证明,所提算法能够将用户干信比降低至0.1以下,将用户可达速率在初始值基础上提升14%,与Q学习算法和PHC算法相比具有更好的抗干扰传输性能。     

Deep Learning Based Object Detection Combined with Internet of Things for Remote Surveillance

Wireless Personal Communications - Object detection is the key process in any video surveillance application. In case of remote surveillance, it is a necessity to accurately detect the target and...     

Reinforcement Learning Based Dynamic Spectrum Access in Cognitive Internet of Vehicles

Cognitive Internet of Vehicles (CIoV) can improve spectrum utilization by accessing the spec-trum licensed to primary user (PU) under the premise of not disturb...     

Mobile Edge Computing Servers Deployment with Improved Genetic Algorithm in Cellular Internet of Things

Mobile edge computing(MEC) provides services to devices and reduces latency in cellular internet of things(Io T) networks. However, the challenging problem is how to deploy MEC servers economically and efficiently. This paper investigates the deployment problem of MEC servers of the real-world road network by employing an improved genetic algorithm(GA) scheme. We first use the threshold-based K-means algorithm to form vehicle clusters according to their locations. We then select base stations(BS...     

不确定性环境下基于进化算法的强化学习

不确定性和隐状态是目前强化学习所要面对的重要难题.本文提出了一种新的算法MA-Q-learning算法来求解带有这种不确定性的POMDP问题近似最优策略.利用Memetic算法来进化策略,而Q学习算法得到预测奖励来指出进化策略的适应度值.针对隐状态问题,通过记忆agent最近经历的确定性的有限步历史信息,与表示所有可能状态上的概率分布的信度状态相结合,共同决策当前的最优策略.利用一种混合搜索方法来提高搜索效率,其中调整因子被用于保持种群的多样性,并且指导组合式交叉操作与变异操作.在POMDP的Benchmark实例上的实验结果证明本文提出的算法性能优于其他的POMDP近似算法.