基于深度确定性策略梯度学习的无线反向散射数据卸载优化

无线驱动通信网络中，无线设备（WD）可以通过无线反向散射和主动射频传输两种方式进行数据卸载。如何合理分配系统中WD的主动传输和反向散射传输工作模式及其对应的工作时间，从而减小传输延迟、提高传输效率就显得尤为必要。在综合考虑卸载数据量大小、信道条件和WD之间公平性情况下，提出一种基于深度确定性策略梯度（DDPG）的数据卸载方法，在连续动作空间内搜索多个WD的最优时间分配。仿真实验结果表明：DDPG可在有限时间步长内实现算法收敛；由于引入了Jain公平指数，多个WD可同时完成数据卸载；与传统的均分算法、贪心算法对比，DDPG算法可将平均传输延迟减小77.4%和24.2%，可有效提高WD的能耗效率，尤其对于卸载数据量较小的WD效果更加显著。

A Deep Deterministic Policy Gradient Optimization Approach for Multi-users Data Offloading in Wireless PoweredCommunication Network

In the wireless powered communication network (WPCN)，the wireless devices can offload data through wireless backscattering and active radio frequency transmission.How to adjust the working mode as well as manage the time allocation of ambient backscattering and active RF transmission properly is a great challenge for reducing the system transmission delay and enhancing the transmission efficiency.A deep deterministic policy gradient(DDPG) algorithm is proposed to search the best time allocation in a continuous domain，in which the data size，the channel conditions and the fairness between wireless devices are considered.The experimental results show that DDPG algorithm achieves the algorithm convergence in finite time step， and all the wireless devices can complete the data offloading at the same time by introducing Jain fairness index.Compared with the traditional Round-Robin and Greedy algorithms，DDPG algorithm can be used to reduce the average transmission delay by 77.7% and 24.2%，respectively，and the energy efficiency is largely improved especially for wireless devices with a small amount of offloading data.