基于强化学习的能量采集传感器传输功率控制方法  

RL based transmission power control for energy harvesting sensor

作　　者：刘南君 贾日恒 许文韬 李明禄 LIU Nanjun;JIA Riheng;XU Wentao;LI Minglu(School of Computer Science and Technology,Zhejiang Normal University,Jinhua 321004,China)

机构地区：[1]浙江师范大学计算机科学与技术学院,浙江金华321004

出　　处：《物联网学报》2025年第1期115-127,共13页Chinese Journal on Internet of Things

基　　金：国家自然科学基金资助项目(No.62272417)。

摘　　要：传感器可以通过能量收集技术从周围环境中采集能量,但自然环境中的能源供给通常具有不稳定性。为实现有效的功率控制,使传感器长期运行的同时提升数据吞吐量等性能指标,设计了基于强化学习的功率控制策略。考虑一个端到端通信系统,发送节点采集能量存储到电池中以用于数据传输,同时持续缓存待发送数据。实际应用中,通常无法完整地预知能量和数据到达的过程。该研究中发送节点仅能获取已收集能量、电池电量、已采集数据、数据缓存量、信道增益等当前状态信息,并基于此进行决策。采用了柔性演员-评论家(SAC,soft actor-critic)算法控制传输功率,并设计了合适的奖励函数和动作剪裁方法。仿真实验结果表明,该算法在性能上优于基线策略,并在部分场景中接近理论最优解。Sensors can harvest energy from the surrounding environment,but the energy supply is always unstable.To achieve effective power control of sensors and enhance their performance metrics,such as data throughput,while ensuring long-term life,a reinforcement learning-based power control strategy was designed.Assume an end-to-end communication system,the sender harvests energy,stores it in a battery for data transmission,and continuously buffers data.In practical scenarios,the arrival of energy and data is random and unpredictable.In this study,the current state was only observed via the sender,which included harvested energy,battery level,collected data,data cache level,and channel gain.Decisions were made solely based on these limited observations.The soft actor-critic(SAC)algorithm was used to control transmission power,with an appropriate reward function and action clipping method.Experimental results demonstrate that the proposed algorithm outperformes baseline strategies and approaches the theoretical optimal in certain scenarios.

关 键 词：柔性演员-评论家 无线传感器网络 能量采集 强化学习 功率控制 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]