基于深度强化学习的含储能船舶的海岛-海上渔排能源运输策略研究  

Research on Energy Transportation Strategies Between Islands and Offshore Fish Farms for Ships With Energy Storage Based on Deep Reinforcement Learning

作　　者：朱振山 陈豪 陈炜龙 黄缨惠 ZHU Zhenshan;CHEN Hao;CHEN Weilong;HUANG Yinghui(College of Electrical Engineering and Automation,Fuzhou University,Fuzhou 350108,Fujian Province,China)

机构地区：[1]福州大学电气工程与自动化学院,福建省福州市350108

出　　处：《中国电机工程学报》2025年第7期2486-2499,I0004,共15页PROCEEDINGS OF THE CHINESE SOCIETY FOR ELECTRICAL ENGINEERING

基　　金：福建省科技创新战略联合研究项目(2023R0153)。

摘　　要：针对海上渔排与风光资源富余岛屿能源交互问题,该文提出含全电力船舶(all-electricship,AES)的岛屿-海上渔排-海岸能源运输策略,利用能够很好处理海面风光不确定性问题以及适应较大规模能源转移模型的深度强化学习方法对上述能源运输模型进行求解。首先,将移动式储能电池组细化为满充电池、空载电池以及不完全充电电池;其次,将上述能源运输问题建模为含混合动作空间的马尔可夫决策过程;考虑到针对混合动作空间问题,提出一种适用于混合动作空间的基于多批次前向传播的参数化双深度Q网络,该方法通过多步前向传递策略对不相关离散与连续动作进行解耦,减少了智能体训练过程中的波动性并能够收敛于更优的解;最后,通过算例仿真可知,所提策略能够有效实现各地点间能量转移,所提算法相较于传统适用于离散动作空间的深度强化学习方法更加灵活,在目标场景下能够实现更优运行。此外,在模型逐渐扩大的情况下,将该文方法与传统方法求解效果进行对比,验证所提方法在解决大规模能源运输问题的优势。This paper addresses the issue of energy interaction between offshore fish farms and islands with surplus wind and solar resources by developing an energy transportation strategy involving fully electric ships for the island-fish farm-coast system.The proposed strategy utilizes deep reinforcement learning,which is well-suited to managing the uncertainties of offshore wind and solar resources and can accommodate large-scale energy transfer models.First,the mobile energy storage battery group is detailed into fully charged,unloaded,and partially charged batteries.Then,the energy transportation problem is modeled as a Markov Decision Process with a hybrid action space.To solve the hybrid action space issue,a parameterized dual deep Q-network based on multi-batch forward propagation is proposed.This method decouples the unrelated discrete and continuous actions using a multi-step forward pass strategy,reducing volatility during the agent's training process and converging to a more optimal solution.Finally,simulation results verify that the proposed strategy effectively facilitates energy transfer between locations.Compared to traditional deep reinforcement learning methods suited for discrete action spaces,the proposed algorithm demonstrates greater flexibility and achieves superior performance in the target scenario.Additionally,comparative analysis in expanding model scales further validates the advantages of the proposed method in addressing large-scale energy transportation challenges.

关 键 词：深度强化学习 全电力船舶 移动式储能电池 混合动作空间 

分 类 号：TM71[电气工程—电力系统及自动化]