基于深度强化学习的水质模型参数率定  

Water quality model parameters calibration based on deep reinforcement learning

作　　者：白冰 董飞[1,2] 彭文启 刘晓波[1,2] BAI Bing;DONG Fei;PENG Wenqi;LIU Xiaobo(State Key Laboratory of Simulation and Regulation of Water Cycle in River Basin,China Institute of Water Resources and Hydropower Research,Beijing 100038;Key Laboratory of Water Safety for Beijing-Tianjin-Hebei Region of Ministry of Water Resources,Beijing 100038)

机构地区：[1]中国水利水电科学研究院流域水循环模拟与调控国家重点实验室,北京100038 [2]水利部京津冀水安全保障重点实验室,北京100038

出　　处：《环境科学学报》2024年第7期271-280,共10页Acta Scientiae Circumstantiae

基　　金：国家重点研发计划(No.2021YFC3200903);中国水利水电科学研究院基本科研项目(No.WE0145B032021)。

摘　　要：为提高水质模型参数率定的精度,避免局部最优及异参同效等现象,提出一种基于深度强化学习的水质模型参数率定方法.通过深度确定性策略梯度算法(DDPG)求解水质模型参数率定这一优化问题,并设置单参数、多参数率定等不同情景对比分析DDPG的参数率定效果及其影响因素.结果表明:(1)DDPG参数率定的相对误差小于1.5%,率定效果良好;(2)根据DDPG参数率定结果能够完成参数的敏感性分析;(3)DDPG参数率定主要影响因素有奖励函数设置、动作选择时噪声的设置、神经网络训练情况等.该方法能够准确地完成水质模型的参数率定,并适用于各类水质模型,为水质模型的参数率定提供了一种可靠的方法.To enhance the precision of parameter calibration in water quality models and address challenges such as local optima and parameter equivalence,a method based on deep reinforcement learning is proposed.The approach specifically employs the Deep Deterministic Policy Gradient(DDPG) algorithm to tackle the optimization problem associated with calibrating parameters in water quality models.Comparative analyses are systematically conducted across scenarios encompassing both single-parameter and multi-parameter calibration to rigorously evaluate the efficacy of DDPG in the context of parameter calibration and its influencing factors.The obtained results reveal several key findings:(1) DDPG successfully achieves parameter calibration with a relative error of less than 1.5%,indicating a commendable level of calibration accuracy.(2) The outcomes derived from DDPG-based parameter calibration facilitate sensitivity analysis of the calibrated parameters,enhancing the interpretability of the model.(3)Crucial influencing factors in DDPG-based parameter calibration include the design of the reward function,configuration of noise during action selection,and the training status of the neural network.This method accurately accomplishes the task of parameter calibration in water quality models and is adaptable across various types of water quality models.It presents a reliable approach for parameter calibration in water quality models.

关 键 词：机器学习 深度强化学习 水质模型 参数率定 

分 类 号：X52[环境科学与工程—环境工程] X32