深度强化学习及其在工业场景的应用与展望  

Deep reinforcement learning applications and prospects in industrial scenarios

作　　者：谭靖 杨利刚 李潇睿 袁兆麟 崔允端 姚超 王宗杰[1] 班晓娟[2,3,5,7] TAN JING;YANG Ligang;LI Xiaorui;YUAN Zhaolin;CUI Yunduan;YAO Chao;WANG Zongjie;BAN Xiaojuan(School of Computer and Communication Engineering,University of Science and Technology Beijing,Beijing 100083,China;Beijing Advanced Innovation Center for Materials Genome Engineering,University of Science and Technology Beijing,Beijing 100083,China;Beijing Key Laboratory of Knowledge Engineering for Materials Science,University of Science and Technology Beijing,Beijing 100083,China;School of Intelligence Science and Technology,University of Science and Technology Beijing,Beijing 100083,China;School of Civil and Resource Engineering,University of Science and Technology Beijing,Beijing 100083,China;Shenzhen Institute of Advanced Technology,Chinese Academy of Sciences,Shenzhen 518055,China;Institute of Materials Intelligent Technology,Liaoning Academy of Materials,Shenyang 110004,China)

机构地区：[1]北京科技大学计算机与通信工程学院,北京100083 [2]北京科技大学北京材料基因工程高精尖创新中心,北京100083 [3]北京科技大学材料领域知识工程北京市重点实验室,北京100083 [4]北京科技大学土木与资源工程学院,北京100083 [5]北京科技大学智能科学与技术学院,北京100083 [6]中国科学院深圳先进技术研究院,深圳518055 [7]辽宁材料实验室材料智能技术研究所,沈阳110004

出　　处：《工程科学学报》2025年第4期768-779,共12页Chinese Journal of Engineering

基　　金：国家重点研发计划资助项目(2022YFE0129200)。

摘　　要：工业控制系统(Industrial control systems,ICS)在现代工业生产中发挥关键作用,负责监控和控制工业过程,确保高效、安全和稳定的生产.随着工业4.0和智能制造的发展,传统工业控制方法难以应对日益复杂且动态变化的生产环境.深度强化学习(Deep reinforcement learning,DRL)结合了深度学习与强化学习的优势,在工业智能控制领域展现出巨大潜力.本文综述了DRL在工业智能控制中的应用现状和研究进展.首先介绍了DRL的基本原理及相关算法,并简述工业控制的背景,分析智能控制的应用需求与现存挑战.随后,详细综述了DRL在工业领域的应用,并对当前研究进行了总结,最后对未来研究方向提出了展望.Industrial production is fundamental to human society.Industrial control systems(ICS)serve as the cornerstone of modern industrial processes and are responsible for monitoring and controlling operations to ensure efficiency,safety,and stability.Central to these systems are control algorithms,which enable the automation of operations,optimization of process parameters,and reduction of operational costs.However,with the rapid advancements in Industry 4.0 and smart manufacturing,traditional control methods are increasingly inadequate to address the growing complexity,high dynamics,and real-time demands of modern industrial environments.Deep reinforcement learning(DRL),which integrates the high-dimensional feature extraction of deep learning with the adaptive decision-making capabilities of reinforcement learning,has emerged as a transformative technology in intelligent industrial control.This paper provides a comprehensive review of DRL’s principles,methodologies,and applications in industrial scenarios.The review begins with an introduction to the fundamental concepts of DRL,including the Markov decision process(MDP)framework and the Bellman equation for optimizing decision-making strategies,followed by an exploration of the latest advancements in both online and offline reinforcement learning algorithms.The paper systematically examines the background and challenges of industrial control systems,highlighting the limitations of traditional methods such as proportional-integral-derivative(PID)control and rule-based systems when faced with multi-variable,nonlinear,and dynamic processes.By analyzing the evolving demands of intelligent control,the review underscores the necessity for advanced,self-learning approaches,such as DRL,that are capable of operating effectively in environments with incomplete information,real-time constraints,and multiple conflicting objectives.As a key contribution,a novel classification framework for DRL applications in industrial scenarios is proposed.Current research is categorized into

关 键 词：深度强化学习 在线强化学习 离线强化学习 工业控制系统 智能控制 

分 类 号：TP391.9[自动化与计算机技术—计算机应用技术]