RTS游戏中基于强化学习的行动参数配置优化  

Action Parameter Optimization Based on Reinforcement Learning in Real-Time Strategy Games

作　　者：田佩 臧兆祥[1,2] 张震[1,2] 郭鸿村 无 TIAN Pei;ZANG Zhao-xiang;ZHANG Zhen;GUO Hong-cun;无(Hubei Key Laboratory of Intelligent Vision Based Monitoring for Hydropower Engineering,China Three Gorges University,Yichang Hubei 443002,China;College of Computer and Information Technology,China Three Gorges University,Yichang Hubei 443002,China;School of Big Data and Intelligent Engineering,Chongqing University of International Business and Economics,Chongqing 401520)

机构地区：[1]三峡大学水电工程智能视觉监测湖北省重点实验室,湖北宜昌443002 [2]三峡大学计算机与信息学院,湖北宜昌443002 [3]重庆对外经贸学院大数据与智能工程学院,重庆401520

出　　处：《计算机仿真》2023年第8期355-359,共5页Computer Simulation

基　　金：国家自然科学基金(61502274);三峡大学水电工程智能视觉监测湖北省重点实验室开放基金(2015KLA08)。

摘　　要：在AI游戏领域,实时策略(Real-Time Strategy,RTS)游戏十分复杂,一方面,实时策略游戏中“策略”的制定过程是即时的,另一方面,RTS游戏的状态动作空间庞大、环境仅部分可观测,而且策略制定过程中,行动指令的参数配置对行动结果影响很大。为了在线学习解决RTS游戏中行动指令参数配置优化这一问题,引入了Q-Learning算法训练作战单位选择使其自身行动指令效果最优的参数配置,同时,为提升算法的收敛速度,引入了几种不同的行动选择策略进行对比。为验证所用方法的有效性,实验在游戏仿真平台上对其中一种作战单位的行动指令进行了综合评估。结果表明,对RTS游戏中行动指令参数配置的优化问题,Q-Learning算法的在线学习能力,能非常迅速地学习到作战单位行动指令的最优参数配置。In the realm of AI games,Real-Time Strategy(RTS)games are very complex.On the one hand,the formulation of "Strategy" in real-time strategy games is real-time;On the other hand,RTS games have large state action spaces andpartially observable environments.The parameter configuration of action command has great influence on the action results.In order to optimize action command parameter configuration in RTS game by online learning,a Q-learning algorithm was introduced to choose the optimal parameter configuration according to the action command effect.At the same time,for the purpose of improve the convergence speed of the algorithm,several different action selection strategies were introduced for comparison.To verify the effectiveness of the method,one action command of the combat units was comprehensively evaluated on a gameplatform.The results show that the online learning ability of Q-learning algorithm can quickly obtain the optimal parameter configuration of action command in RTS game.

关 键 词：实时策略游戏 参数优化 强化学习 决策智能 

分 类 号：TP301.6[自动化与计算机技术—计算机系统结构]