模仿学习算法的研究与实现  

Research and Implementation of Imitation Learning Algorithm

作　　者：张羽萌 季晓君 ZHANG Yu-meng;JI Xiao-jun(Nanjing University of Science and Technology Zijin College,Nanjing 210023,Jiangsu)

机构地区：[1]南京理工大学紫金学院计算机学院,江苏南京210023

出　　处：《电脑与电信》2023年第5期38-44,共7页Computer & Telecommunication

基　　金：江苏省高等学校大学生创新创业训练计划项目,项目编号:202213654010Y。

摘　　要：为优化强化学习因为奖励函数不明确造成极大误差的弊端,深入研究并实现了模仿学习算法中的行为克隆算法和数据聚合算法。通过活动图对算法流程建模,通过类图对各类之间的关系建模,通过顺序图对核心交互流程建模。根据实验结果,比较行为克隆算法和数据聚合算法的优缺点,发现行为克隆算法可以离线训练避免真实环境的交互,但会使错误累加导致结果误差;数据聚合算法必须在线与环境交互,根据策略选择观测值对应状态,解决误差累积的问题。In order to optimize reinforcement learning for the great errors causing by the unclear reward function,this paper deeply studies and implements the behavior cloning algorithm and data aggregation algorithm in the imitation learning algorithm.The algorithm flow is modeled by activity diagram,the relationship between classes is modeled by class diagram,and the core interaction process is modeled by sequence diagram.According to the experimental results,this paper compares the advantages and disadvantages of the behavior cloning algorithm and the data aggregation algorithm,and discovers that the behavior cloning algorithm offline training can avoid interaction with the real environment,but error accumulation will lead to error results;data aggregation algorithms must interact with the environment online,and select the corresponding state of the observation value according to the strategy to solve the problem of error accumulation.

关 键 词：强化学习 模仿学习 行为克隆算法 数据聚合算法 

分 类 号：TP389.1[自动化与计算机技术—计算机系统结构]