针对超临界翼型气动修型策略的强化学习  

Reinforcement learning method for supercritical airfoil aerodynamic design

作　　者：李润泽[1] 张宇飞[1] 陈海昕[1] LI Runze;ZHANG Yufei;CHEN Haixin(School of Aerospace Engineering,Tsinghua University,Beijing 100084,China)

机构地区：[1]清华大学航天航空学院,北京100084

出　　处：《航空学报》2021年第4期269-282,共14页Acta Aeronautica et Astronautica Sinica

基　　金：国家自然科学基金(11872230,91852108);清华自主创新科研基金(2015Z22003)。

摘　　要：强化学习是一类用于学习策略的机器学习方法,通过模拟人的学习过程,与所处环境不断交互来学习动作策略,用以获得最大累积回报。以设计师在翼型气动设计中的增量修型过程为例,给出强化学习在气动优化设计中的要素定义和具体算法的实现。研究了预训练中选择不同示例对预训练和强化学习结果的影响,并将强化学习得到的策略模型在其他环境中进行了迁移测试验证。结果表明,合理的预训练能够有效提高强化学习的效率和最终策略的鲁棒性,且所形成的策略模型具有较好的迁移能力。Reinforcement learning as a machine learning method for learning policies learns in a way similar to human learning process,interacting with the environment and learning how to achieve more rewards.The elements and algorithms of reinforcement learning are defined and adjusted in this paper for the supercritical airfoil aerodynamic design process.The results of imitation learning are then studied,and the policies from the imitation learning are adopted in reinforcement learning.The influence of different pretraining processes is studied,and the final policies tested in other similar environments.The results show that pretraining can improve reinforcement learning efficiency and policy robustness.The final policies obtained in this study can also have satisfactory performance in other similar environments.

关 键 词：强化学习 增量修型 近端策略优化(PPO) 预训练 模仿学习 迁移能力 

分 类 号：V224[航空宇航科学与技术—飞行器设计]