强化学习方法在翼型拍动实验中的应用  

Application of reinforcement learning method in flapping airfoil experiment

作　　者：张进[1] 周雷[1] 曹博超[1] ZHANG Jin;ZHOU Lei;CAO Bochao(Department of Aeronautics and Astronautics,Fudan University,Shanghai 200433,China)

机构地区：[1]复旦大学航天航空系,上海200433

出　　处：《空气动力学学报》2023年第9期20-29,共10页Acta Aerodynamica Sinica

摘　　要：将深度强化学习方法应用于水洞实验,实现了实验室内的自动闭环优化框架,并用该框架优化了雷诺数R_(e)=1.3×10^(4)下纯俯仰运动的NACA0012翼型模型的推进效率。现有的相关研究往往将运动模式限制为某种周期性函数,具有局限性。借助于强化学习方法,实现了在更广的非周期动作空间中的动作搜索。在实验中,模型自动地与水洞环境进行交互,最终学习到了高效推进的非周期运动策略。另外,通过修改奖励函数,实现了在给定推力阈值以上的效率优化。研究结果显示,强化学习模型可以在实验过程中通过不断调整拍动动作的幅度和频率来实现推进效率的持续提升,并且最终通过强化学习方法获得的最优拍动动作均与正弦拍动动作接近,得到的最优推进效率基本位于同等幅度正弦动作效率的上边界。研究展示了强化学习方法用于复杂流动控制问题的可行性。This study presents the application of the deep reinforcement learning(DRL)method in water tunnel experiments,which establishes an automatic closed-loop optimization framework in the laboratory.The framework is used to optimize the propulsion efficiency of a NACA0012 airfoil model under pure pitching motion at a Reynolds number of R_(e)=1.3×10^(4).Existing related studies often limit the motion patterns to periodic functions,thanks to the DRL method,the optimization process can take place in a broader non-periodic action space.In the experiment,the airfoil automatically interacts with the water tunnel environment,and ultimately learns highly efficient non-periodic motions.By modifying the reward function,the efficiency optimization can be achieved above a given thrust threshold.The present findings demonstrate the DRL model can continuously enhance the propulsion efficiency by adjusting the amplitude and frequency of the flapping motion.Moreover,the optimal flapping motion obtained by DRL is close to the sinusoidal motion,and the achieved optimal propulsion efficiency lies on the upper bound of that of the sinusoidal motion with a similar amplitude.The present study demonstrates the feasibility of using the DRL method for complex flow control problems.

关 键 词：拍动翼型 实验流体力学 强化学习 机器学习 效率优化 

分 类 号：O352[理学—流体力学] TP181[理学—力学]