基于深度强化学习的机翼分离流场零质量射流控制  

Zero mass jet control in wing separation flow fields based on deep reinforcement learning

作　　者：何贤军 华越 王依哲 彭江舟 陈志华[1] 吴威涛 He Xianjun;Hua Yue;Wang Yizhe;Peng Jiangzhou;Chen Zhihua;Wu Weitao(National Key Laboratory of Transient Physics,Nanjing University of Science and Technology,Nanjing 210094,China;Sino-French Engineer School,Nanjing University of Science and Technology,Nanjing 210094,China;School of Mechanical Engineering,Nanjing University of Science and Technology,Nanjing 210094,China)

机构地区：[1]南京理工大学瞬态物理国家重点实验室,江苏南京210094 [2]南京理工大学中法工程师学院,江苏南京210094 [3]南京理工大学机械工程学院,江苏南京210094

出　　处：《南京理工大学学报》2022年第6期697-708,共12页Journal of Nanjing University of Science and Technology

基　　金：中央高校基本科研经费(30919011401);江苏省自然科学基金(BK20201302)。

摘　　要：为了降低机翼在飞行中受到的阻力,该文针对NACA0012翼型构建了基于双喷孔零质量合成射流的流动控制框架,利用深度强化学习(DRL)的近端策略优化(PPO)算法获取了一种具有环境实时适应性的主动流动控制策略。研究了不同来流条件下DRL流动控制策略对机翼边界层以及尾部流动分离情况的影响。探索了进一步考虑射流节能作为奖励函数的流动控制策略的学习与最终控制效果。针对DRL模型超参数开展了研究,对比分析了不同网络更新频率(5、10、20)和不同学习率(10^(-3)、10^(-4)、10^(-5))下流动控制模型的训练效率和控制策略效果。结果显示,通过DRL获得的流动控制策略显著地减小了机翼上表面边界层的厚度,实现了35%的减阻和33.7%的升阻比提升。DRL在复杂控制目标下依然能学习到有效的减阻增升策略,且射流能量节省近50%。较小的网络更新频率在训练初期能快速地提升训练效果,但在中后期存在奖励值不升反降,网络过拟合的问题。较大的网络更新频率和较小的学习率则存在训练效果提升缓慢,训练周期过长,学习效率低的问题。In order to reduce the resistance of the wing in flight,a flow control framework based on zero mass synthetic jet with dual-nozzle is built for NACA0012 airfoil,and an active flow control strategy with real-time adaptability to the environment is obtained by using the proximal policy optimization(PPO)algorithm of deep reinforcement learning(DRL).The effect of the DRL flow control strategy on the boundary layer of the wing and the tail flow separation is comprehensively investigated under different incoming flow conditions.The study and final control effect of the flow control strategy are explored further considering the jet energy saving as a reward function.The research is carried out on the super parameters of DRL model,and the training efficiency and control strategy effect of flow control model are compared and analyzed under different network update frequencies(5,10,20)and different learning rates(10^(-3),10^(-4),10^(-5)).The results show that the flow control strategy obtained through DRL significantly reduces the thickness of the boundary layer on the upper surface of the wing,achieving 35% drag reduction and 33.7% lift drag ratio increase.The DRL can still learn effective drag reduction and lift increase strategies under complex control targets,and the jet energy is saved by nearly 50%.The smaller network update frequency can quickly improve the training effect at the initial stage of training,but in the middle and later stages,there is a problem that the reward value does not rise but fall,and the network is over fitted.The larger the network update frequency and the smaller the learning rate,the slower the training effect,the longer the training period and the lower the learning efficiency.

关 键 词：深度强化学习 机翼分离流场 主动流动控制 射流控制 近端策略优化 机翼边界层 尾部 网络更新频率 

分 类 号：V211.41[航空宇航科学与技术—航空宇航推进理论与工程]