基于深度强化学习的综合电子系统重构方法  

Deep Reinforcement Learning-Based Reconfiguration Method for Integrated Electronic Systems

作　　者：马驰 张国群 孙俊格 吕广喆[3] 张涛[1] MA Chi;ZHANG Guoqun;SUN Junge;LYU Guangzhe;ZHANG Tao(School of Computer Science and Engineering,Northwestern Polytechnical University,Xi’an 710072,Shaanxi,China;Shanghai Electro-Mechanical Engineering Institute,Shanghai 201109,China;Aeronautics Computing Technology Research Institute,Xi’an 710119,Shaanxi,China)

机构地区：[1]西北工业大学软件学院,陕西西安710072 [2]上海机电工程研究所,上海201109 [3]西安航空计算技术研究所,陕西西安710119

出　　处：《空天防御》2024年第1期63-70,共8页Air & Space Defense

基　　金：航空科学基金项目(20185853038,201907053004);上海航天科技创新基金项目(SAST2021-054)。

摘　　要：重构作为综合模块化航空电子系统用来提高容错性和稳定性的常用手段,是指发生某一故障后,系统根据事先预设好的重构蓝图,通过一系列应用迁移动作从故障状态转为正常状态的过程。随着综合电子系统的功能多元化和结构复杂化,提高系统的容错性和稳定性显得至关重要。然而现有的人工重构和传统重构算法这两种重构配置蓝图设计方式难以保证综合电子系统的容错性和稳定性。本文针对综合电子系统故障情况,结合深度强化学习算法,对重构蓝图的重构模型进行探索并提出基于优先经验回放的竞争深度Q网络算法(PEP_DDQN),通过优先经验回放机制和SUMTREE批量样本抽取技术提出基于深度强化学习的优先经验回放和竞争深度Q网络重构算法。实验表明,相较于传统强化学习Q-Learning算法和DQN算法实现的重构蓝图生成算法,所提出的PEP_DDQN算法能生成更高质量的蓝图并具有更高的收敛性能与更快的求解速度。Reconfiguration is widely used by integrated electronic systems to enhance its fault tolerance and stability.It involves transforming a system from a faulty state to a normal state using a series migration actions based on a predefined reconfiguration blueprint after fault occurred.Considering the existing functional diversification and structural complexity of integrated electronic systems,it is crucial to enhance the fault tolerance and stability of the system.However,the current manual reconfiguration and conventional reconfiguration algorithms,two methods for designing reconfiguration configuration blueprints,are challenging to the fault tolerance and stability requirements of integrated electronic systems.This study has integrated the deep reinforcement learning algorithm to determine the reconfiguration blueprint model for the integrated electronic system fault situation and has proposed the Prioritized Experience Playback-based Competitive Deep Q-Network algorithm(PEP_DDQN).Utilizing the prioritized experience playback mechanism and SUMTREE's batch sample extraction technique,the proposed algorithm has built a competitive deep Q-network reconstruction algorithm based on deep reinforcement learning.Experiment results demonstrated that the PEP_DDQN method can outperform traditional reinforcement learning Q-Learning and DQN algorithms in generating higher-quality blueprints.It also exhibits better convergence performance and solution speed.

关 键 词：综合模块化航空电子系统 智能重构 深度强化学习 DQN算法 

分 类 号：V243[航空宇航科学与技术—飞行器设计]