工业物联网中基于信息熵的联邦增量学习算法与优化  

Entropy-based Federated Incremental Learning and Optimization in Industrial Internet of Things

作　　者：杨睿哲 谢欣儒[1,2] 滕颖蕾 李萌[1,2] 孙艳华 张大君[4] YANG Ruizhe;XIE Xinru;TENG Yinglei;LI Meng;SUN Yanhua;ZHANG Dajun(Faculty of Information Technology,Beijing University of Technology,Beijing 100124,China;Beijing Laboratory of Advanced Information Networks,Beijing University of Technology,Beijing 100124,China;Electronic Engineering School,Beijing University of Posts and Telecommunications,Beijing 100083,China;School of Information Technology,Carleton University,Ottawa K1S 5B6,Canada)

机构地区：[1]北京工业大学信息学部,北京100124 [2]北京工业大学先进信息网络北京实验室,北京100124 [3]北京邮电大学电子工程学院,北京100083 [4]卡尔顿大学,渥太华K1S 5B6

出　　处：《电子与信息学报》2024年第8期3146-3154,共9页Journal of Electronics & Information Technology

基　　金：国家自然科学基金(62171062,62371012)。

摘　　要：面对工业生产过程中大规模、多样且随时间增长的数据和机器学习任务,该文提出一种基于信息熵的联邦增量学习(FIL)与优化方法。基于联邦框架,各本地计算节点可利用本地数据进行模型训练,并计算信息平均熵上传至服务器,以此辅助识别类增任务;全局服务器则根据本地反馈的平均熵选择参与当前轮次训练的本地节点,并判决任务是否产生增量后,进行全局模型下发与聚合更新。所提方法结合平均熵和阈值进行不同情况下的节点选择,实现低平均熵下的模型稳定学习和高平均熵下的模型增量式扩展。在此基础上,采用凸优化,在资源有限的情况下自适应地调整聚合频率和资源分配,最终实现模型的有效收敛。仿真结果表明,在不同的情景下,该文所提方法都可以加速模型收敛并提升训练精度。In the face of large-scale,diverse,and time-evolving data,as well as machine learning tasks in industrial production processes,a Federated Incremental Learning(FIL)and optimization method based on information entropy is proposed in this paper.Within the federated framework,local computing nodes utilize local data for model training,and compute the average entropy to be transmitted to the server to assist in identifying class-incremental tasks.The global server then selects local nodes for current round training based on the locally provided average entropy and makes decisions on task incrementality,followed by global model deployment and aggregation updates.The proposed method combines average entropy and thresholds for nodes selection in various situations,achieving stable model learning under low average entropy and incremental model expansion under high average entropy.Additionally,convex optimization is employed to adaptively adjust aggregation frequency and resource allocation in resource-constrained scenarios,ultimately achieving effective model convergence.Simulation results demonstrate that the proposed method accelerates model convergence and enhances training accuracy in different scenarios.

关 键 词：工业物联网 联邦增量学习 信息平均熵 

分 类 号：TN919.8[电子电信—通信与信息系统]