Multi-Agent Deep Q-Networks for Efficient Edge Federated Learning Communications in Software-Defined IoT  

作　　者：Prohim Tam Sa Math Ahyoung Lee Seokhoon Kim 

机构地区：[1]Department of Software Convergence,Soonchunhyang University,Asan,31538,Korea [2]Department of Computer Science,Kennesaw State University,Marietta,GA 30060,USA [3]Department of Computer Software Engineering,Soonchunhyang University,Asan,31538,Korea

出　　处：《Computers, Materials & Continua》2022年第5期3319-3335,共17页计算机、材料和连续体（英文）

基　　金：This work was funded by BK21 FOUR(Fostering Outstanding Universities for Research)(No.5199990914048);this research was supported by Basic Science Research Program through the National Research Foundation of Korea(NRF)funded by the Ministry of Education(NRF-2020R1I1A3066543);In addition,this work was supported by the Soonchunhyang University Research Fund.

摘　　要：Federated learning(FL)activates distributed on-device computation techniques to model a better algorithm performance with the interaction of local model updates and global model distributions in aggregation averaging processes.However,in large-scale heterogeneous Internet of Things(IoT)cellular networks,massive multi-dimensional model update iterations and resource-constrained computation are challenging aspects to be tackled significantly.This paper introduces the system model of converging softwaredefined networking(SDN)and network functions virtualization(NFV)to enable device/resource abstractions and provide NFV-enabled edge FL(eFL)aggregation servers for advancing automation and controllability.Multi-agent deep Q-networks(MADQNs)target to enforce a self-learning softwarization,optimize resource allocation policies,and advocate computation offloading decisions.With gathered network conditions and resource states,the proposed agent aims to explore various actions for estimating expected longterm rewards in a particular state observation.In exploration phase,optimal actions for joint resource allocation and offloading decisions in different possible states are obtained by maximum Q-value selections.Action-based virtual network functions(VNF)forwarding graph(VNFFG)is orchestrated to map VNFs towards eFL aggregation server with sufficient communication and computation resources in NFV infrastructure(NFVI).The proposed scheme indicates deficient allocation actions,modifies the VNF backup instances,and reallocates the virtual resource for exploitation phase.Deep neural network(DNN)is used as a value function approximator,and epsilongreedy algorithm balances exploration and exploitation.The scheme primarily considers the criticalities of FL model services and congestion states to optimize long-term policy.Simulation results presented the outperformance of the proposed scheme over reference schemes in terms of Quality of Service(QoS)performance metrics,including packet drop ratio,packet drop counts,packet delivery ra

关 键 词：Deep Q-networks federated learning network functions virtualization quality of service software-defined networking 

分 类 号：TP311.5[自动化与计算机技术—计算机软件与理论]