Optimized Consensus for Blockchain in Internet of Things Networks via Reinforcement Learning  

作　　者：Yifei Zou Zongjing Jin Yanwei Zheng Dongxiao Yu Tian Lan 

机构地区：[1]Institute of Intelligent Computing,School of Computer Science and Technology,Shandong University,Qingdao 266237,China [2]Department of Electrical and Computer Engineering,George Washington University,Washington,DC 20052,USA

出　　处：《Tsinghua Science and Technology》2023年第6期1009-1022,共14页清华大学学报（自然科学版（英文版）

基　　金：This work was partially supported by the National Key Research and Development Program of China(No.2020YFB1005900);the National Natural Science Foundation of China(Nos.62102232,62122042,and 61971269);the Natural Science Foundation of Shandong Province(No.ZR2021QF064).

摘　　要：Most blockchain systems currently adopt resource-consuming protocols to achieve consensus between miners;for example,the Proof-of-Work(PoW)and Practical Byzantine Fault Tolerant(PBFT)schemes,which have a high consumption of computing/communication resources and usually require reliable communications with bounded delay.However,these protocols may be unsuitable for Internet of Things(IoT)networks because the IoT devices are usually lightweight,battery-operated,and deployed in an unreliable wireless environment.Therefore,this paper studies an efficient consensus protocol for blockchain in IoT networks via reinforcement learning.Specifically,the consensus protocol in this work is designed on the basis of the Proof-of-Communication(PoC)scheme directly in a single-hop wireless network with unreliable communications.A distributed MultiAgent Reinforcement Learning(MARL)algorithm is proposed to improve the efficiency and fairness of consensus for miners in the blockchain system.In this algorithm,each agent uses a matrix to depict the efficiency and fairness of the recent consensus and tunes its actions and rewards carefully in an actor-critic framework to seek effective performance.Empirical results from the simulation show that the fairness of consensus in the proposed algorithm is guaranteed,and the efficiency nearly reaches a centralized optimal solution.

关 键 词：consensus in blockchain Proof-of-Communication(PoC) MultiAgent Reinforcement Learning(MARL) Internet of Things(IoT)networks 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程] TP391.41[自动化与计算机技术—控制科学与工程]