基于强化学习的多核芯片动态功耗管理框架  

Multi-core Chip Dynamic Power Management Framework Based on Reinforcement Learning

作　　者：卓成[2,5] 曾旭东 陈宇飞 孙凇昱 罗国杰 贺青 尹勋钊 ZHUO Cheng;ZENG Xudong;CHEN Yufei;SUN Songyu;LUO Guojie;HE Qing;YIN Xunzhao(Polytechnic Institute,Zhejiang University,Hangzhou 310015,China;College of Information Science and Electronics Engineering,Zhejiang University,Hangzhou 310027,China;School of Electronics Engineering and Computer Science,Peking University,Beijing 100871,China;Hangzhou Xingxin Technology Co.,Ltd.,Hangzhou 310052,China;Key Laboratory of Collaborative Sensing and Autonomous Unmanned Systems of Zhejiang Province,Hangzhou 310015,China)

机构地区：[1]浙江大学工程师学院,杭州310015 [2]浙江大学信息与电子工程学院,杭州310027 [3]北京大学信息科学技术学院,北京100871 [4]杭州行芯科技有限公司,杭州310052 [5]浙江省协同感知与自主无人系统重点实验室,杭州310015

出　　处：《电子与信息学报》2023年第1期24-32,共9页Journal of Electronics & Information Technology

基　　金：浙江省重点研发计划(2020C01052);国家自然科学基金(61974133,62034007,62141404)。

摘　　要：多核芯片可以为移动智能终端提供强大算力,但功耗和温度问题始终制约着其性能表现。针对这个问题,该文提出了一种基于强化学习的多核芯片动态功耗管理框架。首先,建立了一个基于GEM5的多核芯片动态电压频率调节仿真系统。然后,采用了一种考虑CMOS芯片物理特性的功耗模型构建方法以实现在线实时功耗监测。最后,设计了一种面向多核芯片的梯度式奖励方法,并使用深度Q神经网络(Deep Q Network, DQN)算法对多核芯片的功耗管理策略进行学习。仿真结果表明,相比于常规的Ondemand,MaxBIPS方案,该文所提出的框架分别实现了2.12%, 4.03%的多核芯片计算性能提升。Multi-core chips can provide mighty computing capability for mobile intelligent terminals, but their performance is constraint by thermal and power issues. For this problem, this paper proposes a multi-core chip dynamic power management framework based on reinforcement learning. First, based on GEM5, a dynamic voltage and frequency scaling simulation system of the multi-core chips is established. Second, a chip power model characterization method is adopted, which takes CMOS physical characteristics into consideration to realize online real-time power monitoring. Finally, a gradient reward method for the multi-core chips is designed, and a Deep Q Network(DQN) algorithm is used to learn the power management strategy for the multi-core chips. Compared with conventional Ondemand and MaxBIPS schemes, the simulation results show that the proposed framework achieves 2.12% and 4.03% improvement in computational performance of the multi-core chips respectively.

关 键 词：多核处理器芯片 动态功耗管理 强化学习 

分 类 号：TN402[电子电信—微电子学与固体电子学] TP315[自动化与计算机技术—计算机软件与理论]