基于强化学习DQN算法的内存控制器设计  

Design of Memory Controller Based on Reinforcement Learning DQN Algorithm

作　　者：陈家扬 王宇 CHEN Jia-yang;WANG Yu(College of Physics and Information Engineering,Fuzhou University)

机构地区：[1]福州大学物理与信息工程学院

出　　处：《中国集成电路》2023年第5期31-35,61,共6页China lntegrated Circuit

摘　　要：为了提高神经网络加速器的访存性能,本文介绍了一种基于强化学习算法DQN的内存控制器(DQNMC)。首先,利用特定的访存序列在强化学习DQN算法中迭代优化,训练出基于BIM(Binary Invertible Matrix)的地址映射策略。其次,将训练好的地址映射策略在Xilinx VC707 FPGA上基于MIG IP硬件实现。实验结果表明,内存控制器DQNMC在10组测试基准中均实现了最高行缓存命中率,系统平均访问延迟降低了23.84%,最大访问延迟降低了33.58%。DQNMC以几乎可以忽略的硬件代价来达到超过主流地址映射方法的性能。In order to improve the memory access performance of neural network accelerator,this article introduces a memory controller based on the reinforcement learning algorithm DQN(DQNMC).Firstly,the address mapping strategies expressed by BIM(Binary Invertible Matrix)are trained by iterative optimization in the reinforcement learning DQN algorithm using the specific access sequences.Secondly,the trained address mapping strategies are implemented on Xilinx VC707 FPGA based on MIG IP.The experimental results show that the memory controller DQNMC achieves the highest row cache hit ratio in the 10 test benchmarks,and the average system access latency was reduced by 23.84%and the maximum access latency was reduced by 33.58%.DQNMC achieves better performance than mainstream address mapping methods at a negligible hardware cost.

关 键 词：强化学习 内存控制器 DRAM FPGA 地址映射 

分 类 号：TN402[电子电信—微电子学与固体电子学]