边缘资源轻量化需求下深度神经网络双角度并行剪枝方法  

Dual-angle parallel pruning method for deep neural networks under requirement for lightweight edge resources

作　　者：张云翔 高圣溥 ZHANG Yunxiang;GAO Shengpu(School of Information Science and Technology,Tsinghua University,Beijing 100062,China;Information Center,Shenzhen Power Supply Bureau Co.,Ltd.,Shenzhen 518000,Guangdong,China)

机构地区：[1]清华大学信息科学技术学院,北京100062 [2]深圳供电局有限公司信息中心,广东深圳518000

出　　处：《沈阳工业大学学报》2025年第2期250-257,共8页Journal of Shenyang University of Technology

基　　金：广东省科技攻关项目(20205899R01);中国南方电网有限责任公司创新项目(090000KK52200083);深圳市科技创新委员会科技攻关项目(重2022163)。

摘　　要：【目的】深度神经网络的应用面临庞大的计算需求和存储开销,这已成为限制其在边缘设备上广泛应用的主要瓶颈。边缘设备因受限于有限的计算资源和存储空间,难以高效运行复杂的深度神经网络模型。因此,在保证模型精度的前提下,如何降低深度神经网络的复杂度和计算量以适应边缘设备对资源轻量化的需求,已成为当前研究的重要方向。提出了一种结合蚁群算法与双角度并行剪枝的深度神经网络优化方法,以提升深度神经网络在边缘设备中的性能。【方法】分析了深度神经网络的结构特点,并构建了包含多个隐藏层的模型。通过蚁群算法模拟蚂蚁觅食过程中的信息素传递机制,在复杂空间中寻找近似最优解,对隐藏层中的相似节点进行聚类,识别并归类高度相似的神经元节点,从而缩减网络规模并降低复杂性。在聚类结果的基础上,提出了对聚类后的冗余节点及游离节点双角度并行剪枝策略:一方面,从权重矩阵的稀疏性出发,裁剪权重较小的节点,以减少计算开销;另一方面,从节点贡献度角度评估每个节点对整体输出的影响,裁剪贡献度较低的节点,从而进一步优化网络结构。【结果】实验结果表明,与未剪枝的原始模型相比,在相同的计算时间内,本文方法剪枝后的深度神经网络在保持较高精度的同时,其数据量高达120 MB、网络复杂度平均值为88.32%、可拓展性为99%。这一结果表明,在有限的资源条件下,该方法能够显著提升深度神经网络的运行效率,更好地满足边缘设备的应用需求。实验结果不仅验证了该方法的有效性,也为深度神经网络在边缘设备上的部署和应用提供了新思路。【结论】提出的优化方法通过在剪枝过程中应用蚁群算法,实现了隐藏层相似节点的精准聚类,为后续的剪枝处理提供了明确目标。同时,双角度并行剪枝策略提升了剪枝的效�[Objective]In the application process of deep neural networks,their huge computing requirements and storage overhead have become bottlenecks that restrict their widespread application on edge devices.Edge devices are limited by deficient computing resources and storage space,which makes it particularly difficult for them to efficiently run complex deep neural network models.Therefore,how to reduce the complexity and computational load of deep neural networks while maintaining model accuracy to meet the requirements of edge devices for lightweight edge resources has become an important research topic at present.To improve the performance of deep neural networks in edge devices,an optimization method for deep neural networks was proposed which combines the ant colony algorithm and dual-angle parallel pruning.[Methods]The structural characteristics of deep neural networks were analyzed,and a deep neural network model with multiple hidden layers was constructed.The ant colony algorithm was utilized to search for approximate optimal solutions in complex spaces by simulating the pheromone transmission mechanism in the process of ants foraging.Similar nodes in the hidden layers of the constructed model were clustered to identify highly similar neuron nodes and group them into the same category,which reduced the scale and complexity of the network.On this basis,dual-angle parallel pruning processing was further carried out on redundant nodes and free nodes after clustering.On the one hand,from the perspective of the sparsity of the weight matrix,nodes with small weights were pruned to reduce computational overhead.On the other hand,from the perspective of node contribution,the contribution of each node to the overall output result was evaluated,and nodes with small contribution were pruned.[Results]The experimental results show that compared to the original model without pruning,the deep neural network pruned using the proposed method has a higher data volume of 120 MB,an average network complexity of 88.32%,and scalabil

关 键 词：边缘资源 轻量化需求 深度神经网络 双角度并行 剪枝方法 蚁群算法 冗余节点 游离节点 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]