轻量级深度神经网络模型适配边缘智能研究综述  

Lightweight Deep Neural Network Models for Edge Intelligence:A Survey

作　　者：徐小华 周长兵[1] 胡忠旭[2] 林仕勋[2] 喻振杰 XU Xiaohua;ZHOU Zhangbing;HU Zhongxu;LIN Shixun;YU Zhenjie(School of Information Engineering,China University of Geosciences(Beijing),Beijing 100083,China;Information Technology Education Center,Zhaotong University,Zhaotong,Yunnan 657000,China;School of Computer Information Engineering,Changzhou Institute of Technology,Changzhou,Jiangsu 213000,China)

机构地区：[1]中国地质大学(北京)信息工程学院,北京100083 [2]昭通学院信息技术教育中心,云南昭通657000 [3]常州工学院计算机信息工程学院,江苏常州213000

出　　处：《计算机科学》2024年第7期257-271,共15页Computer Science

基　　金：地球科学文献知识服务与决策支撑二级项目(DD20230139);云南省地方本科高校基础研究联合专项-面上项目(202301BA070001-095)。

摘　　要：随着物联网和人工智能的迅猛发展,边缘计算和人工智能的结合催生了边缘智能这一新的研究领域。边缘智能具备一定的计算能力,能够提供实时、高效和智能的响应。它在智能城市、工业物联网、智能医疗、自动驾驶以及智能家居等领域都具有重要的应用。为了提升模型的准确度,深度神经网络往往采用更深、更大的架构,导致了模型参数的显著增加、存储需求的上升和计算量的增大。受限于物联网边缘设备在计算能力、存储空间和能源资源方面的局限,深度神经网络难以被直接部署到这些设备上。因此,低内存、低计算资源、高准确度且能实时推理的轻量级深度神经网络成为了研究热点。文中首先回顾边缘智能的发展历程,并分析轻量级深度神经网络适应边缘智能的现实需求,提出了两种构建轻量级深度神经网络模型的方法:深度模型压缩技术和轻量化架构设计。接着详细讨论了参数剪枝、参数量化、低秩分解、知识蒸馏以及混合压缩5种主要的深度模型压缩技术,归纳它们各自的性能优势与局限,并评估它们在常用数据集上的压缩效果。之后深入分析轻量化架构设计中的调整卷积核大小、降低输入通道数、分解卷积操作和调整卷积宽度的策略,并比较了几种常用的轻量化网络模型。最后,展望轻量级深度神经网络在边缘智能领域的未来研究方向。With the rapid development of the Internet of Things(IoT)and artificial intelligence(AI),the combination of edge computing and AI has given rise to a new research field called edge intelligence.Edge intelligence possess appropriate computing power and can provide real-time,efficient,and intelligent responses.It has significant applications in areas such as smart cities,industrial IoT,smart healthcare,autonomous driving,and smart homes.In order to improve the accuracy of models,traditional deep neural networks often adopt deeper and larger architectures,resulting in significant increases in model parameters,storage requirements,and computational complexity.However,due to the limitations of IoT terminal devices in terms of computing power,storage space,and energy resources,deep neural networks are difficult to be directly deployed on these devices.Therefore,lightweight deep neural networks with low memory,low computational resources,high accuracy,and real-time inference capability have become a research hotspot.This paper first reviews the development process of edge intelligence and analyzes the practical requirements for lightweight deep neural networks to adapt to intelligent terminals.Two methods for constructing lightweight deep neural network models:model compression techniques and lightweight architecture design are proposed.Next,it discusses in detail five main model compression techniques:parameter pruning,parameter quantization,low-rank decomposition,knowledge distillation,and mixed compression techniques.It summarizes their respective performance advantages and limitations,and eva-luates their compression effects on commonly used datasets.Then,the paper analyzes in depth the strategies of adjusting the size of the convolution kernel,reducing input channel number,decomposing convolution operations,and adjusting convolution width in lightweight architecture design,and compares several commonly used lightweight network models.Finally,the future research direction of lightweight deep neural networks in the f

关 键 词：边缘智能 深度神经网络 轻量级神经网络 模型压缩 轻量化架构设计 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]