面向事件相机的轻量化脉冲识别网络  

Towards event camera signal recognition using a lightweight spiking neural network

作　　者：刘昭辛 吴金建 石光明 赵庆行 Zhaoxin LIU;Jinjian WU;Guangming SHI;Qinghang ZHAO(School of Artificial Intelligence,Xidian University,Xi’an 710071,China;Pazhou Lab,Huangpu,Guangzhou 510555,China)

机构地区：[1]西安电子科技大学人工智能学院,西安710071 [2]琶洲实验室(黄埔),广州510555

出　　处：《中国科学：信息科学》2023年第7期1333-1347,共15页Scientia Sinica(Informationis)

基　　金：国家自然科学基金(批准号:62022063,62293483,61836008,62104182);鹏城实验室重大攻关项目(批准号:PCL2021A12)资助。

摘　　要：事件相机是一种用脉冲表达信息的仿生成像传感器,具有高时域分辨率、高动态范围、低功耗和高速率等优势.由于事件驱动特性,传统人工神经网络(artificial neural networks,ANN)无法直接处理事件相机输出的脉冲信号.而脉冲神经网络(spiking neural network,SNN)作为一种神经形态计算方法,具有高时域分辨率及事件驱动的特性,这与事件相机高度契合.但是,深层脉冲神经网络需要消耗大量存储空间以及神经元计算资源,严重限制了其在边缘计算场景的部署.本文基于特征维度映射原理,提出面向嵌入式系统的轻量化脉冲神经网络,降低存储需求、提高运行效率并提高网络性能.首先,通过分析网络参数量与网络拟合功能间的关系,明确了约束脉冲神经网络能力的参数瓶颈问题.随后,基于低维特征提取–融合策略提出一种通用轻量化特征提取结构SpikeFire,该模块在保证感受野和特征维度等基本性质不变的前提下大幅减少了网络参数.此外,模拟脑神经元复杂连接特性,模块中采用跳层连接,这既增加多尺度信息提取又有助于深层次网络的优化.最后,将本文所提轻量化网络部署在嵌入式硬件中,开发出了事件驱动的成像识别一体化系统.实验表明,无论是在公开数据集还是自建真实场景和极端成像场景中,所提方法在保证识别性能的前提下大幅减少了参数量并提高运行速度.Event cameras are event-driven bio-inspired sensors,owing to the following advantages:high temporal resolution,high dynamic range,low power,and high imaging speed.Artificial neural networks(ANNs)cannot directly process their output spike signal.Spiking neural networks(SNNs)are neuromorphic computing methods with a high temporal resolution and are event-driven,which fits well with event cameras.However,deep SNNs require large storage space and neuronal computing resources,which limits their deployment in mobile edge computing.Based on the feature dimension mapping,this paper proposes a lightweight SNN for embedded devices to reduce storage requirements,improve efficiency,and enhance network performance.First,by analyzing the relationship between the parameter number and the fitting function of the network,the SNN parameter bottleneck is clear.Subsequently,a general lightweight feature extraction structure,SpikeFire,is proposed based on a low-dimensional feature extraction-fusion strategy,which significantly reduces network parameters on the premise that the basic properties of the receptive field and feature dimension are the same as those of a spiking convolution.Additionally,the module simulates the complex connection of the brain and adopts the skip connection,which not only increases the extraction of multi-scale information but also contributes to deep network optimization.Finally,the lightweight network is deployed in embedded hardware to build an event-driven imaging recognition system.Experiments show that our method can reduce the parameter number and improve the running speed without weakening the performance in public,self-built real scenes,and extreme imaging scene datasets.

关 键 词：脉冲神经网络 事件相机 轻量化网络 特征融合 嵌入式硬件 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术] TP183[自动化与计算机技术—计算机科学与技术]