模态自适应权值学习机制下的多光谱行人检测网络  

Multispectral pedestrian detection network under modal adaptive weight learning mechanism

作　　者：陈莹 朱宇 CHEN Ying;ZHU Yu(Key Laboratory of Advanced Process Control for Light Industry,Ministry of Education,Jiangnan University,Wuxi 214122,China)

机构地区：[1]江南大学轻工过程先进控制教育部重点实验室,江苏无锡214122

出　　处：《光学精密工程》2020年第12期2700-2709,共10页Optics and Precision Engineering

基　　金：国家自然科学基金资助项目(No.61573168)。

摘　　要：针对目前基于红外与可见光模态融合的行人检测方法难以自适应外界环境变化的问题,提出基于多模态信息融合权值学习的行人检测网络。首先,区别于目前大多数研究采用的两模态直接堆叠融合方法,权值学习融合网络考虑两种模态在不同环境条件下对行人检测任务的不同贡献比重,通过双流交互学习二者差异,然后根据各模态特征的当前特性自主获得各模态特征的相应权重,进行加权融合得到融合特征,最后基于融合特征生成新的特征金字塔,并改变先验框的尺寸和密集度以丰富行人先验信息,完成行人检测任务。实验结果表明:在Kaist多光谱行人检测数据集上获得26.96%的平均漏检率,相比目前采用直接堆叠的最优方法以及baseline方法分别降低了2.77%和27.84%,因此自适应权值融合红外和可见光两种模态的信息可以有效获得互补的模态信息以自适应外界环境变化,大幅提升行人检测的性能。A pedestrian detection network based on the weight learning of fusing multimodal information was developed to address the issues of the pedestrian detection method based on infrared and visible modal fusion in adapting to changes in the external environment.First,unlike the fusion method used in several recent studies in which two modalities are stacked directly,the weight learning fusion network reflects different contributions of the modalities to the pedestrian detection task under different environmental conditions.The differences between the two modalities were determined through dual-stream interaction learning.Next,based on the current characteristics of each modal feature,the weight learning fusion network assigned the corresponding weights to each modal feature to generate the fusion feature by performing weighted fusion autonomously.Finally,a new feature pyramid based on the fusion feature was generated,and previous information about the pedestrian was improved by changing the size and density of prior boxes to complete the pedestrian detection task.The experimental results indicated that the log-average miss rate of the Kaist multispectral pedestrian detection dataset reached 26.96%,which was 2.77%and 27.84%lower than that of the direct stacking method and baseline method,respectively.The adaptive weight fusion of infrared and visible modal information could effectively be used to obtain complementary modal information to adapt to external environmental changes and significantly improve pedestrian detection performance.

关 键 词：行人检测 多模态信息 权值学习 自适应融合 深度学习 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]