基于双分支特征融合的跨模态行人检测算法  

Cross-modal pedestrian detection algorithm based on dual-branch feature fusion

作　　者：陈广秋[1] 张桐森 段锦[1] 黄丹丹 CHEN Guangqiu;ZHANG Tongsen;DUAN Jin;HUANG Dandan(School of Electronic and Information Engineering,Changchun University of Science and Technology,Changchun 130022,China)

机构地区：[1]长春理工大学电子信息工程学院,吉林长春130022

出　　处：《华中科技大学学报(自然科学版)》2025年第3期14-22,共9页Journal of Huazhong University of Science and Technology(Natural Science Edition)

基　　金：国家自然科学基金资助项目(62127813)。

摘　　要：针对目前普遍采用的可见光单光谱行人检测容易受到环境和光照的影响,当在夜晚和复杂环境下,以及在检测被遮挡和小尺度目标时,都会出现检测结果偏差大或漏检的现象,提出了一种基于双分支特征融合的跨模态行人检测算法.在YOLOv7框架内将主干网络改进为双主干结构,分别提取可见光图像和红外图像特征信息,并设计差分交叉融合模块(DCF)融合两种模态特征,对融合特征进行训练学习,利用检测网络实现各类行人检测.为了提升检测精度,在双主干网络中添加了注意力机制SeNet模块;为了提升检测效率,设计了轻量化ELAN-G和ELAN-WT模块替换原网络中的对应模块.实验结果表明:本文算法检测精度优秀,同时满足实时性需求.The commonly used visible light single spectrum pedestrian detection is easily affected by the environment and lighting conditions.In night and complex environments,as well as when detecting occluded and small-scale targets,there may be large deviations or missed detections in the detection results.To address these issues,a cross modal pedestrian detection algorithm based on dual branch feature fusion was proposed.Within the YOLOv7 framework,the backbone network was enhanced into a dualbackbone structure to separately extract feature information from both visible light and infrared images.A differential cross fusion(DCF) module was designed to integrate these two modalities' features,followed by training the fused features for comprehensive pedestrian detection via the detection network.To enhance detection accuracy,an attention mechanism,SeNet module,was incorporated into the dual-backbone network.Furthermore,to boost efficiency,lightweight ELAN-G and ELAN-WT modules were designed to replace corresponding components in the original network architecture.Experimental results demonstrate that the detection accuracy of the proposed algorithm is consistently higher than that of existing cross-modal detection algorithms,while also satisfying real-time requirements.

关 键 词：多模态深度学习 特征融合 行人检测 跨模态 YOLOv7框架 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]