Pseudo-Multispectral Pedestrian Detection with Deep Thermal Feature Guidance  

作　　者：Fuchen Chu Yanwei Pang Xuebin Sun Jiale Cao Zhanjie Song 

机构地区：[1]School of Electrical and Information Engineering Tianjin University,Tianjin 300072,P.R.China [2]Shanghai Artificial Intelligence Laboratory Shanghai 200232,P.R.China [3]School of Mathematics,Tianjin University Tianjin 300072,P.R.China

出　　处：《Guidance, Navigation and Control》2024年第3期92-107,共16页制导、导航与控制(英文)

基　　金：supported by the National Key Research and Development Program of China (Grant No. 2022ZD0160400);the National Natural Science Foundation of China (Grant No. 62106152)

摘　　要：With complementary multi-modal information(i.e. visible and thermal), multispectral pedestrian detection is essential for around-the-clock applications, such as autonomous driving, video surveillance, and vicinagearth security. Despite its broad applications, the requirements for expensive thermal device and multi-sensor alignment limit the utilization in real-world applications. In this paper, we propose a pseudo-multispectral pedestrian detection(called Pseudo MPD) method,which employs the gray image converted from the RGB image to replace the real thermal image,and learns the pseudo-thermal feature through deep thermal feature guidance(TFG). To achieve this goal, we first introduce an image base-detail decomposition(IBD) module to decompose image information into base and detail parts. Afterwards, we design a base-detail hierarchical feature fusion(BHFF) module to deeply exploit the information between these two parts, and employ a TFG module to guide pseudo-thermal base and detail feature learning. As a result, our proposed method does not require the real thermal image during inference. The comprehensive experiments are performed on two public multispectral pedestrian datasets. The experimental results demonstrate the effectiveness of our proposed method.

关 键 词：Multispectral pedestrian detection vicinagearth security thermal feature guidance image decomposition base-detail hierarchical fusion 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]