基于深度学习的多模态生物特征融合模型  

Multimodal Biometric Fusion Model Based on Deep Learning

作　　者：李卓容 唐云祁[1] LI Zhuorong;TANG Yunqi(School of Criminal Investigation,People’s Public Security University of China,Beijing 100038,China)

机构地区：[1]中国人民公安大学侦查学院,北京100038

出　　处：《计算机工程与应用》2023年第7期180-189,共10页Computer Engineering and Applications

基　　金：中央高校基本科研业务费(2021JKF203);上海市现场物证重点实验室开放课题基金(2021XCWZK04)。

摘　　要：面对公安实战中获取的低质量生物特征数据,单模态生物特征识别技术的精度并不理想,现有的多模态融合算法存在融合层次单一、泛化性不强等问题,深度神经网络的发展为其提供了有效的解决途径。构建基于深度神经网络的多模态生物特征融合模型,将像素层、特征层、分数层等不同层次的融合方法统一到融合模型中,在像素层采用空间、通道和强度融合三种策略;在特征层通过反向传播整体优化模态专用分支与联合表示层,构建模态之间一阶依赖关系;在分数层使用基于Rank1评价和基于模态评价两种方法完成匹配分数融合。模拟实战数据构建虚拟同源多模态数据集进行模型验证。实验结果表明,多模态像素层融合方法提升效果有限,难以增强数据的区分度;多模态特征层融合方法相比单模态算法提升2.2个百分点;分数层融合方法相比单模态算法提升3.5个百分点,最佳检索精度可达99.6%。基于深度学习方法提出的多模态生物特征融合模型极大地提高了模型的泛化性和检索精度。This paper constructs a multimodal biometric fusion model based on deep neural networks and combines several fusion strategies at different levels,such as pixel,feature,and score.The pixel level employs the three fusion strategies of spatial,channel,and intensity.The feature level builds first-order dependencies between modes by back-propagating the overall optimized modal specific branch and joint representation,and the score level completes the matching score fusion using both Rank1-based evaluation and modal-based evaluation methods.Besides,virtual homogeneous multimodal datasets are constructed by simulating real-world data.The experimental results show that the pixel level fusion method has limited improvement and it is difficult to improve data differentiation.The feature level fusion method combines image features with semantic features,which improves 2.2 percentage points when compared to the unimodal algorithm.The score level fusion method offers 3.5 percentage points improvement over the unimodal algorithm,with the optimal accuracy up to 99.6%.The generalizability and accuracy of the proposed multimodal biometric fusion model are significantly increased.

关 键 词：图像处理 卷积神经网络 多模态 生物特征检索 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]