基于张量计算的卷积神经网络语义表示学习  

Semantic Representation Learning of Convolutional Neural Network Based on Tensor Computation

作　　者：杨礼吉 王家祺 景丽萍[1,2] 于剑[1,2] YANG Li-Ji;WANG Jia-Qi;JING Li-Ping;YU Jian(School of Computer and Information Technology,Beijing Jiaotong University,Beijing 100044;Beijing Key Laboratory of Traffic Data Analysis and Mining,Beijing Jiaotong University,Beijing 100044)

机构地区：[1]北京交通大学计算机与信息技术学院,北京100044 [2]北京交通大学交通数据分析与挖掘北京市重点实验室,北京100044

出　　处：《计算机学报》2023年第3期568-578,共11页Chinese Journal of Computers

基　　金：国家科技研发计划(2020AAA0106800);国家自然科学基金项目(62176020);北京市自然科学基金项目(Z180006);中国人工智能学会-华为MindSpore学术奖励基金;中国科学院光电信息处理重点实验室开放课题基金(OEIP-O-202004)资助.

摘　　要：卷积神经网络已在多个领域取得了优异的性能表现,然而由于其不透明的内部状态,其可解释性依然面临很大的挑战.其中一个原因是卷积神经网络以像素级特征为输入,逐层地抽取高级别特征,然而这些高层特征依然十分抽象,人类不能直观理解.为了解决这一问题,我们需要表征出网络中隐藏的人类可理解的语义概念.本文通过预先定义语义概念数据集(例如红色、条纹、斑点、狗),得到这些语义在网络某一层的特征图,将这些特征图作为数据,训练一个张量分类器.我们将与分界面正交的张量称为语义激活张量(Semantic Activation Tensors,SATs),每个SAT都指向对应的语义概念.相对于向量分类器,张量分类器可以保留张量数据的原始结构.在卷积网络中,每个特征图中都包含了位置信息和通道信息,如果将其简单地展开成向量形式,这会破坏其结构信息,导致最终分类精度的降低.本文使用SAT与网络梯度的内积来量化语义对分类结果的重要程度,此方法称为TSAT(Testing with SATs).例如,条纹对斑马的预测结果有多大影响.本文以图像分类网络作为解释对象,数据集选取ImageNet,在ResNet50和Inceptionv3两种网络架构上进行实验验证.最终实验结果表明,本文所采用的张量分类方法相较于传统的向量分类方法,在数据维度较大或数据不易区分的情况下,分类精度有显著的提高,且分类的稳定性也更加优秀.这从而保证了本文所推导出的语义激活张量更加准确,进一步确保了后续语义概念重要性量化的准确性.Convolutional neural networks have achieved excellent performance in several areas,but their interpretability still faces significant challenges due to their opaque internal stale.One reason for this is that convolulional neural networks take pixel-level features as input and extract high-level features layer by layer,but these high-level features are still very abstract and cannot be understood intuitively by humans.To solve this problem,we need to characterize the humanunderstandable semantic concepts hidden in the network.In this paper,we obtain feature maps of these semantic concepts at one layer of the network by pre-defining a dataset of semantic concepts(e.g.red,stripes,spots,dogs) and use these feature maps as data to train a tensor classifier.We refer to the tensor orthogonal to the partitioned interface as Semantic Activation Tensors(SATs),and each SAT points to a corresponding semantic concept.In contrast to vector classifiers,tensor classifiers can preserve the original structure of the tensor data.In a convolutional network,each feature map contains location and channel information,and if it is simply expanded into vector form,this would destroy its structural information and lead to a reduction in the final classification accuracy.In this paper,we use the inner product of SATs and network gradients to quantify the importance of semantics on classification results,a method known as TSAT(Testing with SATs).For example,how much streaks affect the prediction results of zebras.In this paper,the image classification network is used as the object of interpretation,and ImageNet is selected as the dataset for experimental validation on both ResNel50 and Inceptionv3 network architectures.The final experimental results show that the tensor classification method used in this paper has a significant improvement in classification accuracy and better classification stability compared to the traditional vector classification method in the case of larger data dimensions or data that are not easily distinguishable.Th

关 键 词：深度学习 卷积神经网络 语义建模 张量表示 支持张量机 张量分类 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]