基于逐层增量分解的深度网络神经元相关性解释方法  

Layer-wise Increment Decomposition-based Neuron Relevance Explanation for Deep Networks

作　　者：陈艺元 李建威 邵文泽[1,2] 孙玉宝[3] CHEN Yi-Yuan;LI Jian-Wei;SHAO Wen-Ze;SUN Yu-Bao(Jiangsu Key Laboratory of Intelligent Information Processing and Communication Technology,Nanjing University of Posts and Telecommunications,Nanjing 210003;College of Telecommunications and Information Engineering,Nanjing University of Posts and Telecommunications,Nanjing 210003;Engineering Research Center for Digital Forensics,Ministry of Education,Nanjing University of Information Science and Technology,Nanjing 210044)

机构地区：[1]智能信息处理与通信技术省高校重点实验室(南京邮电大学),南京210003 [2]南京邮电大学通信与信息工程学院,南京210003 [3]南京信息工程大学教育部数字取证工程研究中心,南京210044

出　　处：《自动化学报》2024年第10期2049-2062,共14页Acta Automatica Sinica

基　　金：国家自然科学基金(61771250,61972213,62276139,U2001211),青蓝工程资助。

摘　　要：神经网络的黑箱特性严重阻碍了人们关于网络决策的直观分析与理解.尽管文献报道了多种基于神经元贡献度分配的决策解释方法,但是现有方法的解释一致性难以保证,鲁棒性更是有待改进.本文从神经元相关性概念入手,提出一种基于逐层增量分解的神经网络解释新方法LID-Taylor(Layer-wise increment decomposition),且在此基础上先后引入针对顶层神经元相关性的对比提升策略,以及针对所有层神经元相关性的非线性提升策略,最后利用交叉组合策略得到最终方法SIG-LID-IG,实现了决策归因性能的鲁棒跃升.通过热力图对现有工作与提出方法的决策归因性能做了定性定量评估.结果显示,SIG-LID-IG在神经元的正、负相关性的决策归因合理性上均可媲美甚至优于现有工作.SIG-LID-IG在多尺度热力图下同样取得了精确性更高、鲁棒性更强的决策归因.The black box nature of deep neural networks seriously hinders one's intuitive analysis and understanding of network decision-making.Although various decision explanation methods based on neural contribution allocation have been reported in the literature,the consistency of existing methods is difficult to ensure,and their robustness still needs improvement.This article starts with the concept of neuron relevance and proposes a new neural network explanation method LID-Taylor(layer-wise increment decomposition).Aiming at LID-Taylor,a contrast lifting strategy for top-layer neuron relevance and a non-linear lifting strategy for all-layer neuron relevance are introduced,respectively.Finally,a cross combination strategy is applied,obtaining the final method SIG-LID-IG and achieving a robust leap in decision attribution performance.Both qualitative and quantitative evaluation have been conducted via heatmaps on the decision attribution performance of existing works and the proposed method.Results show that SIG-LID-IG is comparable or even superior to existing works in the rationality of positive and negative relevance of neurons in decision-making attribution.SIG-LID-IG has also achieved better accuracy and stronger robustness in decision-making attribution in terms of multi-scale heatmaps.

关 键 词：神经网络 可解释性 决策相关性 逐层相关性传播 类激活图 积分梯度 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]