Lateral interaction by Laplacian‐based graph smoothing for deep neural networks  

作　　者：Jianhui Chen Zuoren Wang Cheng‐Lin Liu 

机构地区：[1]Institute of Neuroscience,State Key Laboratory of Neuroscience,Center for Excellence in Brain Science and Intelligence Technology,Chinese Academy of Sciences,Shanghai,China [2]State Key Laboratory of Multimodal Artificial Intelligence Systems,Institute of Automation of Chinese Academy of Sciences,Beijing,China [3]University of Chinese Academy of Sciences,Beijing,China [4]School of Future Technology,University of Chinese Academy of Sciences,Beijing,China [5]School of Life Science and Technology,ShanghaiTech University,Shanghai,China

出　　处：《CAAI Transactions on Intelligence Technology》2023年第4期1590-1607,共18页智能技术学报（英文）

基　　金：supported by the National Natural Science Foundation of China grants 61836014 to CL,and the STI2030‐Major Projects(2022ZD0205100);the Strategic Priority Research Program of Chinese Academy of Science,Grant No.XDB32010300;Shanghai Municipal Science and Technology Major Project(Grant No.2018SHZDZX05);the Innovation Academy of Artificial Intelligence,Chinese Academy of Sciences to ZW.

摘　　要：Lateral interaction in the biological brain is a key mechanism that underlies higher cognitive functions.Linear self‐organising map(SOM)introduces lateral interaction in a general form in which signals of any modality can be used.Some approaches directly incorporate SOM learning rules into neural networks,but incur complex operations and poor extendibility.The efficient way to implement lateral interaction in deep neural networks is not well established.The use of Laplacian Matrix‐based Smoothing(LS)regularisation is proposed for implementing lateral interaction in a concise form.The authors’derivation and experiments show that lateral interaction implemented by SOM model is a special case of LS‐regulated k‐means,and they both show the topology‐preserving capability.The authors also verify that LS‐regularisation can be used in conjunction with the end‐to‐end training paradigm in deep auto‐encoders.Additionally,the benefits of LS‐regularisation in relaxing the requirement of parameter initialisation in various models and improving the classification performance of prototype classifiers are evaluated.Furthermore,the topologically ordered structure introduced by LS‐regularisation in feature extractor can improve the generalisation performance on classification tasks.Overall,LS‐regularisation is an effective and efficient way to implement lateral interaction and can be easily extended to different models.

关 键 词：artificial neural networks biologically plausible Laplacian‐based graph smoothing lateral interaction machine learning 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]