通过深度神经网络与皮层网络的多级整合增强自然感知的神经编码模型  

Enhancing neural encoding models for naturalistic perception with a multi-level integration of deep neural networks and cortical networks

作　　者：李远宁 杨虎铮 顾实 Yuanning Li;Huzheng Yang;Shi Gu(School of Biomedical Engineering&State Key Laboratory of Advanced Medical Materials and Devices,ShanghaiTech University,Shanghai 201210,China;School of Computer Science and Engineering,University of Electronic Science and Technology of China,Chengdu 611731,China;Department of Computer and Information Science,University of Pennsylvania,Philadelphia,PA 19104,USA;Shenzhen Institute for Advanced Study,University of Electronic Science and Technology of China,Shenzhen 518110,China)

机构地区：[1]School of Biomedical Engineering&State Key Laboratory of Advanced Medical Materials and Devices,ShanghaiTech University,Shanghai 201210,China [2]School of Computer Science and Engineering,University of Electronic Science and Technology of China,Chengdu 611731,China [3]Department of Computer and Information Science,University of Pennsylvania,Philadelphia,PA 19104,USA [4]Shenzhen Institute for Advanced Study,University of Electronic Science and Technology of China,Shenzhen 518110,China

出　　处：《Science Bulletin》2024年第11期1738-1747,共10页科学通报（英文版）

基　　金：supported by the National Natural Science Foundation of China (62236009,61876032,32371154);Shenzhen Science and Technology Program (JCYJ20210324140807019);Shanghai Pujiang Program (22PJ1410500)。

摘　　要：Cognitive neuroscience aims to develop computational models that can accurately predict and explain neural responses to sensory inputs in the cortex. Recent studies attempt to leverage the representation power of deep neural networks(DNNs) to predict the brain response and suggest a correspondence between artificial and biological neural networks in their feature representations. However, typical voxel-wise encoding models tend to rely on specific networks designed for computer vision tasks, leading to suboptimal brain-wide correspondence during cognitive tasks. To address this challenge, this work proposes a novel approach that upgrades voxel-wise encoding models through multi-level integration of features from DNNs and information from brain networks. Our approach combines DNN feature-level ensemble learning and brain atlas-level model integration, resulting in significant improvements in predicting whole-brain neural activity during naturalistic video perception. Furthermore, this multi-level integration framework enables a deeper understanding of the brain's neural representation mechanism, accurately predicting the neural response to complex visual concepts. We demonstrate that neural encoding models can be optimized by leveraging a framework that integrates both data-driven approaches and theoretical insights into the functional structure of the cortical networks.

关 键 词：Neural encoding Visual perception Artificial neural networks Functional neuroimaging Naturalistic stimuli 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]