一种整合语义对象特征的视觉注意力模型  

Incorporating semantic object features into a visual attention model

作　　者：李娜[1,2] 赵歆波 LI Na;ZHAO Xinbo(School of Computer Science,Northwestern Polytechnical University,Xi'an 710129,China;Shaanxi Provincial Key Laboratory of Speech and Image Information Process,Xi'an 710129,China)

机构地区：[1]西北工业大学计算机学院,西安710129 [2]陕西省语音与图像信息处理重点实验室,西安710129

出　　处：《哈尔滨工业大学学报》2020年第5期99-105,共7页Journal of Harbin Institute of Technology

基　　金：国家自然科学基金(61231016,61871326)。

摘　　要：视觉注意力建模作为预测人类在观察场景时注意力分布的关键技术,在计算机视觉的众多领域均有广泛应用.传统的视觉注意力模型着重研究人眼注视点,计算出的显著图更多的是反映眼动信息,并未将大脑的感知出的语义信息反映出来.针对这一问题,本文提出了一种整合了语义对象特征的视觉注意力模型.首先,本文建立了眼动跟踪数据库V0C2012-E,研究并记录普通人在观察自然场景时的眼动数据.然后,受语义分割启发,利用全卷积神经网络(Fully Convolution;Networks,FCN)提取语义对象特征,同时用激活函数PReLu和优化函数Adam改进FCN网络使其更有效地提取的语义对象特征,来模仿大脑对语义对象特征的感知.接着,提取在人类潜意识层吸引人注意力的如方向,颜色,强度特征等28个低级特征.最后利用支持向量机(Support Vector Machine,SVM)将之前提取的语义对象特征及低级特征映射到人类视觉空间,同时引入真实眼动数据进行有监督的训练,得到可以预测人眼视觉显著图的视觉注意力模型.实验结果表明,在VOC2012-E及MIT300数据库上与其他8种经典模型及4种先进模型相比,本文提出的视觉注意力模型性能更好,更有生物学优势.Visual attention modeling is a key technique for predicting the distribution of human attention when people are observing scenes,which is widely used in the fields of computer vision.Traditional visual attention models focus on the human eyes fixation points to reflect the eye movement information by calculating saliency maps,while they cannot reflect the perceived semantic information of the brain.To solve this problem,a visual attention model was proposed based on extracting semantic features.First of all,the eye tracking database VOC2012-E was established to study and record the eye movement data of human while observing natural scenes.Then,inspired by image semantic segmentation,the Fully Convolutional Networks(FCN)was used to extract the semantic object features.In order to extract the semantic object features more effectively,the FCN8s network was improved by activation function PReLu and optimization function Adam to mimic the brain’s perception of semantic objeet features.Next,28 low-level features such as direction,color,and intensity characteristics were extracted,which attract attention in the human subconscious layer.Finally,Support Vector Machine(SVM)was used to map the previously extracted semantic object features and the low-level features into the human visual space.The real eye movement data was introduced for supervised training,and a visual attention model was obtained which can predict the human visual saliency map.Experimental results showed that the visual attention model proposed in this paper had better performance and biological advantages over the other eight classical models and four advanced models on the VOC2012-E and MIT300 databases.

关 键 词：视觉注意力模型 语义对象特征 FCN SVM 深度学习 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]