NLWSNet:a weakly supervised network for visual sentiment analysis in mislabeled web images  

作　　者：Luo-yang XUE Qi-rong MAO Xiao-hua HUANG Jie CHEN 

机构地区：[1]Computer Science and Communication Engineering.Jiangsu University,Zhenjiang 212013,China [2]Jiangsu Key Laboratory of Security Technology for Industrial Cyberspace,Zhenjiang 212013,China [3]School of Computer Engineering,Narjing Institute of Technology,Nanjing 211167,China [4]Center for Machine Vision and Signal Analysis,University of Ouls,Oulu 8000,Finland

出　　处：《Frontiers of Information Technology & Electronic Engineering》2020年第9期1321-1333,共13页信息与电子工程前沿（英文版）

基　　金：Project supported by the Key Project of the National Natural Science Foundation of China(No.U1836220);the National Nat-ural Science Foundation of China(No.61672267);the Qing Lan Talent Program of Jiangsu Province,China,the Jiangsu Key Laboratory of Security Technology for Industrial Cyberspace,China,the Finnish Cultural Foundation,the Jiangsu Specially-Appointed Professor Program,China(No.3051107219003);the liangsu Joint Research Project of Sino-Foreign Cooperative Education Platform,China,and the Talent Startup Project of Nanjing Institute of Technology,China(No.YKJ201982)。

摘　　要：Large-scale datasets are driving the rapid developments of deep convolutional neural networks for visual sentiment analysis.However,the annotation of large-scale datasets is expensive and time consuming.Instead,it iseasy to obtain weakly labeled web images from the Internet.However,noisy labels st.ill lead to seriously degraded performance when we use images directly from the web for training networks.To address this drawback,we propose an end-to-end weakly supervised learning network,which is robust to mislabeled web images.Specifically,the proposed attention module automatically eliminates the distraction of those samples with incorrect labels bv reducing their attention scores in the training process.On the other hand,the special-class activation map module is designed to stimulate the network by focusing on the significant regions from the samples with correct labels in a weakly supervised learning approach.Besides the process of feature learning,applying regularization to the classifier is considered to minimize the distance of those samples within the same class and maximize the distance between different class centroids.Quantitative and qualitative evaluations on well-and mislabeled web image datasets demonstrate that the proposed algorithm outperforms the related methods.

关 键 词：Visual sentiment analysis Weakly supervised learning Mislabeled samples Significant sentiment regions 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]