Joint regression and learning from pairwise rankings for personalized image aesthetic assessment  

作　　者：Jin Zhou Qing Zhang Jian-Hao Fan Wei Sun Wei-Shi Zheng 

机构地区：[1]School of Electronics and Information Technology,Sun Yat-sen University,Guangzhou 510006,China. [2]School of Computer Science and Engineering,Sun Yat-sen University,Guangzhou 510006,China [3]Key Laboratory of Machine Intelligence and Advanced Computing,Ministry of Education(Sun Yat-sen University),Guangzhou 510006,China [4]Peng Cheng Laboratory,Shenzhen 518000,China

出　　处：《Computational Visual Media》2021年第2期241-252,共12页计算可视媒体（英文版）

基　　金：supported partially by the National Key Research and Development Program of China(2018YFB1004903);National Natural Science Foundation of China(61802453,U1911401,U1811461);Fundamental Research Funds for the Central Universities(19lgpy216);Research Projects of Zhejiang Lab(2019KD0AB03).

摘　　要：Recent image aesthetic assessment methods have achieved remarkable progress due to the emergence of deep convolutional neural networks(CNNs).However,these methods focus primarily on predicting generally perceived preference of an image,making them usually have limited practicability,since each user may have completely different preferences for the same image.To address this problem,this paper presents a novel approach for predicting personalized image aesthetics that fit an individual user’s personal taste.We achieve this in a coarse to fine manner,by joint regression and learning from pairwise rankings.Specifically,we first collect a small subset of personal images from a user and invite him/her to rank the preference of some randomly sampled image pairs.We then search for the K-nearest neighbors of the personal images within a large-scale dataset labeled with average human aesthetic scores,and use these images as well as the associated scores to train a generic aesthetic assessment model by CNN-based regression.Next,we fine-tune the generic model to accommodate the personal preference by training over the rankings with a pairwise hinge loss.Experiments demonstrate that our method can effectively learn personalized image aesthetic preferences,clearly outperforming state-of-the-art methods.Moreover,we show that the learned personalized image aesthetic benefits a wide variety of applications.

关 键 词：s personalized image aesthetic assessment deep convolutional neural networks pairwise ranking regression 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术] TP18[自动化与计算机技术—计算机科学与技术]