Saliency-guided meta-hallucinator for few-shot learning  

作　　者：Hongguang ZHANG Chun LIU Jiandong WANG Linru MA Piotr KONIUSZ Philip HSTORR Lin YANG 

机构地区：[1]Systems Engineering Institute,Academy of Military Sciences,Beijing 100141,China [2]Data61,Commonwealth Scientific and Industrial Research Organisation,Canberra ACT 2601,Australia [3]Australian National University,Canberra ACT 2601,Australia [4]Oxford University,Oxford OX36PJ,UK

出　　处：《Science China(Information Sciences)》2024年第10期185-206,共22页中国科学（信息科学）（英文版）

基　　金：supported by National Natural Science Foundation of China (Grant No. 62106282);Beijing Nova Program (Grant No. 20220484139)。

摘　　要：Learning novel object concepts from limited samples remains a considerable challenge in deep learning. The main directions for improving the few-shot learning models include(i) designing a stronger backbone,(ii) designing a powerful(dynamic) meta-classifier, and(iii) using a larger pre-training set obtained by generating or hallucinating additional samples from the small scale dataset. In this paper, we focus on item(iii) and present a novel meta-hallucination strategy. Presently, most image generators are based on a generative network(i.e., GAN) that generates new samples from the captured distribution of images. However, such networks require numerous annotated samples for training. In contrast, we propose a novel saliency-based end-to-end meta-hallucinator, where a saliency detector produces foregrounds and backgrounds of support images. Such images are fed into a two-stream network to hallucinate feature samples directly in the feature space by mixing foreground and background feature samples. Then, we propose several novel mixing strategies that improve the quality and diversity of hallucinated feature samples.Moreover, as not all saliency maps are meaningful or high quality, we further introduce a meta-hallucination controller that decides which foreground feature samples should participate in mixing with backgrounds. To our knowledge, we are the first to leverage saliency detection for few-shot learning. Our proposed network achieves state-of-the-art results on publicly available few-shot image classification and anomaly detection benchmarks, and outperforms competing sample mixing strategies such as the so-called Manifold Mixup.

关 键 词：few-shot learning saliency detection object recognition anomaly detection computer vision 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程] TP391.41[自动化与计算机技术—控制科学与工程]