基于文本表达的指向性目标分割方法研究  

Research on Referring Expression Segmentation Method Based on Text Expression

作　　者：魏庆为 张丽红[1] WEI Qingwei;ZHANG Lihong(College of Physics and Electmnic Engineering, Shanxi University, Taiyuan 030006, China)

机构地区：[1]山西大学物理电子工程学院,山西太原030006

出　　处：《测试技术学报》2022年第1期42-47,59,共7页Journal of Test and Measurement Technology

基　　金：山西省研究生教育改革课题(2019JG031);山西省研究生创新项目(2020SY024)。

摘　　要：本文提出了一种基于文本表达式理解的图像分割网络,旨在用给定的文本表达式分割出图片中的目标区域.该模型完成了对目标区域的精确定位,实现了视觉和文本的对齐,同时很好地处理了预测分歧问题.首先,使用卷积神经网络和双向门控循环神经网络分别提取视觉特征和文本特征;然后,将提取的视觉特征和文本特征融合生成多模态特征,同时使用自适应特征选择的方法提取不同尺度的视觉特征;最后,利用改进的空洞空间金字塔池化模块加快网络的训练速度,经模型的后处理操作输出分割图.本文的网络模型在数据集RefCOCO和RefCOCO+上进行训练和测试,实验表明,本文方法可有效提高模型的分割效果.In this paper,an image segmentation network based on text expression comprehension is proposed,which aims to segment the target area in a image with a given text expression.The model can accurately locate the target area,realize the alignment of vision and text,and deal with the problem of prediction conflict well.Firstly,visual features and text features were extracted using Convolutional Neural Network and bi-directional gated recurrent neural network respectively.Then,visual features and text features are combined to generate multimodal features,and visual features of different scales are extracted using adaptive feature selection method.Finally,the improved Atrous Spatial Pyramid Pooling module is used to speed up the training speed of the network,and the segmentation map is output through the post-processing operation of the model.The network model of this paper is trained and tested on the datasets RefCOCO and RefCOCO+.Experiments show that the method in this paper can effectively improve the segmentation effect of the model.

关 键 词：文本表达式 多模态 自适应特征选择 指向性目标分割 协同能量最大化 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]