基于双向长效注意力特征表达的少样本文本分类模型研究  

Classification Model for Few-shot Texts Based on Bi-directional Long-term Attention Features

作　　者：徐彤彤 孙华志[1] 马春梅[1] 姜丽芬[1] 刘逸琛 Xu Tongtong;Sun Huazhi;Ma Chunmei;Jiang Lifen;Liu Yichen(College of Computer and Information Engineering,Tianjin Normal University,Tianjin 300387,China)

机构地区：[1]天津师范大学计算机与信息工程学院,天津300387

出　　处：《数据分析与知识发现》2020年第10期113-123,共11页Data Analysis and Knowledge Discovery

基　　金：天津市教委科研计划项目“基于免疫原理的营养均衡量化配餐推荐算法的研究”(项目编号:JW1702);天津市自然科学基金项目“物联网环境中的上下文感知关键技术研究”(项目编号:18JCYBJC85900);天津市自然科学基金项目“面向智能交通基于智能手机的车辆检测技术及其应用研究”(项目编号:18JCQNJC70200)的研究成果之一。

摘　　要：【目的】针对当前文本分类任务中存在的训练数据匮乏以及模型泛化性能低等问题,在少样本环境下研究文本分类问题,提出一种少样本文本分类模型。【方法】基于元学习中的分段训练机制将文本分类任务划分为多个子任务;为了捕捉每个子任务中文本的长效上下文信息,提出双向时间卷积网络;为了捕获辨别力更强的特征,联合双向时间卷积网络和注意力机制提出双向长效注意力网络;利用一种新的神经网络模型度量每个子任务中查询样本与支持集的相关性,从而实现少样本文本分类。【结果】在ARSC数据集上进行实验,实验结果表明,在少样本环境下,该模型的分类准确率高达86.80%,比现有先进的少样本文本分类模型ROBUSTTC-FSL和Induction-Network-Routing的准确率分别提高了3.68%和1.17%。【局限】仅针对短文本分类问题,对于篇幅较长的文本,其分类能力有限。【结论】双向长效注意力网络克服了训练数据匮乏问题且充分捕获文本的语义信息,有效提高了少样本文本分类性能。[Objective] This paper proposes a classification model for few-shot texts, aiming to address the issues of data scarcity and low generalization performance. [Methods] First, we divided the text classification tasks into multiple subtasks based on episode training mechanism in meta-learning. Then, we proposed a Bi-directional Temporal Convolutional Network(Bi-TCN) to capture the long-term contextual information of the text in each subtask. Third, we developed a Bi-directional Long-term Attention Network(BLAN) to capture more discriminative features based on Bi-TCN and multi-head attention mechanism. Finally, we used the Neural Tensor Network to measure the correlation between query samples and support set of each subtask to finish few-shot text classification. [Results] We examined our model with the ARSC dataset. The classification accuracy of this model reached 86.80% in few-shot learning setting, which was 3.68% and 1.17% better than those of the ROBUSTTC-FSL and Induction-Network-Routing models. [Limitations] The performance of BLAN on long text is not satisfactory. [Conclusions] BLAN overcomes the issue of data scarcity and captures comprehensive text features, which effectively improves the performance of few-shot text classification.

关 键 词：少样本文本分类 注意力机制 少样本学习 双向时间卷积网络 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]