融合多粒度动态语义表征的文本分类模型  

Text classification model incorporating multi-granularity dynamic semantic representation

作　　者：张骏强 高尚兵 苏睿 李文婷 ZHANG Junqiang;GAO Shangbing;SU Rui;LI Wenting(School of Computer and Software Engineering/Jiangsu Internet of Things Mobile Interconnection Technology Engineering Laboratory,Huaiyin Institute of Technology,Huaian)

机构地区：[1]淮阴工学院计算机与软件工程学院/江苏省物联网移动互联技术工程实验室,淮安223003

出　　处：《南京信息工程大学学报（自然科学版）》2023年第2期148-159,共12页Journal of Nanjing University of Information Science & Technology（Natural Science Edition）

基　　金：国家重点研发计划(2018YFB1004904);江苏省“六大人才高峰”资助项目(XYDX XJS-011);江苏省“333工程”资助项目(BRA2016454);江苏省教育厅重大研究项目(18KJA520001);淮阴工学院研究生科技创新计划项目(HGYK202121)。

摘　　要：在对化工领域类文本进行分类任务时,由于文本的专业性以及复杂多样性,仅仅依靠现有的词向量表征方式,很难对其中的专业术语以及其他化工领域内相关字词的语义进行充分表征,从而导致分类任务的准确率不高.本文提出一种融合多粒度动态语义表征的文本分类模型,首先在词嵌入层使用动态词向量表征语义信息并引入对抗扰动,使得词向量具有更好的表征能力,然后利用多头注意力机制进行词向量权重分配,获得带有关键语义信息的文本表示,最后使用提出的多尺度残差收缩深层金字塔形的卷积神经网络与混合注意力胶囊双向LSTM网络模型分别提取不同粒度的文本表示,融合后对得到的最终文本表示进行分类.实验结果表明,相比于现有模型,所提出的模型使用不同词向量表示时,在化工领域文本数据集上F1-Score最高可达84.62%,提升了0.38~5.58个百分点;在公开中文数据集THUCNews和谭松波酒店评论数据集ChnSentiCorp上进行模型泛化性能评估,模型也有较好表现.The widely used word vector representation is incapable of fully representing the specialized texts and phrases in sphere of highly specialized chemical industry,which were quite professional and complex,resulting in the low accuracy of classification.Here,we propose a text classification model incorporating multi-granularity dynamic semantic representation.First,the adversarial perturbation was introduced into the word embedding layer of the model to enhance the ability of dynamic word vectors to represent the semantics.Then the word vector weights were redistributed by a multi-headed attention mechanism to obtain a better textual representation of key semantic information.Finally,text representations of different granularities were extracted through the proposed multi-scale residual shrinkage deep pyramidal convolutional neural network(MSRS-DPCNN)and hybrid attention capsule bidirectional LSTM(HAC-BiLSTM)network model,which were then fused for classification.The experimental results showed that the proposed model achieved an F1-score up to 84.62%on the chemical domain text dataset when using different word vector representations,an improvement of 0.38-5.58 percentage points compared with existing models.The model also had pretty good generalization performance on the publicly available Chinese dataset THUCNews and the Tan Songbo hotel review dataset ChnSentiCorp.

关 键 词：文本分类 对抗扰动 多粒度 多头注意力机制 深度残差收缩 预训练语言模型 

分 类 号：TP391[自动化与计算机技术—计算机应用技术] TQ072[自动化与计算机技术—计算机科学与技术]