针对低资源场景下连续情感分析任务的持续注意力建模  

Continual Attention Modeling for Successive Sentiment Analysis in Low-resource Scenarios

作　　者：张涵 王晶晶[1] 罗佳敏 周国栋[1] ZHANG Han;WANG Jing-Jing;LUO Jia-Min;ZHOU Guo-Dong(School of Computer Science and Technology,Soochow University,Suzhou 215006,China)

机构地区：[1]苏州大学计算机科学与技术学院,江苏苏州215006

出　　处：《软件学报》2024年第12期5470-5486,共17页Journal of Software

基　　金：国家自然科学基金(62006166,62076175,62076176);江苏高校优势学科建设工程。

摘　　要：目前情感分析的研究普遍基于大数据驱动型模型,严重依赖高昂的标注成本和算力成本,因此针对低资源场景下的情感分析研究显得尤为迫切.然而,存在的低资源场景下的情感分析研究主要集中在单个任务上,这导致模型难以获取外部任务知识.因此构建低资源场景下的连续情感分析任务,旨在利用持续学习方法,让模型随时间步学习多个情感分析任务.这样可以充分利用不同任务的数据,并学习不同任务的情感信息,从而缓解单个任务训练数据匮乏问题.认为低资源场景下的连续情感分析任务面临两大核心问题,一方面是单个任务的情感信息保留问题,另一方面是不同任务间的情感信息融合问题.为了解决上述两大问题,提出针对低资源场景下连续情感分析任务的持续注意力建模方法.所提方法首先构建情感掩码Adapter,用于为不同任务生成硬注意力情感掩码,这可以保留不同任务的情感信息,从而缓解灾难性遗忘问题.其次,所提方法构建动态情感注意力,根据当前时间步和任务相似度动态融合不同Adapter抽取的特征,这可以融合不同任务间的情感信息.在多个数据集上的实验结果表明:所提方法的性能显著超过了目前最先进的基准方法.此外,实验分析表明,所提方法较其他基准方法具有最优的情感信息能力和情感信息融合能力,并且能同时保持较高的运行效率.Currently,sentiment analysis research is generally based on big data-driven models,which heavily rely on expensive annotation and computational costs.Therefore,research on sentiment analysis in low-resource scenarios is particularly urgent.However,existing research on sentiment analysis in low-resource scenarios mainly focuses on a single task,making it difficult for models to acquire external task knowledge.Therefore,this study constructs successive sentiment analysis in low-resource scenarios,aiming to allow models to learn multiple sentiment analysis tasks over time by continual learning methods.This can make full use of data from different tasks and learn sentiment information from different tasks,thus alleviating the problem of insufficient training data for a single task.There are two core problems with successive sentiment analysis in low-resource scenarios.One is preserving sentiment information for a single task,and the other is fusing sentiment information between different tasks.To solve these two problems,this study proposes continual attention modeling for successive sentiment analysis in low-resource scenarios.Sentiment masked Adapter(SMA)is first constructed,which is used to generate hard attention emotion masks for different tasks.This can preserve sentiment information for different tasks and mitigate catastrophic forgetting.Secondly,dynamic sentiment attention(DSA)is proposed,which dynamically fuses features extracted by different Adapters based on the current time step and task similarity.This can fuse sentiment information between different tasks.Experimental results on multiple datasets show that the proposed approach significantly outperforms the state-of-the-art benchmark approaches.Additionally,experimental analysis indicates that the proposed approach has the best sentiment information retention ability and sentiment information fusion ability compared to other benchmark approaches while maintaining high operational efficiency.

关 键 词：情感分析 低资源场景 持续学习 ADAPTER 注意力机制 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]