基于多窗口注意力机制的中文命名实体识别  

Chinese Named Entity Recognition Based on Multi-window Attention Mechanism

作　　者：占文韬 吴晓鸰 凌捷[1] ZHAN Wentao;WU Xiaoling;LING Jie(School of Computing,Guangdong University of Technology,Guangzhou 510006,China)

机构地区：[1]广东工业大学计算机学院,广州510006

出　　处：《小型微型计算机系统》2024年第6期1325-1330,共6页Journal of Chinese Computer Systems

基　　金：工业装备质量大数据工业和信息化部重点实验室开放课题项目(2021-1EQBD-02)资助;广东省国际科技合作领域项目(2019A050513010)资助。

摘　　要：近年来,由于Transformer模型中应用的多头注意力机制能够有效地捕获较长的上下文信息,基于Transformer的模型已经成为主流方法.尽管这些模型能够有效捕获全局上下文信息,它们在局部特征和位置信息提取方面仍然有限.因此,本文提出了一种基于多窗口注意力机制的中文命名实体识别模型.首先,通过基于Transformer的预训练语言模型RoBERTa把文本表征为字符级嵌入向量,捕捉深度上下文信息,得到全局上下文信息;其次,局部特征提取模块利用多窗口循环机制,在全局特征提取模块的引导下提取局部特征和位置信息;最后,所提出的多窗口注意力机制有效地融合全局特征和多个局部特征来预测实体标签.在CMeEE和MSRA数据集上进行了实验验证;结果表明,本文所提出的模型分别获得了64.31%和94.14%的F1值,性能优于其他同类模型,验证了其在中文命名实体识别的有效性.In recent years,Transformer-based models have become the dominant approach due to the ability of the multi-headed attention mechanism applied in Transformer models to effectively capture longer contextual information.Although these models can effectively capture global contextual information,they are still limited in terms of local feature and location information extraction.Therefore,this paper proposes a Chinese named entity recognition model based on a multi-window attention mechanism.Firstly,the text is characterised as a character-level embedding vector by the Transformer-based pre-trained language model RoBERTa,which captures deep contextual information and obtains global contextual information;secondly,the local feature extraction module uses a multi-window loop mechanism to extract local features and location information under the guidance of the global feature extraction module;finally,the proposed multi-window attention mechanism effectively Finally,the proposed multi-window attention mechanism effectively fuses global features and multiple local features to predict entity labels.Experimental validation was conducted on the CMeEE and MSRA datasets;the results showed that the proposed model obtained 64.31%and 94.14%of F1 values respectively,outperforming other similar models and proving its effectiveness in Chinese named entity recognition.

关 键 词：命名实体识别 多窗口注意力机制 特征融合 RoBERTa 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]