融合多头图注意力的文本相似度计算  

Text similarity calculation with graph attention

作　　者：申晓旭 王忠燎 包训成 吴醒宇 张茂萱 张艳迪 SHEN Xiaoxu;WANG Zhongliao;BAO Xuncheng;WU Xingyu;ZHANG Maoxuan;ZHANG Yandi(Zhejiang Xingfu Rail Transit Operation Management Co.,Ltd.,Wenzhou 325000,China;Tukrin Technology Co.,Ltd.Beijing,Beijing 101300,China)

机构地区：[1]浙江幸福轨道交通运营管理有限公司,浙江温州325000 [2]航天图景(北京)科技有限公司,北京101300

出　　处：《应用科技》2025年第1期205-212,共8页Applied Science and Technology

摘　　要：现有的文本相似度计算方法在处理长文本或具有复杂结构句子时,无法有效挖掘文本的深层语义,从而影响相似度计算的精度。为了解决这一问题,本文提出了一种融合图注意力的孪生网络框架,该框架结合了图神经网络的图嵌入和节点嵌入2种编码方式。在图嵌入支路中,引入图注意力机制,解决了长距离节点受限于图卷积层数无法充分传播和整合远距离节点的信息的问题;在节点嵌入支路中,采用残差结构以缓解节点过度平滑的问题,从而更好地挖掘节点间的语义关系。实验结果表明,本文方法在语义文本相似性(semantic textual similarity,STS)基准的多个数据集上表现优异,且相较于当前先进的双向编码器表征模型模型,具有更高的文本相似度计算精度。Existing text similarity calculation methods fail to effectively capture the deep semantics of text when processing long texts or sentences with complex structures,thus affecting the accuracy of similarity computation.To address this issue,this paper proposes a twin-network framework integrating graph attention.The framework combines two encoding methods of graph neural networks:node embedding and graph embedding.In the graph embedding branch,a graph attention mechanism is introduced to overcome the problem of long-distance node limitations due to the restricted number of graph convolution layers that prevents sufficient information propagation and integration of long-distance nodes.In the node embedding branch,a residual structure is adopted to alleviate the over-smoothing problem of nodes,thereby better capturing the semantic relationships between nodes.Experimental results demonstrate that the proposed method outperforms current advanced models,such as BERT,in terms of text similarity calculation accuracy across multiple datasets in the STS benchmark.

关 键 词：图 图神经网络 图嵌入 注意力机制 文本表示 文本相似度 自然语言处理 深度学习 

分 类 号：TP399[自动化与计算机技术—计算机应用技术]