基于多头注意力机制Tree-LSTM的句子语义相似度计算  

Sentence Semantic Similarity Computation Based on Tree-LSTM with Multi-head Attention

作　　者：胡艳霞 王成[1] 李弼程[1] 李海林[2] 吴以茵[1] HU Yanxia;WANG Cheng;LI Bicheng;LI Hailin;WU Yiyin(School of Computer Science and Technology,Huaqiao University,Xiamen,Fujian 361021,China;School of Business Administration,Huaqiao University,Quanzhou,Fujian 362021,China)

机构地区：[1]华侨大学计算机科学与技术学院,福建厦门361021 [2]华侨大学工商管理学院,福建泉州362021

出　　处：《中文信息学报》2020年第3期23-33,共11页Journal of Chinese Information Processing

基　　金：国家自然科学基金(71771094;71571056);福建省科技计划引导性项目(2017H01010065);2018年华侨大学研究生教育教学改革研究立项项目(18YJG28);泉州市丰泽区科技计划项目(2018FZ06);国家社会科学基金(19BXW110)。

摘　　要：针对现有句子语义相似度计算由于缺乏语义结构信息导致精度低的问题,该文在依存关系树的基础上,提出了一种基于多头注意力机制Tree-LSTM(multi-head attention Tree-LSTM,MA-Tree-LSTM)的句子语义相似度计算方法。首先,MA-Tree-LSTM将外部具有指导意义的特征作为输入,再将输入结合多头注意力机制作用在Tree-LSTM树节点的所有孩子节点上,为每个孩子节点赋予不同的权重值,从而实现多头注意力机制和Tree-LSTM的融合;其次,将三层的MA-Tree-LSTM应用于句子语义相似度计算并实现句子对的相互指导,从而得到句子对语义特征的多层表示;最后联合多层的语义特征建立句子对语义相似度计算模型,从而实现句子对间相关的语义结构特征的充分利用。该文提出的方法鲁棒性强,可解释性强,对句子单词的顺序不敏感,不需要特征工程。在SICK和STS数据集上的实验结果表明,基于MA-Tree-LSTM的句子语义相似度计算的精度优于非注意力机制的Tree-LSTM方法以及融合了多头注意力机制的BiLSTM方法。A sentence semantic similarity computation based on Tree-LSTM with multi-head attention(MA-Tree-LSTM) is proposed according to the dependency tree. First, with external instructive features as input, the multi-head attention mechanism is applied to weigh the tree nodes in the Tree-LSTM. Second, the three-layer MA-Tree-LSTM is trained on the sentence semantic similarity calculation task to obtain the multi-layers representation of the semantic features. Finally, the semantic feature of the multi-layer is chosen to establish the semantic similarity calculation model of sentence, to make full use of the semantic structure features in the sentence pairs. The proposed method is robust, interpretable, and insensitive to the order of sentence words without feature engineering. Experimental results on the SICK and STS datasets prove the proposed method is better than Tree-LSTM and BiLSTM.

关 键 词：句子语义相似度计算 多头注意力机制 Tree-LSTM 语义依存树 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]