基于Transformer模型的问句语义相似度计算  

Semantic similarity calculation of questions based on Transformer model

作　　者：丁邱 迟海洋 严馨[1,2] 徐广义 邓忠莹 DING Qiu;CHI Hai-yang;YAN Xin;XU Guang-yi;DENG Zhong-ying(Faculty of Information Engineering and Automation,Kunming University of Science and Technology,Kunming 650500,China;Yunnan Key Laboratory of Artificial Intelligence,Kunming University of Science and Technology,Kunming 650500,China;Information Center,Kunming University,Kunming 650214,China;Kunming Nantian Computer System Limited Company,Yunnan Nantian Electronic Information Industry Limited Company,Kunming 650040,China)

机构地区：[1]昆明理工大学信息工程与自动化学院,云南昆明650500 [2]昆明理工大学云南省人工智能重点实验室,云南昆明650500 [3]昆明学院信息中心,云南昆明650214 [4]云南南天电子信息产业股份有限公司昆明南天电脑系统有限公司,云南昆明650040

出　　处：《计算机工程与设计》2023年第3期887-893,共7页Computer Engineering and Design

基　　金：国家自然科学基金项目(61562049、61462055)。

摘　　要：针对现有方法准确率不高、不能充分捕捉句子深层次语义特征的问题,提出一种基于Transformer编码器网络的问句相似度计算方法。在获取句子语义特征前引入交互注意力机制比较句子间词粒度的相似性,通过注意力矩阵和句子矩阵相互生成彼此注意力加权后的新的句子表示矩阵,将获取的新矩阵同原始矩阵拼接融合,丰富句子特征信息;将拼接后的句子特征矩阵作为Transformer编码器网络的输入,由Transformer编码器分别对其进行深层次语义编码,获得句子的全局语义特征;通过全连接网络和Softmax函数对特征进行权重调整,得到句子相似度。在中文医疗健康问句数据集上模型取得了90.2%的正确率,较对比模型提升了将近4.2%,验证了该方法可以有效提高句子的语义表示能力和语义相似度的准确性。To solve the problem that the existing methods are not accurate and can not fully capture the deep semantic features of sentences,a question similarity calculation method based on transformer encoder network was proposed.Before acquiring sentence semantic features,interactive attention mechanism was introduced to compare the similarity of word granularity between sentences.The attention matrix and sentence matrix were used to generate a new sentence representation matrix weighted by each other’s attention.The new matrix was combined with the original matrix to enrich the sentence feature information.The spliced sentence feature matrix was used as the input of the transformer encoder network,and the transformer encoder encoded them in deep level to obtain the global semantic features of sentences.The feature weight was adjusted and the sentence similarity was calculated through the full connection network and Softmax function.On the data set of Chinese medical and health questions,the accuracy of the model is 90.2%,which is nearly 4.2%higher than that of the comparative model.It is verified that this method can effectively improve the semantic representation ability of sentences and the accuracy of semantic similarity.

关 键 词：自然语言处理 Transformer编码器 交互注意力机制 特征融合 语义相似度 语义编码 句子表示 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]