基于对比学习的大型语言模型反向词典任务提示生成方法  

Contrastive Learning-based Prompt Generation Method for Large-scale Language Model Reverse Dictionary Task

作　　者：田思成 黄少滨[1] 王锐[1] 李熔盛 杜治娟 TIAN Sicheng;HUANG Shaobin;WANG Rui;LI Rongsheng;DU Zhijuan(College of Computer Science and Technology,Harbin Engineering University,Harbin 150001,China;Engineering Research Center of Ecological Big Data,Ministry of Education,Inner Mongolia,010021,China;College of Computer,Inner Mongolia University,Inner Mongolia,010021,China)

机构地区：[1]哈尔滨工程大学计算机科学与技术学院,哈尔滨150001 [2]生态大数据教育部工程研究中心,内蒙古010021 [3]内蒙古大学计算机学院,内蒙古010021

出　　处：《计算机科学》2024年第8期256-262,共7页Computer Science

基　　金：生态大数据教育部工程研究中心开放课题。

摘　　要：反向词典任务是一种新兴的任务,目的是根据给定的定义来查找对应的单词。大规模语言模型为这一任务提供了新的可能性,但是提示语句的质量会影响大模型的性能。为此,提出了一种基于对比学习的提示生成方法。该方法在从多个语义层面上理解定义语义的同时,还利用对比学习的原理在训练过程中引入了负例,提升了模型的泛化能力。通过这种方法,可以将目标单词缩小到一个小范围内,然后用大模型从这个范围内选择最符合定义语义的单词。实验结果表明,该方法可以有效地提升大规模语言模型在反向词典任务上的表现。提示生成模型有94.7%的概率生成包含目标词的范围,大规模语言模型有58.03%的概率直接选出目标单词,有74.55%的概率在给出5个候选单词时包含目标单词。Reverse dictionary task is an emerging task that aims to find the corresponding word based on a given definition.Large-scale language models offer new possibilities for this task,but the quality of the prompt sentences affects the performance of the large models.To this end,this paper proposes a contrastive learning-based prompt generation method.This method extracts definition semantics from multiple semantic levels.It also enhances the model’s generalization ability by incorporating negative examples through contrastive learning.With this method,we can narrow down the target word to a small range,and use a large model to select the most semantically consistent word from this range.Experimental results show that the proposed method can effectively improve the performance of large-scale language models on the reverse dictionary task.The prompt generation model has a 94.7%probability of generating a range that contains the target word.The large-scale language model has a 58.03%pro-bability of directly selecting the target word,and a 74.55%probability of including the target word when five candidate words are given.

关 键 词：反向词典 大规模语言模型 对比学习 多个语义层面 对比损失 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]