大语言模型驱动的立场感知事实核查  

Large language model-driven stance-aware fact-checking

作　　者：姜雨杉 张仰森[1] JIANG Yushan;ZHANG Yangsen(Institute of Intelligent Information Processing,Beijing Information Science and Technology University,Beijing 100101,China)

机构地区：[1]北京信息科技大学智能信息处理研究所,北京100101

出　　处：《计算机应用》2024年第10期3067-3073,共7页journal of Computer Applications

基　　金：国家自然科学基金资助项目(62176023)。

摘　　要：为解决事实核查领域的证据立场不平衡和忽略立场信息的问题,提出一种大语言模型(LLM)驱动的立场感知事实核查(LLM-SA)方法。首先,使用LLM推理并生成一系列与原始声明立场不同的辩证声明,使事实核查任务能够获取不同立场的视角;其次,通过语义相似度计算衡量每个证据句子与原始声明及辩证声明之间的相关性,并从证据句子中分别选择与两者语义上最相近的k个句子,作为支持或反对原始声明的证据,从而获得代表不同立场的证据,有助于事实核查模型综合多角度的信息,更准确地评估声明的真实性;最后,引入BERT-StuSE(Bidirectional Encoder Representations from Transformers-based Stance-infused Semantic Encoding network)模型,利用多头注意力机制充分融合证据的语义和立场信息,并更全面、客观地判断声明和证据的关系。在CHEF数据集上的实验结果表明,与BERT方法相比,所提方法在测试集上的Micro F1值和Macro F1值分别提高了3.52、3.90个百分点,达到较好的水平。验证了所提方法的有效性,以及考虑不同立场的证据和充分利用证据的立场信息对事实核查的性能提升具有重要意义。To address the issues of evidence stance imbalance and neglect of stance information in the field of Fact-Checking(FC),a Large Language Model-driven Stance-Aware fact-checking(LLM-SA)method was proposed.Firstly,a series of dialectical claims that differed from the original claim were generated by using a large language model,to capture different perspectives for fact-checking.Secondly,through semantic similarity calculations,the relevances of each piece of evidence sentence to the original claim and the dialectical claim were separately assessed,and the top k sentences with the highest semantic similarity to each of them were selected as the evidence to either support or oppose the original claim,which obtained evidences representing different stances,and helped the fact-checking model integrate information from multiple perspectives and evaluate the veracity of the claim more accurately.Finally,the BERT-StuSE(BERT-based Stance-infused Semantic Encoding network)model was introduced to fully incorporate the semantic and stance information of the evidence through the multi-head attention mechanism and make a more comprehensive and objective judgment on the relationship between the claim and the evidence.The experimental results on the CHEF dataset show that,compared to the BERT method,the Micro F1 value and Macro F1 value of the proposed method on the test set were improved by 3.52 and 3.90 percentage points,respectively,achieving a good level of performance.The experimental results demonstrate the effectiveness of the proposed method,and the significance of considering evidence from different stances and leveraging the stance information of the evidence for enhancing fact-checking performance.

关 键 词：事实核查 自然语言处理 大语言模型 提示工程 立场感知 多头注意力机制 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]