AWeCita:Generating Answer with Appropriate and Well-grained Citations Using LLMs  

作　　者：Suifeng Zhao Tong Zhou Zhuoran Jin Hongbang Yuan Yubo Chen Kang Liu Sujian Li 

机构地区：[1]International School,Beijing University of Posts and Telecommunications,Beijing100876,China [2]The Laboratory of Cognition and Decision Intelligence for Complex Systems,Institute of Automation,Chinese Academy of Sciences,Beijing 100190,China [3]School of Artificial Intelligence,University of Chinese Academy of Sciences,Beijing 100190,China [4]School of Computer Science,Peking University,Beijing 100871,China

出　　处：《Data Intelligence》2024年第4期1134-1157,共24页数据智能(英文)

基　　金：supported by the National Natural Science Foundation of China(No.62176257);the Youth Innovation Promotion Association CAS

摘　　要：Large language models(LLMs)excel in various Natural Language Processing tasks but struggle with hallucinations,leading to potentially misleading responses.Researchers have extensively explored LLMs'citation practices.However,existing efforts often overlook the crucial aspects of the appropriateness and granularity of citation,which are vital for mitigating hallucination and enhancing interpretability.To bridge this gap and improve the quality of citations,we propose the Generating Answers with Appropriate and Well-grained Citations using LLMs task(AWeCita),with a focus on citing appropriately with a well granularity.Based on the traditional evaluation metrics of answer accuracy and citation correctness,we introduce two new evaluation metrics,citation appropriateness and citation granularity,to assess LLMs'performance on this task more comprehensively and accurately.We conduct a series of exploratory experiments on ASQA and ELI5 datasets.The experimental results show that,AWeCita outperforms traditional tasks in the metric of citation granularity,most of our methods show a certain advantage incitation appropriateness,however,the improvement towards well-grained citation affects the quote-level citation correctness.

关 键 词：Question answering Large language models CITATION Citation appropriateness Citation granularity 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程] TP391.1[自动化与计算机技术—控制科学与工程]