大语言模型驱动的开源情报认知  

Large language model-driven open-source intelligence cognition

作　　者：张华平[1] 李春锦 魏顺平 耿国桐 李伟伟 李玉岗[1] ZHANG Huaping;LI Chunjin;WEI Shunping;GENG Guotong;LI Weiwei;LI Yugang(Beijing Institute of Technology,Beijing 100081,China;Minzu University of China,Beijing 100081,China;Military Science Information Research Center,Academy of Military Science,Beijing 100011,China;National Innovation Institute of Defense Technology,Academy of Military Science,Beijing 100071,China)

机构地区：[1]北京理工大学,北京100081 [2]中央民族大学,北京100081 [3]军事科学院军事科学信息研究中心,北京100011 [4]军事科学院国防科技创新研究院,北京100071

出　　处：《国防科技》2024年第3期51-57,共7页National Defense Technology

基　　金：北京市自然科学基金项目(4212026)。

摘　　要：随着开源情报在军事领域的广泛应用,对相关情报的认知和分析需求日益增长。然而,当前研究人员所使用的大语言模型存在严重的幻觉现象,导致其生成的信息不可靠,无法直接用于军事开源情报的认知任务。为了解决这一问题,通过网上收集构建一个包含约10万条对话记录的军事开源情报数据集;利用LLaMA-13B模型作为基座,通过微调训练得到一个新的模型——ChatBIT,专门针对军事领域的对话和问答任务进行优化。对比分析ChatBIT模型与Vicuna-13B模型在军事知识问答方面的能力,通过一系列标准化的指标评估,包括Bleu值、Rouge-1、Rouge-2和Rouge-L,可知ChatBIT在所有指标上均优于Vicuna-13B。具体来说,相比Vicuna-13B,ChatBIT的Bleu值高2.3909,Rouge-1值高3.2079,Rouge-2值高2.2562,Rouge-L值高1.5939。结果表明,ChatBIT模型在处理军事领域的对话和问答任务时,能够提供更准确、更可靠的信息。With the extensive application of open-source intelligence in the military field,the demand for cognition and analysis of relevant intelligence is growing.However,the large language models currently used by researchers are prone to severe hallucination,rendering the information generated unreliable and unsuitable to direct utilization for the cognition of open-source military intelligence.To address this problem,the present study collected approximately 100,000 dialogue records online and constructed an open-source military intelligence dataset.Subsequently,a new model,ChatBIT,which is specifically optimized for dialogue and question answering tasks in the military field,was obtained by fine-tuning and training the LLaMA-13B base question answering model.This study further compared the military knowledge question answering capabilities of the ChatBIT model with those of the Vicuna-13B model.ChatBIT was found to outperform Vicuna-13B in a series of standardized evaluation metrics including the BLEU score,ROUGE-1,ROUGE-2,and ROUGE-L.Specifically,ChatBIT’s BLEU score was 2.3909 higher than that of Vicuna-13B.Furthermore,ChatBIT’s ROUGE-1,ROUGE-2,and ROUGE-L scores were respectively 3.2079,2.2562,and 1.5939 points higher than those of Vicuna-13B.These results indicate that the ChatBIT model provides more accurate and reliable information when dealing with military dialogue and question answering tasks.

关 键 词：大语言模型 ChatBIT 开源情报 人工智能 

分 类 号：N99[文化科学—情报学]