古农文语义检索模型构建及其应用研究  

Construction and Application of Semantic Retrieval Model for Ancient Agricultural Literature

作　　者：刘楠竹 崔运鹏[1,2] 王末 LIU Nanzhu;CUI Yunpeng;WANG Mo(Institute of Agricultural Information,Chinese Academy of Agricultural Sciences,Beijing 100081;Key Laboratory of Agricultural Big Data,Ministry of Agriculture and Rural Affairs,Beijing 100081)

机构地区：[1]中国农业科学院农业信息研究所,北京100081 [2]农业农村部农业大数据重点实验室,北京100081

出　　处：《农业图书情报学报》2023年第7期52-62,共11页Journal of Library and Information Science in Agriculture

基　　金：国家社会科学基金重大项目“中国古农书的搜集、整理与研究”(21&ZD332)。

摘　　要：[目的/意义]构建能实现以白话文作为查询,系统自动返回与输入最相关的古农文段落的语义检索模型,为学者提供更加便利的古代农业知识检索方式和古代农业知识溯源方式。[方法/过程]使用基于四库全书作为训练语料的SikuBERT作为基础模型,基于对比学习的方法,使用自建的古农文数据集对模型进行继续训练,得到能够支持使用白话文作为查询,返回与查询语义最相似的古农文段落的语义检索模型。[结果/结论]古农文语义检索模型的Spearman系数在测试集上的表现能够达到86.51%,较基线模型在测试集上的表现83.69%有一定程度的提升,在自建的古农文检索测试集上的召回情况(recall@k)较基线模型有一定程度提升,模型在古农文上能够有比较好的检索效果。但受限于古农文训练语料规模,模型的训练效果还有很大提升空间。[Purpose/Significance]The ancient Chinese agricultural books are the main carrier of traditional agricultural experience,and represent the productivity and the essence of agricultural history in China.The value of agricultural knowledge in them has not disappeared with the progress of the times,and still has practical guidance for the problems that arise in modern agriculture.However,the ancient Chinese agricultural books are written in ancient Chinese,which are obscure and without punctuation,making them difficult to use.Semantic retrieval is a retrieval method that automatically queries and extracts relevant information from information sources at the semantic level.It can accurately capture the true intention behind user problems and conduct searches based on it,and thereby it is capable of returning more accurate and the most consistent results to users.However,currently most relevant research only focuses on major languages,and there is insufficient research on sentence embedding in ancient Chinese prose.In order to fill the gap in the field and provide scholars with more convenient methods for retrieving ancient agricultural knowledge and tracing ancient agricultural knowledge,this study is based on comparative learning methods to construct a semantic retrieval model that can automatically return the most relevant ancient agricultural paragraph with input,using vernacular Chinese as the query.[Method/Process]SikuBERT,which is based on Siku Quanshu as the training corpus,is used as the basic model.Based on the method of comparative learning,the model is continued to be trained using the self-built ancient agricultural dataset,and a semantic retrieval model that can support the use of vernacular as a query and return the ancient agricultural paragraphs most similar to the query semantics is obtained.[Results/Conclusions]The Spearman coefficient of the ancient agricultural text semantic retrieval model can achieve 86.51%performance on the test set,which is a certain degree of improvement compared to the baseli

关 键 词：古农文 语义检索 对比学习 模型构建 深度学习 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]