大型预训练语言模型在网络健康信息鉴别中的应用探讨  

Application of Large-scale Pre-Training Language Model in Network Health Information Identification

作　　者：王超 孔祥辉 WANG Chao;KONG Xianghui(Library of Liaoning University of Technology,Jinzhou 121000;Library of Jinzhou Medical University,Jinzhou 121000)

机构地区：[1]辽宁工业大学图书馆,锦州121000 [2]锦州医科大学图书馆,锦州121000

出　　处：《农业图书情报学报》2023年第6期51-59,共9页Journal of Library and Information Science in Agriculture

基　　金：2022年度辽宁省社会科学规划基金青年项目“可重复性危机视域下医学院校图书馆数据素养教育研究”(L22CTQ004)。

摘　　要：[目的/意义]探讨ChatGPT等大规模预训练语言模型在网络健康信息识别中的应用效果,为人工智能在健康信息领域的应用提供参考。[方法/过程]以国内某权威辟谣平台与健康相关的信息为研究对象,使用“ChatGPT”和“讯飞星火”对其真实性进行鉴定,对其性能进行评估,并将鉴定结果与医学专家或权威机构的鉴定结果进行比较。[结果/结论]ChatGPT和讯飞星火的鉴别准确率分别为93.9%和92.9%,F1值分别为0.951和0.946,应用效果良好。两者生成的解释文本内容比较详细,语言比较流畅,文本长度和语义相似度与专家文本高度接近,但对个别信息的解释仍存在科学依据不够详细、逻辑错误等问题。实验结果表明,大规模预训练语言模型在辅助网络健康信息识别任务方面具有一定的优势,但仍需要人工干预以保证结果的准确性和可靠性。[Purpose/Significance]Taking the popular"chat robot"ChatGPT and the recently launched similar product"iFLYTEK Spark"as the research object,this paper explores their applications in the identification of online health information,and discusses their advantages and disadvantages,in order to provide reference for the large-scale pre-training language model in the field of health information identification.Based on the review of relevant literature on online health information authentication,deep learning models have been widely applied in the task of online health information authentication in recent years.With the rapid development of large pre-training language models such as ChatGPT,it is a novel idea to explore their discriminating ability in online health information.[Method/Process]Researchers selected health-related information from the most authoritative rumor-refuting websites in China,used"ChatGPT"and"iFLYTEK Spark"to verify the authenticity of the online health information,evaluated their performance,and compared their identification results with the expert identification results.The identification accuracy of ChatGPT and iFLYTEK Spark language model was 93.9%and 92.9%,respectively,and the F1 value was 0.951 and 0.946,respectively,which had a good application effect.The generated explanatory texts were more detailed and the language was relatively smooth.In terms of the length and dispersion of the explanatory text,ChatGPT is closer to that of medical experts,while iFLYTEK Spark's explanatory text is relatively long and less discrete.In terms of semantic similarity,ChatGPT and iFLYTEK Spark were almost equal in performance,and their understanding of health information was close to that of human experts to some extent.Through the analysis of typical samples,it can be seen that an AI large model cannot accurately identify news or emergency information for the time being,and the understanding of individual health propositions with complex semantics will occasionally be biased.[Results/Conclusions]The experim

关 键 词：人工智能 健康信息 鉴别 ChatGPT 

分 类 号：G252[文化科学—图书馆学]