应用大语言模型解答儿童哮喘问题的效果研究  

Efficiency of large language models in answering questions about childhood asthma

作　　者：韩伟鹏 尹小梅 王健[2,3] 李学军[2,3] 杨吉江[1] HAN Weipeng;YIN Xiaomei;WANG Jian;LI Xuejun;YANG Jijiang(Digital Medical Health Engineering Research Center of Department of Automation of Tsinghua University,Beijing,100084;Department of Pediatrics,Beijing Tsinghua Changgeng Hospital Affiliated to Tsinghua University,Beijing,102218;School of Clinical Medicine of Tsinghua University,Beijing,100084)

机构地区：[1]清华大学自动化系数字医疗健康工程研究中心,北京100084 [2]清华大学附属北京清华长庚医院儿科,北京102218 [3]清华大学临床医学院,北京100084

出　　处：《实用临床医药杂志》2024年第11期6-11,17,共7页Journal of Clinical Medicine in Practice

基　　金：北京市卫生与健康科技成果和适宜技术推广项目(TG-2017-41)。

摘　　要：目的评估大语言模型解答儿童哮喘问题的表现,全面了解其提供儿童健康相关信息的质量,并识别其局限以促进模型的改进。方法制订出60个儿童哮喘相关的常见问题,分别向2种在国内向公众开放使用的文心一言、智谱清言大语言模型提问。由3名儿科哮喘专业医师采用盲法评估大语言模型的回答质量。结果在准确性、理解力、可靠性和逻辑性维度方面,文心一言得分较高;在安全性维度方面,智谱清言的得分较高。对比5个不同的维度的得分发现,大语言模型在理解力、可靠性、逻辑性方面得分较高,而在准确性与安全性方面相对不足。结论大语言模型在儿童哮喘患者教育中的应用能够为儿童哮喘患者及其家长提供有益的参考。然而,当前大语言模型技术在准确性、安全性等方面仍存在一定的局限性,需要进一步改进和优化。Objective To evaluate the performance of large language models in answering questions about childhood asthma,comprehensively understand the quality of their provision of information on children′s health,and identify their limitations to facilitate model improvement.Methods Sixty common questions related to childhood asthma were formulated and put to two large language models known as Wenxin Yiyan and Zhipu Qingyan,which were publicly available in China.Three pediatric asthma specialists assessed the quality of the large language models′responses by using a blind method.Results Wenxin Yiyan scored higher in terms of accuracy,understanding,reliability,and logicality;Zhipu Qingyan scored higher in term of safety.Comparing the scores of the five different dimensions,it was found that large language models scored higher in terms of understanding,reliability and logicality,but relatively insufficient in terms of accuracy and safety.Conclusion Application of large language models in the education of children with asthma can provide useful references for asthma children and their parents.However,the current large language model technology still has certain limitations in terms of accuracy and safety,which requires further improvement and optimization.

关 键 词：大语言模型 儿童哮喘 医学人工智能 智能助理 文心一言 智谱清言 

分 类 号：R562.2[医药卫生—呼吸系统] R494[医药卫生—内科学] R319[医药卫生—临床医学]