Enhancing Orthopedic Knowledge Assessments:The Performance of Specialized Generative Language Model Optimization  

作　　者：Hong ZHOU Hong-lin WANG Yu-yu DUAN Zi-neng YAN Rui LUO Xiang-xin LV Yi XIE Jia-yao ZHANG Jia-ming YANG Ming-di XUE Ying FANG Lin LU Peng-ran LIU Zhe-wei YE 

机构地区：[1]Department of Orthopedics Surgery,Union Hospital,Tongji Medical College,Huazhong University of Science and Technology,Wuhan,430022,China [2]Laboratory of Intelligent Medicine,Union Hospital,Tongji Medical College,Huazhong University of Science and Technology,Wuhan,430022,China [3]College of Chinese Medicine,Hubei University of Chinese Medicine,Wuhan,433065,China [4]Department of Orthopedics,Renmin Hospital of Wuhan University,Wuhan,433060,China

出　　处：《Current Medical Science》2024年第5期1001-1005,共5页当代医学科学（英文）

基　　金：supported by the National Natural Science Foundation of China(Grant No.81974355 and No.82172524).

摘　　要：Objective This study aimed to evaluate and compare the effectiveness of knowledge base-optimized and unoptimized large language models(LLMs)in the field of orthopedics to explore optimization strategies for the application of LLMs in specific fields.Methods This research constructed a specialized knowledge base using clinical guidelines from the American Academy of Orthopaedic Surgeons(AAOS)and authoritative orthopedic publications.A total of 30 orthopedic-related questions covering aspects such as anatomical knowledge,disease diagnosis,fracture classification,treatment options,and surgical techniques were input into both the knowledge base-optimized and unoptimized versions of the GPT-4,ChatGLM,and Spark LLM,with their generated responses recorded.The overall quality,accuracy,and comprehensiveness of these responses were evaluated by 3 experienced orthopedic surgeons.Results Compared with their unoptimized LLMs,the optimized version of GPT-4 showed improvements of 15.3%in overall quality,12.5%in accuracy,and 12.8%in comprehensiveness;ChatGLM showed improvements of 24.8%,16.1%,and 19.6%,respectively;and Spark LLM showed improvements of 6.5%,14.5%,and 24.7%,respectively.Conclusion The optimization of knowledge bases significantly enhances the quality,accuracy,and comprehensiveness of the responses provided by the 3 models in the orthopedic field.Therefore,knowledge base optimization is an effective method for improving the performance of LLMs in specific fields.

关 键 词：artificial intelligence large language models generative articial intelligence ORTHOPEDICS 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术] R68[自动化与计算机技术—计算机科学与技术]