Identification of ChatGPT answers and physician answers in the online medical community  

作　　者：Shengli Deng Haowei Wang 

机构地区：[1]School of Information Management,Wuhan University,Wuhan,China

出　　处：《Data Science and Informetrics》2023年第3期18-31,共14页数据科学与信息计量学（英文）

基　　金：supported in part by National Natural Science Foundation,PR China(Grant No.72374158)。

摘　　要：ChatGPT changes the way of knowledge production and information space structure of human society.In the healthcare industry,ChatGPT's powerful question-and-answer capability will drive its application in automated question answering in online healthcare communities.However,because ChatGPT answers are limited by factors such as the quality of data sets,their authority and accuracy cannot be guaranteed,and they are prone to misdiagnosis and damage to life and health.Therefore,the identification of ChatGPT answers in online medical communities with physician answers is crucial.In this paper,we collected medical question-answering data generated by the Haodafu platform and ChatGPT,respectively,constructed feature vectors from semantic features,syntactic features,and the fusion of both,and combined different feature vectors with XGBoost models to construct BERT-XGBoost,POS-XGBoost and Merge-XGBoost models for identifying ChatGPT answers and physician answers in online medical communities.The three models achieved accuracy rates of 0.960,0.968,and 0.986,respectively.The difference in performance between the three models reflects the degrees of variation in different features of ChatGPT answers versus physician answers.The results indicate that the differences between ChatGPT and physicians in syntactic features,i.e.,linguistic expression habits,are greater than their differences in semantic features,i.e.,specific content suggestions.

关 键 词：ChatGPT Online medical community Text classification 

分 类 号：G350[文化科学—情报学]