政务智能问答系统评价指标体系构建与测评问题编制  

Developing an Evaluation Index System and Its Testing Questions for Government Intelligent Q&A Systems

作　　者：王芳[1,2] 魏中瀚[3] 连芷萱 WANG Fang;WEI Zhonghan;LIAN Zhixuan(Department of Information Resources Management,Business School,Nankai University,Tianjin,300071;Center for Network Society Governance,Nankai University,Tianjin,300072;Information Center of Ministry of Science and Technology,Beijing,100862)

机构地区：[1]南开大学商学院信息资源管理系,天津300071 [2]南开大学网络社会治理研究中心,天津300072 [3]科学技术部信息中心,北京100862

出　　处：《图书情报知识》2023年第6期98-111,共14页Documentation,Information & Knowledge

基　　金：国家社会科学基金重大项目“基于数据共享与知识复用的数字政府智能化治理研究”(20ZDA039)的研究成果之一。

摘　　要：[目的/意义]智能问答系统正成为提供公共服务的重要设施。通过评价可以促进政务智能问答系统建设,降低用户行政负担。[研究设计/方法]基于文献调研、理论分析和案例研究,构建政务智能问答系统评价指标体系,并运用专家调查和AHP法确定了指标权重;根据政务热点咨询问题编制了与问题解决质量维度配套的测试问题集;对我国30个省级政府网站智能问答系统进行了实际测评。[结论/发现]政务智能问答系统评价指标体系包含问题解决质量、服务交互质量、基础建设质量3个维度以及高频问题解答率、同义问题解答率、交互次数等18个细化指标;测试题集包括高频、同义、错误、省略、英文五大类问题。实际评价证明了指标体系和测试题集的可用性和有效性。[创新/价值]从降低行政负担视角出发构建政务智能问答系统评价指标体系,运用文本挖掘技术构建测评问题集,并通过实际测评证明其可用性。[Purpose/Significance]Intelligent question and answer(Q&A)system is becoming an important facility for public services delivery.The purpose of this study is to promote the construction of government intelligent Q&A systems by evaluation,and reduce the administrative burden of citizens.[Design/Methodology]Firstly,based on literature research,theoretical analysis and case study,an evaluation indicator system for government intelligent Q&A system was constructed,and the indicator weight was determined by expert survey and AHP method.Secondly,a testing question set for problem solving quality was developed based on the hot consultation issues.Finally,30 provincial government Q&A systems in China were evaluated with the indicator system and the matched questions.[Findings/Conclusion]The evaluation index system of government intelligent question and answering system includes three dimensions,namely,the quality of problem solving,the quality of service interaction,and the quality of infrastructure construction,as well as 18 refined indicators,such as the rate of high-frequency question answering,the rate of synonymous question answering,and the number of interactions.The testing question set includes five categories of questions:high-frequency,synonymous,error,ellipsis,and English ones.The practical evaluation proves the usability and effectiveness of the indicator system and testing questions set.[Originality/Value]This study constructs an evaluation index system for government intelligent Q&A systems from the perspective of reducing administrative burden,compiles testing question sets using text mining technique,and proves their usability through an actual evaluation.

关 键 词：问答系统 政府网站 语义理解能力 智能问答 测评问题 智能客服 

分 类 号：G203[文化科学—传播学]