期刊审稿专家一致性评价方法及其有效性验证  

Evaluating Consistency of Scholarly Article Reviewers

作　　者：岳名亮[1] 李富山 汤宏波[1] 吕新华[1] 马廷灿[1,2] Yue Mingliang;Li Fushan;Tang Hongbo;Lv Xinhua;Ma Tingcan(Wuhan Library,Chinese Academy of Sciences,Wuhan 430071,China;Department of Library,Information and Archives Management,School of Economics and Management,University of Chinese Academy of Sciences,Beijing 100190,China)

机构地区：[1]中国科学院武汉文献情报中心,武汉430071 [2]中国科学院大学经济与管理学院图书情报与档案管理系,北京100190

出　　处：《数据分析与知识发现》2021年第4期115-122,共8页Data Analysis and Knowledge Discovery

基　　金：中国科学院文献情报中心青年创新团队项目(项目编号:Y8KZ491);中国科学院文献情报能力建设专项(项目编号:Y9290001);国家自然科学基金项目(项目编号:71603252)的研究成果之一。

摘　　要：【目的】针对期刊论文评议,设计一种审稿专家一致性评价方法。【方法】同时考虑蕴含于评议数据中的专家知识与蕴含于发表文献的计量数据中的学术共同体知识,提出一种审稿专家一致性评价方法,基于《长江流域资源与环境》期刊评议数据与已发表论文的计量数据计算专家一致性指数,设计假设检验方法检验一致性更高的专家是否能对论文做出更为准确的评价。【结果】检验结果表明高一致性专家能更有效地区分论文的学术共同体认可度(区分度为低一致性专家的两倍以上),且该能力随时间的推移可以得到保持。【局限】本文的专家一致性指数无法替代期刊编辑进行专家选择,但可在期刊编辑选择审稿专家时为其提供客观的数据参考,以提高遴选效率与效果。【结论】研究结果表明基于历史数据计算一致性指数并辅助审稿专家遴选具备可行性。[Objective] This paper proposes a method to evaluate the consistency of scholarly journal article reviewers. [Methods] We developed a consistency index based on the knowledge from reviews and bibliometric data. Then, we conducted hypothesis test to examine whether experts with higher consistency scores make a more accurate evaluation of the paper. [Results] We found high-consistency experts could identify papers with academic community recognition, which was also maintained over time. [Limitations] The proposed consistency index could not replace journal editors in selecting experts, however, it helps to make reviewer selection efficiently and effectively. [Conclusions] It is feasible to calculate the consistency index based on historical data to select reviewers of scholarly articles.

关 键 词：论文评审 专家一致性评价 假设检验 

分 类 号：G350[文化科学—情报学]