Evaluating grant proposals:lessons from using metrics as screening device  

作　　者：Katerina Guba Alexey Zheleznov Elena Chechik 

机构地区：[1]Center for Institutional Analysis of Science and Education,European University at Saint Petersburg,Gagarinskaya 6/1A Saint Petersburg 191178,Russian Federation

出　　处：《Journal of Data and Information Science》2023年第2期66-92,共27页数据与情报科学学报（英文版）

基　　金：This work is supported by Russian Science Foundation(Grant No.21-78-10102).

摘　　要：Purpose:This study examines the effects of using publication-based metrics for the initial screening in the application process for a project leader.The key questions are whether formal policy affects the allocation of funds to researchers with a better publication record and how the previous academic performance of principal investigators is related to future project results.Design/methodology/approach:We compared two competitions,before and after the policy raised the publication threshold for the principal investigators.We analyzed 9,167 papers published by 332 winners in physics and the social sciences and humanities(SSH),and 11,253 publications resulting from each funded project.Findings:We found that among physicists,even in the first period,grants tended to be allocated to prolific authors publishing in high-quality journals.In contrast,the SSH project grantees had been less prolific in publishing internationally in both periods;however,in the second period,the selection of grant recipients yielded better results regarding awarding grants to more productive authors in terms of the quantity and quality of publications.There was no evidence that this better selection of grant recipients resulted in better publication records during grant realization.Originality:This study contributes to the discussion of formal policies that rely on metrics for the evaluation of grant proposals.The Russian case shows that such policy may have a profound effect on changing the supply side of applicants,especially in disciplines that are less suitable for metric-based evaluations.In spite of the criticism given to metrics,they might be a useful additional instrument in academic systems where professional expertise is corrupted and prevents allocation of funds to prolific researchers.

关 键 词：Grant funding Research evaluation Publication-based metrics Funding allocation 

分 类 号：G301[文化科学] G254