Quality assessment in competition-based software crowdsourcing  

作　　者：Zhenghui HU Wenjun WU Jie LUO Xin WANG Boshu LI 

机构地区：[1]State Key Laboratory of Software Development Environment,Beihang University,Beijing 100191,China

出　　处：《Frontiers of Computer Science》2020年第6期17-30,共14页中国计算机科学前沿（英文版）

基　　金：supported by grants from State KeyLaboratory of Software Development Environment of BUAA of China(SKLSDE-2018ZX-03);NSFC(Grant No.61532004).

摘　　要：Quality assessment is a critical component incrowdsourcing-based software engineering(CBSE)as soft-ware products arc developed by the crowd with unknownor varied skills and motivations.In this paper.we proposea novel metric called the project score to measure the perfor-mance of projects and the quality of products for compctition-based software crowdsourcing devclopment(CBSCD)activ-ities.To the best of our knowledge,this is the first work to deal with the quality issue of CBSE in the perspective ofprojccts instead of contcsts.In particular,we develop a hi-crarchical quality evaluation framework for CBSCD projects and come up with two metric aggregatibn modcls for projectscores.The first model is a modified squale model that canlocate the sofiware modules of poor quality,and the secondone is a clustering-based aggregation model,which takes dif-ferent impacts of phases into account.To test the effective-ness of the proposed metrics.we conduct an empirical studyon TopCoder,which is a famous CBSCD platform.Resultsshow that the proposed project score is a strong indicator ofthe performance and product quality of CBSCD projects.wealso find that the clustering-based aggregation model outpcr-forms the Squale one by increasing the percentage of the per-formance evaluation criterion of aggregation models by anadditional 29%.Our approach to quality assessment for CB-sCD projects could potentially facilitate software managersto assess the overall quality of a crowdsourced project con-sisting of programming contests.

关 键 词：crowdsourcing software engineering.productquality.competition evaluation framework metric aggregation 

分 类 号：TP31[自动化与计算机技术—计算机软件与理论]