An evaluation framework for software crowdsourcing  

作　　者：Wenjun WU Wei-Tek TSAI Wei LI 

机构地区：[1]State Key Laboratory of Software Development Environment, Beihang University, Beijing 100191, China [2]School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, Tempe, AZ85281, USA [3]Department of Computer Science and Technology, INLIST, Tsinghua University, Beijing 100084, China

出　　处：《Frontiers of Computer Science》2013年第5期694-709,共16页中国计算机科学前沿（英文版）

摘　　要：Recently software crowdsourcing has become an emerging area of software engineering. Few papers have pre- sented a systematic analysis on the practices of software crowdsourcing. This paper first presents an evaluation frame- work to evaluate software crowdsourcing projects with re- spect to software quality, costs, diversity of solutions, and competition nature in crowdsourcing. Specifically, competi- tions are evaluated by the min-max relationship from game theory among participants where one party tries to minimize an objective function while the other party tries to maximize the same objective function. The paper then defines a game theory model to analyze the primary factors in these min- max competition rules that affect the nature of participation as well as the software quality. Finally, using the proposed eval- uation framework, this paper illustrates two crowdsourcing processes, Harvard-TopCoder and AppStori. The framework demonstrates the sharp contrasts between both crowdsourc- ing processes as participants will have drastic behaviors in engaging these two projects.Recently software crowdsourcing has become an emerging area of software engineering. Few papers have pre- sented a systematic analysis on the practices of software crowdsourcing. This paper first presents an evaluation frame- work to evaluate software crowdsourcing projects with re- spect to software quality, costs, diversity of solutions, and competition nature in crowdsourcing. Specifically, competi- tions are evaluated by the min-max relationship from game theory among participants where one party tries to minimize an objective function while the other party tries to maximize the same objective function. The paper then defines a game theory model to analyze the primary factors in these min- max competition rules that affect the nature of participation as well as the software quality. Finally, using the proposed eval- uation framework, this paper illustrates two crowdsourcing processes, Harvard-TopCoder and AppStori. The framework demonstrates the sharp contrasts between both crowdsourc- ing processes as participants will have drastic behaviors in engaging these two projects.

关 键 词：crowdsourcing software engineering competi-tion rules game theory 

分 类 号：TP311.5[自动化与计算机技术—计算机软件与理论] TU375.4[自动化与计算机技术—计算机科学与技术]