Empirical Research in Software Engineering -- A Literature Survey  

作　　者：Li Zhang Jia-Hao Tian Jing Jiang Yi-Jun Liu Meng-Yuan Pu Tao Yue 

机构地区：[1]State Key Laboratory of Software Development Environment, Beihang University, Beijing 100191, China [2]College of Software, Beihang University, Beijing 100191, China [3]Simula Research Laboratory, Martin Lingesvei 25, 1364 Fornebu, Norway

出　　处：《Journal of Computer Science & Technology》2018年第5期876-899,共24页计算机科学技术学报（英文版）

基　　金：This work was supported by the National Natural Science Foundation of China under Grant Nos. 61672078 and 61732019, and the National Key Research and Development Program of China under Grant No. 2018YFB1004202.

摘　　要：Empirical research is playing a significant role in software engineering （SE）, and it has been applied to evaluate software artifacts and technologies. There have been a great number of empirical research articles published recently. There is also a large research community in empirical software engineering （ESE）. In this paper, we identify both the overall landscape and detailed implementations of ESE, and investigate frequently applied empirical methods, targeted research purposes, used data sources, and applied data processing approaches and tools in ESE. The aim is to identify new trends and obtain interesting observations of empirical software engineering across different sub-fields of software engineering. We conduct a mapping study on 538 selected articles from January 2013 to November 2017, with four research questions. We observe that the trend of applying empirical methods in software engineering is continuously increasing and the most commonly applied methods are experiment, case study and survey. Moreover, open source projects are the most frequently used data sources. We also observe that most of researchers have paid attention to the validity and the possibility to replicate their studies. These observations are carefully analyzed and presented as carefully designed diagrams. We also reveal shortcomings and demanded knowledge/strategies in ESE and propose recommendations for researchers.Empirical research is playing a significant role in software engineering （SE）, and it has been applied to evaluate software artifacts and technologies. There have been a great number of empirical research articles published recently. There is also a large research community in empirical software engineering （ESE）. In this paper, we identify both the overall landscape and detailed implementations of ESE, and investigate frequently applied empirical methods, targeted research purposes, used data sources, and applied data processing approaches and tools in ESE. The aim is to identify new trends and obtain interesting observations of empirical software engineering across different sub-fields of software engineering. We conduct a mapping study on 538 selected articles from January 2013 to November 2017, with four research questions. We observe that the trend of applying empirical methods in software engineering is continuously increasing and the most commonly applied methods are experiment, case study and survey. Moreover, open source projects are the most frequently used data sources. We also observe that most of researchers have paid attention to the validity and the possibility to replicate their studies. These observations are carefully analyzed and presented as carefully designed diagrams. We also reveal shortcomings and demanded knowledge/strategies in ESE and propose recommendations for researchers.

关 键 词：empirical software engineering empirical method systematic mapping study 

分 类 号：TP311.5[自动化与计算机技术—计算机软件与理论] I206.7[自动化与计算机技术—计算机科学与技术]