An analysis of correctness for API recommendation:are the unmatched results useless?  

作　　者：Xianglong KONG Weina HAN Li LIAO Bixin LI 

机构地区：[1]School of Computer Science and Engineering,Southeast University,Nanjing 211189,China

出　　处：《Science China(Information Sciences)》2020年第9期39-53,共15页中国科学（信息科学）（英文版）

基　　金：National Key R&D Program of China(Grant No.2018YFB100-3900);in part by National Natural Science Foundation of China(Grant Nos.61402103,61572126,61872078);in part by Open Research Fund of Key Laboratory of Safety-Critical Software Fund(Nanjing University of Aeronautics and Astronautics)(Grant No.NJ2019006);in part by Key Laboratory of Computer Network and Information Integration of the Ministry of Education of China(Grant No.93K-9)。

摘　　要：API recommendation is a promising approach which is widely used during software development.However,the evaluation of API recommendation is not explored with sufficient rigor.The current evaluation of API recommendation mainly focuses on correctness,the measurement is conducted by matching recommended results with ground-truth results.In most cases,there is only one set of ground-truth APIs for each recommendation attempt,but the object code can be implemented in dozens of ways.The neglect of code diversity results in a possible defect in the evaluation.To address the problem,we invite15 developers to analyze the unmatched results in a user study.The online evaluation confirms that some unmatched APIs can also benefit to programming due to the functional correlation with ground-truth APIs.Then we measure the API functional correlation based on the relationships extracted from API knowledge graph,API method name,and API documentation.Furthermore,we propose an approach to improve the measurement of correctness based on API functional correlation.Our measurement is evaluated on a dataset of 6141 requirements and historical code fragments from related commits.The results show that 28.2%of unmatched APIs can contribute to correctness in our experiments.

关 键 词：API recommendation onsite programming CORRECTNESS evaluation of recommendation 

分 类 号：TP311.52[自动化与计算机技术—计算机软件与理论]