基于开源代码大语言模型提示的学生代码修复  

Prompting open-source code large language models for student program repair

作　　者：陈郅睿 陆雪松 CHEN Zhirui;LU Xuesong(School of Data Science and Engineering,East China Normal University,Shanghai 200062,China)

机构地区：[1]华东师范大学数据科学与工程学院,上海200062

出　　处：《华东师范大学学报（自然科学版）》2024年第5期93-103,共11页Journal of East China Normal University(Natural Science)

基　　金：国家自然科学基金(62277017)。

摘　　要：随着机器学习技术的进步,旨在学习人类修复错误代码模式的自动程序修复技术可以辅助学生修复错误代码,提高学生的自主学习效率.在过去,自动程序修复模型或是基于人工设计的符号规则,或是基于数据驱动的方法.随着具有强大自然语言理解能力和代码生成能力的大语言模型的出现,一些研究尝试使用提示工程进行自动程序修复.然而,现有研究主要评估诸如Codex和GPT-4这样的商用模型,一方面大规模使用的成本较高,另一方面在教育场景下存在数据隐私隐患.此外,这些研究大多使用简单的提示形式来评估模型修复程序的能力,且缺乏对结果的深入分析.为弥补上述工作的不足,通过提示工程评估了两个代表性的开源代码大语言模型,测试了不同的提示方法,例如思维链和少样本学习,并对结果进行了深入分析,最后提出了一些将大语言模型和编程教育场景结合的建议.Advancements in machine-learning technology has enabled automated program-repair techniques that learn human patterns of erroneous-code fixing,thereby assisting students in debugging and enhancing their self-directed learning efficiency.Automatic program-repair models are typically based on either manually designed symbolic rules or data-driven methods.Owing the availability of large language models that possess excellent natural-language understanding and code-generation capabilities,researchers have attempted to use prompt engineering for automatic program repair.However,existing studies primarily evaluate commercial models such as Codex and GPT-4,which may incur high costs for large-scale adoption and cause data-privacy issues in educational scenarios.Furthermore,these studies typically employ simple prompt forms to assess the program-repair capabilities of large language models,whereas the results are not analyzed comprehensively.Hence,we evaluate two representative open-source code large language models with excellent code-generation capability using prompt engineering.We evaluate different prompting methods,such as chain-of-thought and few-shot learning,and analyze the results comprehensively.Finally,we provide suggestions for integrating large language models into programming educational scenarios.

关 键 词：自动程序修复 大语言模型 提示工程 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]