基于大语言模型的试题自动生成路径研究  

Research on the Approach to Automatic Test Item Generation Based on Large Language Models

作　　者：陈欣[1] 李蜜如 周悦琦 周同[1] 张峰[1] CHEN Xin;LI Miru;ZHOU Yueqi;ZHOU Tong;ZHANG Feng(Shandong University of Science and Technology,Qingdao 266590,China)

机构地区：[1]山东科技大学,山东青岛266590

出　　处：《中国考试》2024年第12期39-48,共10页journal of China Examinations

基　　金：2023年度山东省教育科学“十四五”规划一般课题“基于大语言模型的试题资源生成方法研究”(2023YB162)。

摘　　要：大语言模型具有强大的自然语言理解与生成能力,将大语言模型应用于试题自动生成领域,能够极大地提高我国教育考试发展的数字化水平。在具体教育场景中,尚需克服大模型普遍缺少专业知识、知识水平难以达到专业课程的教学精度、难度等挑战。为提高大模型自动生成试题的质量,本研究构建一种基于大语言模型的试题自动生成路径,并对其进行验证研究。首先是基于检索增强进行知识集成,其次是将课程知识以知识点的形式融入提示模板,最后是结合提示工程的效用,使大模型在理解课程知识的基础上执行大量试题生成任务。验证结果发现,自动生成试题的合格率为86.47%,随机抽取试题组成的测验难度为0.67,试题接受度良好。Large language models(LLMs)demonstrate strong capabilities in natural language understanding and generation.By applying LLMs to automatic test item generation,we can significantly enhance the digitalization of educational examination development in China.However,challenges persist in specific educational scenarios,such as the deficiency of specialized knowledge in LLMs and the difficulty in aligning their output with the accuracy and complexity required for specialized courses.To address these issues and improve the quality of automatic test item generation using LLMs,this study proposes a structured approach which involves three key steps.First,knowledge integration is implemented through retrieval-enhanced generation.Second,course-specific knowledge is incorporated into prompt templates in the form of key concepts.Third,prompt engineering is leveraged to enable LLMs to generate a wide range of test items based on their understanding of course content.The validation study yielded promising results,with an 86.47%pass rate for the automatically generated test items.Additionally,the difficulty level of the randomly selected test items was measured at 0.67,indicating good acceptance among users.

关 键 词：大语言模型 生成式人工智能 试题自动生成 试题质量 提示工程 

分 类 号：G405[文化科学—教育学原理]