任务认知层次对用户与生成式人工智能交互行为与系统评价的影响研究  

Research on the Impact of Task Cognitive Level on User-Generative Artificial Intelligence Interaction Behavior and System Evaluation

作　　者：李雨佳 冉晓雅 刘畅[1] Li Yujia;Ran Xiaoya;Liu Chang(Department of Information Management,Peking University,Beijing,100871)

机构地区：[1]北京大学信息管理系,北京100871

出　　处：《情报资料工作》2025年第2期51-60,共10页Information and Documentation Services

摘　　要：[目的/意义]生成式人工智能的迅速发展推动用户交互行为与思维模式的变革。文章关注不同层次认知复杂度的任务下,用户与生成式人工智能的交互行为及其对系统表现评价的差异。[方法/过程]采用用户实验法,通过Kruskal-Wallis检验探究任务认知层次对用户交互行为及对系统评价的差异,通过主题分析法对用户评价生成式人工智能的新指标进行归纳。[结果/结论]在交互行为方面,用户在不同认知层次的任务完成中信息使用的时长几乎是不变的,变化的一直是信息获取的时长;评估创造类任务花费时间最多,提问与问答轮次最多,平均回答长度最短;应用分析类任务的回答中,总复制比低于记忆理解与评估创造类任务。在对系统的评价方面,评估创造类任务总体表现最差。在新评价指标方面,文章归纳出完整程度、一致程度等6大类、17小类新指标。文章在理论层面补充了以用户为中心的生成式人工智能评价指标,在实践层面有助于相关智能系统了解用户在不同任务类型下的行为特质,更有针对性地提供信息服务。[Purpose/significance]The rapid development of generative artificial intelligence has propelled changes in user interaction behaviors and cognitive patterns.This study focuses on the differences in users'interactive behaviors with generative artificial intelligence and their evaluations of system performance under tasks with different levels of cognitive complexity.[Method/process]This study employs the user experimentation method and uses the Kruskal-Wallis test to explore the impact of task cognitive levels on user interaction behaviors and system evaluations.It also employs thematic analysis to induce new metrics for user evaluations of generative AI.[Result/conclusion]In terms of interaction behavior,during the completion of tasks at different cognitive levels,the duration of information usage by us⁃ers remains almost unchanged,while it is always the duration of information acquisition that varies.Tasks of evaluation and creation take the most time,involve the most question-and-answer rounds,and have the shortest average response length.In the answers to application and analysis tasks,the total copy ratio is lower than that of memory,understanding,evaluation and creation tasks.In terms of system evaluation,tasks of evaluation and creation receive the poorest overall performance.Regarding new evaluation metrics,the study has induced 6 major categories and 17 minor categories of new metrics.This paper supplements user-centered evaluation metrics for generative AI at the theoretical level and helps intelligent systems understand user behavioral characteristics under different task types at the practical level,pro⁃viding more targeted information services.

关 键 词：生成式人工智能 认知层次 交互行为 系统评价 

分 类 号：G250[文化科学—图书馆学] G250.2