LLaMA-LoRA Neural Prompt Engineering:A Deep Tuning Framework for Automatically Generating Chinese Text Logical Reasoning Thinking Chains  

作　　者：Songlin Chen Weicheng Wang Xiaoliang Chen Peng lu Zaiyan Yang Yajun Du 

机构地区：[1]School of Computer and Software Engineering,Xihua University,chengdu 610039,P.R.china [2]Department of Computer Science and Operations Research,University of Montreal,Montreal,QC H3C3J7,Canada [3]College of Artificial intelligence,Beijing University of Posts and Telecommunications,Beijing,100876,China

出　　处：《Data Intelligence》2024年第2期375-408,共34页数据智能（英文）

基　　金：supported by the the Science and Technology Program of Sichuan Province(Grant no.2023YFS0424);the"Open bidding for selecting the best candidates"Science and Technology Project of Chengdu(Grant no.2023-JB00-00020-GX);the National Natural Science Foundation(Grant nos.61902324,11426179,and 61872298).

摘　　要：The exption of Chinese natural language processing(NLP)has stimulated research in the broader NLP domain.However,existing large language models have limitations in comprehending and reasoning in Chinese.This paper addresses these limitations by enhancing Chinese language models comprehension and reasoning capabilities while minimizing resource requirements.We propose LLaMA-LoRA,a neural prompt engineering framework that builds upon the LLaMA-13B model and incorporates the Low-Rank Adaptation(LoRA)of Large Language Models technique for refinement.Chain-of-Thought(CoT)are crucial for generating intermediate reasoning chains in language models,but their effectiveness can be limited by isolated language patterns.Erroneous reasoning resulting from conventional prompts negatively impacts model performance.Automatic prompts are introduced to encourage reasoning chain generation and accurate answer inference.Training the model with an extensive corpus of Chinese CoT data enhances its comprehension and reasoning abilities.The LLaMA-LoRA model demonstrates exceptional performance across numerous Chinese language tasks,surpassing benchmark performance achieved by related language models such as GPT-3.5,Chat-GLM,and OpenAssistant,delivering accurate,comprehensive,and professional answers.The availability of our open-source model code facilitates further research in the field of Chinese text logical reasoning thinking chains.

关 键 词：Chinese natural language processing Neural prompt engineering Large language models Low-Rank adaptation Chain-of-thought 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]