融合局部感知Transformer模型的微积分方程求解  

Transformer model with locally aware for calculus equation solving

作　　者：卢林 朱兆旻 吴宁 LU Lin;ZHU Zhaomin;WU Ning(School of Computer and Electronic Information,Guangxi University,Nanning 530004,China;School of Electronic Information and Automation,Guilin Institute of Aerospace Technology,Guilin 541004,China;School of Electronic and Information Engineering,Beibu-gulf University,Qinzhou 535011,China)

机构地区：[1]广西大学计算机与电子信息学院,广西南宁530004 [2]桂林航天工业学院电子信息与自动化学院,广西桂林541004 [3]北部湾大学电子与信息工程学院,广西钦州535011

出　　处：《广西大学学报（自然科学版）》2024年第3期606-619,共14页Journal of Guangxi University（Natural Science Edition）

基　　金：广西自然科学基金项目(2021GXNSFAA220048);北部湾大学高层次人才启动项目(18KYQD36)。

摘　　要：针对局部信息在数学表达式的符号计算中非常重要,而Transformer模型通常存在局部缺失从而忽略字符间的语义信息的问题,提出一种将一维卷积(Conv1d)和Transformer模型相结合的符号计算模型Convld_Transformer。该模型通过在嵌入层中引入卷积网络提取局部特征信息,可有效增强局部感知。此外,还提出了一种生成一类偏微分方程数据集的算法,该算法结合特征线法和常微分方程变换,可实现对常系数一阶线性(pde1_cc)、变系数一阶线性(pde1_vc)以及满足一定条件的二阶抛物线偏微分方程(parapde2_cc)的求解。实验结果表明:Conv1d_Transformer模型在函数积分任务中相比Transformer模型精度更高,在解决pde1_cc、pde1_vc和parapde2_cc问题时准确率分别达到了96.00%、77.18%和86.18%,其性能要优于Mathematica和SymPy数学求解器。Aiming at the problem that local information is very important in the symbolic computation of mathematical expressions,while Transformer model usually has the problem of local missing and thus ignoring the semantic information between characters,a symbolic computation model Conv1d_Transformer combining one-dimensional convolution(Conv1d)and Transformer is proposed.The model can effectively enhance local aware by introducing convolutional networks into the embedding layer to extract local feature information.In addition,an algorithm for generating a class of partial differential equation datasets is proposed,which combines the method of characteristics line and the ordinary differential equation transformation.It can realize the solution of first-order linear with constant coefficients(pde1_cc),first-order linear with variable coefficients(pde1_vc),as well as second-order parabolic partial differential equations that satisfy certain conditions(parapde2_cc).The experimental results show that the Conv1d_Transformer model is more accurate compared with Transformer model in function integration task,and achieves an accuracy of 96.00%,77.18%and 86.18%in solving the pde1_cc,pde1_vc and parapde2_cc problems,respectively.It outperforms the Mathematica and SymPy mathematical solvers.

关 键 词：符号计算 一维卷积 函数积分 微分方程 特征线法 

分 类 号：TP391.2[自动化与计算机技术—计算机应用技术]