Optimization of Random Feature Method in the High-Precision Regime  

作　　者：Jingrun Chen Weinan E Yifei Sun 

机构地区：[1]School of Mathematical Sciences and Suzhou Institute for Advanced Research,Suzhou 215006,Jiangsu,China [2]University of Science and Technology of China,Hefei 230026,Anhui,China [3]Center for Machine Learning Research and School of Mathematical Sciences,Peking University,Beijing 100871,China [4]AI for Science Institute,Beijing 100084,China [5]School of Mathematical Sciences,Soochow University,Suzhou 215006,Jiangsu,China

出　　处：《Communications on Applied Mathematics and Computation》2024年第2期1490-1517,共28页应用数学与计算数学学报（英文）

基　　金：supported by the NSFC Major Research Plan--Interpretable and Generalpurpose Next-generation Artificial Intelligence(No.92370205).

摘　　要：Machine learning has been widely used for solving partial differential equations(PDEs)in recent years,among which the random feature method(RFM)exhibits spectral accuracy and can compete with traditional solvers in terms of both accuracy and efficiency.Potentially,the optimization problem in the RFM is more difficult to solve than those that arise in traditional methods.Unlike the broader machine-learning research,which frequently targets tasks within the low-precision regime,our study focuses on the high-precision regime crucial for solving PDEs.In this work,we study this problem from the following aspects:(i)we analyze the coeffcient matrix that arises in the RFM by studying the distribution of singular values;(ii)we investigate whether the continuous training causes the overfitting issue;(ii)we test direct and iterative methods as well as randomized methods for solving the optimization problem.Based on these results,we find that direct methods are superior to other methods if memory is not an issue,while iterative methods typically have low accuracy and can be improved by preconditioning to some extent.

关 键 词：Random feature method(RFM) Partial differential equation(PDE) Least-squares problem Direct method Iterative method 

分 类 号：O24[理学—计算数学] O29[理学—数学]