TianXing:A Linear Complexity Transformer Model with Explicit Attention Decay for Global Weather Forecasting  

作　　者：Shijin YUAN Guansong WANG Bin MU Feifan ZHOU 

机构地区：[1]School of Computer Science and Technology,Tongji University,Shanghai 200092,China [2]Laboratory of Cloud-Precipitation Physics and Severe Storms,Institute of Atmospheric Physics,Chinese Academy of Sciences,Beijing 101408,China

出　　处：《Advances in Atmospheric Sciences》2025年第1期9-25,共17页大气科学进展(英文版)

基　　金：supported in part by the Meteorological Joint Funds of the National Natural Science Foundation of China under Grant U2142211;in part by the National Natural Science Foundation of China under Grant 42075141,42341202;in part by the National Key Research and Development Program of China under Grant 2020YFA0608000;in part by the Shanghai Municipal Science and Technology Major Project(2021SHZDZX0100);the Fundamental Research Funds for the Central Universities。

摘　　要：In this paper,we introduce TianXing,a transformer-based data-driven model designed with physical augmentation for skillful and efficient global weather forecasting.Previous data-driven transformer models such as Pangu-Weather,FengWu,and FuXi have emerged as promising alternatives for numerical weather prediction in weather forecasting.However,these models have been characterized by their substantial computational resource consumption during training and limited incorporation of explicit physical guidance in their modeling frameworks.In contrast,TianXing applies a linear complexity mechanism that ensures proportional scalability with input data size while significantly diminishing GPU resource demands,with only a marginal compromise in accuracy.Furthermore,TianXing proposes an explicit attention decay mechanism in the linear attention derived from physical insights to enhance its forecasting skill.The mechanism can reweight attention based on Earth's spherical distances and learned sparse multivariate coupling relationships,promptingTianXing to prioritize dynamically relevant neighboring features.Finally,to enhance its performance in mediumrange forecasting,TianXing employs a stacked autoregressive forecast algorithm.Validation of the model's architecture is conducted using ERA5 reanalysis data at a 5.625°latitude-longitude resolution,while a high-resolution dataset at 0.25°is utilized for training the actual forecasting model.Notably,the TianXing exhibits excellent performance,particularly in the Z500(geopotential height)and T850(temperature)fields,surpassing previous data-driven models and operational fullresolution models such as NCEP GFS and ECMWF IFS,as evidenced by latitude-weighted RMSE and ACC metrics.Moreover,the TianXing has demonstrated remarkable capabilities in predicting extreme weather events,such as typhoons.

关 键 词：weather forecast deep learning physics augmentation linear attention 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程] P456[自动化与计算机技术—控制科学与工程]