Optimal pre-train/fine-tune strategies for accurate material property predictions  

作　　者：Reshma Devi Keith T.Butler Gopalakrishnan Sai Gautam 

机构地区：[1]Department of Materials Engineering,Indian Institute of Science,Bengaluru,560012 Karnataka,India [2]Department of Chemistry,University College London,London,WC1E 6BT,UK

出　　处：《npj Computational Materials》2024年第1期27-37,共11页计算材料学(英文)

基　　金：G.S.G.and K.T.B.would like to acknowledge financial support from the Royal Society under grant number IES\R3\223036;the United Kingdom Research and Innovation(UKRI)Engineering and Physical Sciences Research Council(EPSRC)under projects EP/Y000552/1 and EP/Y014405/1;G.S.G.acknowledges financial support fromthe Science and Engineering Research Board(SERB)of the Department of Science and Technology,Government of India,under sanction number IPA/2021/000007;We acknowledge the National Supercomputing Mission(NSM)for providing computing resources for‘Param Utkarsh’at CDAC Knowledge Park,Bengaluru.PARAMUtkarsh is implemented by CDAC and supported by the Ministry of Electronics and Information Technology(MeitY)and the Department of Science and Technology(DST),Government of India.Via our membership of the UK’s HEC Materials Chemistry Consortium,which is funded by EPSRC(EP/X035859/1)。

摘　　要：A pathway to overcome limited data availability in materials science is to use the framework of transfer learning,where a pre-trained(PT)machine learning model(on a larger dataset)can be fine-tuned(FT)on a target(smaller)dataset.Wesystematically explore the effectiveness of various PT/FT strategies to learn and predict material properties and create generalizable models by PT on multiple properties(MPT)simultaneously.Specifically,we leverage graph neural networks(GNNs)to PT/FT on seven diverse curated materials datasets,with sizes ranging from 941 to 132,752.Besides identifying optimal PT/FT strategies and hyperparameters,we find our pair-wise PT-FT models to consistently outperformmodels trained from scratch on target datasets.Importantly,ourMPT models outperform pair-wisemodels on several datasets and,more significantly,on a 2Dmaterial band gap dataset that is completely out-of-domain.Finally,we expect our PT/FT and MPT frameworks to accelerate materials design and discovery for various applications.

关 键 词：OVERCOME SIZES property 

分 类 号：G62[文化科学—教育学]