JARVIS-Leaderboard:a large scale benchmark of materials design methods  

作　　者：Kamal Choudhary Daniel Wines Kangming Li Kevin F.Garrity Vishu Gupta Aldo H.Romero Jaron T.Krogel Kayahan Saritas Addis Fuhr Panchapakesan Ganesh Paul R.C.Kent Keqiang Yan Yuchao Lin Shuiwang Ji Ben Blaiszik Patrick Reiser Pascal Friederich Ankit Agrawal Pratyush Tiwary Eric Beyerle Peter Minch Trevor David Rhone Ichiro Takeuchi Robert B.Wexler Arun Mannodi-Kanakkithodi Elif Ertekin Avanish Mishra Nithin Mathew Mitchell Wood Andrew Dale Rohskopf Jason Hattrick-Simpers Shih-Han Wang Luke E.K.Achenie Hongliang Xin Maureen Williams Adam J.Biacchi Francesca Tavazza 

机构地区：[1]Material Measurement Laboratory,National Institute of Standards and Technology,Gaithersburg,MD20899,USA [2]Department of Materials Science and Engineering,University of Toronto,27 King’s College Cir,Toronto,ON,Canada [3]Department of Electrical and Computer Engineering,Northwestern University,Evanston,IL 60208,USA [4]Lewis-Sigler Institute for Integrative Genomics,Princeton University,Princeton,NJ 08544,USA [5]Ludwig Institute for Cancer Research,Princeton University,Princeton,NJ 08544,USA [6]Department of Physics and Astronomy,West Virginia University,Morgantown,WV 26506,USA [7]Materials Science and Technology Division,Oak Ridge National Laboratory,Oak Ridge,TN 37831,USA [8]Center for Nanophase Materials Science,Oak Ridge National Laboratory,Oak Ridge,TN 37831,USA [9]Computational Sciences and Engineering Division,Oak Ridge National Laboratory,Oak Ridge,TN 37831,USA [10]Department of Computer Science and Engineering,Texas A&M University,College Station,TX 77843,USA [11]Globus,University of Chicago,Chicago,IL 60637,USA [12]Data Science and Learning Division,Argonne National Lab,Lemont,IL 60439,USA [13]Institute of Nanotechnology,Karlsruhe Institute of Technology,Kaiserstraße 12,76131 Karlsruhe,Germany [14]Institute of Theoretical Informatics,Karlsruhe Institute of Technology,Kaiserstraße 12,76131 Karlsruhe,Germany [15]Department of Chemistry and Biochemistry and Institute for Physical Science and Technology,University of Maryland,College Park,MD 20742,USA [16]Department of Physics,Applied Physics and Astronomy,Rensselaer Polytechnic Institute,Troy,NY12180,USA [17]Department of Materials Science and Engineering,University of Maryland,College Park,MD20742,USA [18]Department of Chemistry and Institute of Materials Science and Engineering,Washington University in St.Louis,St.Louis,MO63130,USA [19]School of Materials Engineering,Purdue University,West Lafayette,IN47907,USA [20]Department of Mechanical Science and Engineering,University of Illinois Urbana-Champaign,Urbana,Illinois 61801,USA [21]Materials Researc

出　　处：《npj Computational Materials》2024年第1期2280-2296,共17页计算材料学(英文)

基　　金：supported by the financial assistance award 70NANB19H117 from the U.S.Department of Commerce,National Institute of Standards and Technology;supported by the U.S.Department of Energy,Office of Science,Basic Energy Sciences,Materials Sciences and Engineering Division,as part of the Computational Materials Sciences Program and Center for Predictive Simulation of Functional Materials;supported by the Center for Nanophase Materials Sciences,which is a US Department of Energy,Office of Science User Facility at Oak Ridge National Laboratory;AHR thanks the Supercomputer Center and San Diego Supercomputer Center through allocation DMR140031 from the Advanced Cyberinfrastructure Coordination Ecosystem:Services&Support(ACCESS)program,which is supported by National Science Foundation grants#2138259,#2138286,#2138307,#2137603,and#2138296;supported by NIST award 70NANB19H005 and NSF award CMMI-2053929;S.H.W.especially thanks to the NSF Non-Academic Research Internships for Graduate Students(INTERN)program(CBET-1845531)for supporting part of the work in NIST under the guidance of K.C;A.M.K.acknowledges support from the School of Materials Engineering at Purdue University under startup account F.10023800.05.002;support by the Federal Ministry of Education and Research(BMBF)under Grant No.01DM21001B(German-Canadian Materials Acceleration Center).

摘　　要：Lack of rigorous reproducibility and validation are significant hurdles for scientific development across many fields.Materials science,in particular,encompasses a variety of experimental and theoretical approaches that require careful benchmarking.Leaderboard efforts have been developed previously to mitigate these issues.However,a comprehensive comparison and benchmarking on an integrated platform with multiple data modalities with perfect and defect materials data is still lacking.This work introduces JARVIS-Leaderboard,an open-source and community-driven platform that facilitates benchmarking and enhances reproducibility.The platform allows users to set up benchmarks with customtasks and enables contributions in the form of dataset,code,and meta-data submissions.We cover the following materials design categories:Artificial Intelligence(AI),Electronic Structure(ES).

关 键 词：rigorous PERFECT enable 

分 类 号：O57[理学—粒子物理与原子核物理]