Prospective Role of Foundation Models in Advancing Autonomous Vehicles  

作　　者：Jianhua Wu Bingzhao Gao Jincheng Gao Jianhao Yu Hongqing Chu Qiankun Yu Xun Gong Yi Chang H Eric Tseng Hong Chen Jie Chen 

机构地区：[1]School of Automotive Studies,Tongji University,Shanghai 201804,China [2]Frontiers Science Center for Intelligent Autonomous Systems,Tongji University,Shanghai 201210,China [3]SAIC Intelligent Technology,Shanghai 201805,China [4]College of Artificial Intelligence,Jilin University,Changchun 130012,China [5]Research and Advanced Engineering,Ford Motor Company,Dearborn,MI 48124,USA [6]College of Electronic and Information Engineering,Tongji University,Shanghai 201804,China [7]National Key Laboratory of Autonomous Intelligent Unmanned Systems,Shanghai 201210,China

出　　处：《Research》2025年第1期667-691,共25页研究(英文)

基　　金：supported in part by the National Nature Science Foundation of China(nos.62373289,62273256,and 62088101);the Fundamental Research Funds for the Central Universities.

摘　　要：With the development of artificial intelligence and breakthroughs in deep learning,large-scale foundation models(FMs),such as generative pre-trained transformer(GPT),Sora,etc.,have achieved remarkable results in many fields including natural language processing and computer vision.The application of FMs in autonomous driving holds considerable promise.For example,they can contribute to enhancing scene understanding and reasoning.By pre-training on rich linguistic and visual data,FMs can understand and interpret various elements in a driving scene,and provide cognitive reasoning to give linguistic and action instructions for driving decisions and planning.Furthermore,FMs can augment data based on the understanding of driving scenarios to provide feasible scenes of those rare occurrences in the long tail distribution that are unlikely to be encountered during routine driving and data collection.The enhancement can subsequently lead to improvement in the accuracy and reliability of autonomous driving systems.Another testament to the potential of FMs'applications lies in world models,exemplified by the DREAMER series,which showcases the ability to comprehend physical laws and dynamics.Learning from massive data under the paradigm of self-supervised learning,world models can generate unseen yet plausible driving environments,facilitating the enhancement in the prediction of road users'behaviors and the off-line training of driving strategies.In this paper,we synthesize the applications and future trends of FMs in autonomous driving.By utilizing the powerful capabilities of FMs,we strive to tackle the potential issues stemming from the long-tail distribution in autonomous driving,consequently advancing overall safety in this domain.

关 键 词：autonomous driving computer visionthe artificial intelligence natural language processing deep learninglarge scale autonomous vehicles understand interpr enhancing scene understanding reasoningby 

分 类 号：U463.6[机械工程—车辆工程]